Sustainable Kibana Architecture: Move modules owned by @elastic/kibana-esql (#202722)

## Summary

This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.

> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
>     * customised eslint rules
>     * docs pointing to source code

> [!NOTE]
> This PR has been auto-generated.
> Do not attempt to push any changes unless you know what you are doing.
> Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.




#### 2 plugin(s) are going to be relocated:

| Id | Target folder |
| -- | ------------- |
| `@kbn/esql` | `src/platform/plugins/shared/esql` |
| `@kbn/esql-datagrid` | `src/platform/plugins/shared/esql_datagrid` |


#### 5 package(s) are going to be relocated:

  | Id | Target folder |
  | -- | ------------- |
  | `@kbn/esql-ast` | `src/platform/packages/shared/kbn-esql-ast` |
| `@kbn/esql-editor` | `src/platform/packages/private/kbn-esql-editor` |
| `@kbn/esql-utils` | `src/platform/packages/shared/kbn-esql-utils` |
| `@kbn/esql-validation-autocomplete` |
`src/platform/packages/shared/kbn-esql-validation-autocomplete` |
| `@kbn/language-documentation` |
`src/platform/packages/private/kbn-language-documentation` |


<details>
<summary>Updated references</summary>

```
./.buildkite/scripts/steps/esql_generate_function_metadata.sh
./.buildkite/scripts/steps/esql_grammar_sync.sh
./.eslintignore
./.eslintrc.js
./.i18nrc.json
./docs/developer/plugin-list.asciidoc
./examples/esql_ast_inspector/public/plugin.ts
./examples/esql_validation_example/public/plugin.ts
./package.json
./packages/kbn-esql-validation-autocomplete/src/validation/validation.test.ts
./packages/kbn-monaco/BUILD.bazel
./packages/kbn-repo-packages/package-map.json
./packages/kbn-ts-projects/config-paths.json
./packages/kbn-ui-shared-deps-src/BUILD.bazel
./src/dev/storybook/aliases.ts
./src/platform/packages/private/kbn-esql-editor/jest.config.js
./src/platform/packages/private/kbn-language-documentation/jest.config.js
./src/platform/packages/private/kbn-language-documentation/package.json
./src/platform/packages/shared/kbn-esql-ast/jest.config.js
./src/platform/packages/shared/kbn-esql-utils/jest.config.js
./src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md
./src/platform/packages/shared/kbn-esql-validation-autocomplete/jest.config.js
./src/platform/packages/shared/kbn-esql-validation-autocomplete/jest.integration.config.js
./src/platform/packages/shared/kbn-esql-validation-autocomplete/package.json
./src/platform/packages/shared/kbn-esql-validation-autocomplete/src/autocomplete/helper.ts
./src/platform/plugins/shared/esql/jest.config.js
./src/platform/plugins/shared/esql_datagrid/jest.config.js
./tsconfig.base.json
./yarn.lock
```
</details>
<details>
<summary>Updated relative paths</summary>

```
src/platform/packages/private/kbn-esql-editor/jest.config.js:12
src/platform/packages/private/kbn-esql-editor/tsconfig.json:2
src/platform/packages/private/kbn-language-documentation/jest.config.js:12
src/platform/packages/private/kbn-language-documentation/package.json:12
src/platform/packages/private/kbn-language-documentation/tsconfig.json:2
src/platform/packages/shared/kbn-esql-ast/jest.config.js:12
src/platform/packages/shared/kbn-esql-ast/tsconfig.json:2
src/platform/packages/shared/kbn-esql-utils/jest.config.js:12
src/platform/packages/shared/kbn-esql-utils/tsconfig.json:2
src/platform/packages/shared/kbn-esql-validation-autocomplete/jest.config.js:12
src/platform/packages/shared/kbn-esql-validation-autocomplete/jest.integration.config.js:12
src/platform/packages/shared/kbn-esql-validation-autocomplete/package.json:10
src/platform/packages/shared/kbn-esql-validation-autocomplete/package.json:11
src/platform/packages/shared/kbn-esql-validation-autocomplete/scripts/generate_function_definitions.ts:389
src/platform/packages/shared/kbn-esql-validation-autocomplete/tsconfig.json:2
src/platform/plugins/shared/esql/jest.config.js:12
src/platform/plugins/shared/esql/tsconfig.json:2
src/platform/plugins/shared/esql/tsconfig.json:7
src/platform/plugins/shared/esql_datagrid/jest.config.js:12
src/platform/plugins/shared/esql_datagrid/tsconfig.json:2
src/platform/plugins/shared/esql_datagrid/tsconfig.json:7
```
</details>
<details>
<summary>Script errors</summary>

```

```
</details>

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Rudolf Meijering <skaapgif@gmail.com>

.buildkite/scripts/steps/esql_generate_function_metadata.sh

@@ -1,8 +1,8 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
-VALIDATION_PACKAGE_DIR="packages/kbn-esql-validation-autocomplete"
-EDITOR_PACKAGE_DIR="packages/kbn-language-documentation"
+VALIDATION_PACKAGE_DIR="src/platform/packages/shared/kbn-esql-validation-autocomplete"
+EDITOR_PACKAGE_DIR="src/platform/packages/private/kbn-language-documentation"
 GIT_SCOPE="$VALIDATION_PACKAGE_DIR/**/* $EDITOR_PACKAGE_DIR/**/*"
 
 report_main_step () {

.buildkite/scripts/steps/esql_grammar_sync.sh

@@ -4,7 +4,7 @@ set -euo pipefail
 synchronize_lexer_grammar () {
   license_header="$1"
   source_file="$PARENT_DIR/elasticsearch/x-pack/plugin/esql/src/main/antlr/EsqlBaseLexer.g4"
-  destination_file="./packages/kbn-esql-ast/src/antlr/esql_lexer.g4"
+  destination_file="./src/platform/packages/shared/kbn-esql-ast/src/antlr/esql_lexer.g4"
 
   # Copy the file
   cp "$source_file" "$destination_file"
@@ -26,7 +26,7 @@ synchronize_lexer_grammar () {
 synchronize_parser_grammar () {
   license_header="$1"
   source_file="$PARENT_DIR/elasticsearch/x-pack/plugin/esql/src/main/antlr/EsqlBaseParser.g4"
-  destination_file="./packages/kbn-esql-ast/src/antlr/esql_parser.g4"
+  destination_file="./src/platform/packages/shared/kbn-esql-ast/src/antlr/esql_parser.g4"
 
   # Copy the file
   cp "$source_file" "$destination_file"
@@ -104,7 +104,7 @@ main () {
   .buildkite/scripts/bootstrap.sh
 
   # Build ANTLR stuff
-  cd ./packages/kbn-esql-ast/src
+  cd ./src/platform/packages/shared/kbn-esql-ast/src
   yarn build:antlr4:esql
 
   # Make a commit

.eslintignore

@@ -39,7 +39,7 @@ snapshots.js
 /packages/kbn-ui-framework/dist
 /packages/kbn-flot-charts/lib
 /packages/kbn-monaco/src/**/antlr
-/packages/kbn-esql-ast/src/**/antlr
+/src/platform/packages/shared/kbn-esql-ast/src/**/antlr
 
 # Bazel
 /bazel-*

.eslintrc.js

@@ -189,7 +189,7 @@ const DEV_PATTERNS = [
   'x-pack/performance/**/*',
   'src/setup_node_env/index.js',
   'src/cli/dev.js',
-  'packages/kbn-esql-validation-autocomplete/scripts/**/*',
+  'src/platform/packages/shared/kbn-esql-validation-autocomplete/scripts/**/*',
 ];
 
 /** Restricted imports with suggested alternatives */

.github/CODEOWNERS

@@ -352,10 +352,6 @@ packages/kbn-eslint-plugin-eslint @elastic/kibana-operations
 packages/kbn-eslint-plugin-i18n @elastic/obs-knowledge-team @elastic/kibana-operations
 packages/kbn-eslint-plugin-imports @elastic/kibana-operations
 packages/kbn-eslint-plugin-telemetry @elastic/obs-knowledge-team
-packages/kbn-esql-ast @elastic/kibana-esql
-packages/kbn-esql-editor @elastic/kibana-esql
-packages/kbn-esql-utils @elastic/kibana-esql
-packages/kbn-esql-validation-autocomplete @elastic/kibana-esql
 packages/kbn-event-annotation-common @elastic/kibana-visualizations
 packages/kbn-event-annotation-components @elastic/kibana-visualizations
 packages/kbn-expandable-flyout @elastic/security-threat-hunting-investigations
@@ -394,7 +390,6 @@ packages/kbn-jest-serializers @elastic/kibana-operations
 packages/kbn-journeys @elastic/kibana-operations @elastic/appex-qa
 packages/kbn-json-ast @elastic/kibana-operations
 packages/kbn-kibana-manifest-schema @elastic/kibana-operations
-packages/kbn-language-documentation @elastic/kibana-esql
 packages/kbn-lens-embeddable-utils @elastic/obs-ux-infra_services-team @elastic/kibana-visualizations
 packages/kbn-lens-formula-docs @elastic/kibana-visualizations
 packages/kbn-lint-packages-cli @elastic/kibana-operations
@@ -622,7 +617,14 @@ packages/shared-ux/storybook/config @elastic/appex-sharedux
 packages/shared-ux/storybook/mock @elastic/appex-sharedux
 packages/shared-ux/table_persist @elastic/appex-sharedux
 src/core @elastic/kibana-core
+src/platform/packages/private/kbn-esql-editor @elastic/kibana-esql
+src/platform/packages/private/kbn-language-documentation @elastic/kibana-esql
 src/platform/packages/shared/kbn-doc-links @elastic/docs
+src/platform/packages/shared/kbn-esql-ast @elastic/kibana-esql
+src/platform/packages/shared/kbn-esql-utils @elastic/kibana-esql
+src/platform/packages/shared/kbn-esql-validation-autocomplete @elastic/kibana-esql
+src/platform/plugins/shared/esql @elastic/kibana-esql
+src/platform/plugins/shared/esql_datagrid @elastic/kibana-esql
 src/plugins/advanced_settings @elastic/appex-sharedux @elastic/kibana-management
 src/plugins/ai_assistant_management/selection @elastic/obs-ai-assistant
 src/plugins/bfetch @elastic/appex-sharedux
@@ -650,8 +652,6 @@ src/plugins/discover @elastic/kibana-data-discovery
 src/plugins/discover_shared @elastic/kibana-data-discovery @elastic/obs-ux-logs-team
 src/plugins/embeddable @elastic/kibana-presentation
 src/plugins/es_ui_shared @elastic/kibana-management
-src/plugins/esql @elastic/kibana-esql
-src/plugins/esql_datagrid @elastic/kibana-esql
 src/plugins/event_annotation @elastic/kibana-visualizations
 src/plugins/event_annotation_listing @elastic/kibana-visualizations
 src/plugins/expression_error @elastic/kibana-presentation

.i18nrc.json

@@ -76,7 +76,7 @@
     "interpreter": "src/legacy/core_plugins/interpreter",
     "imageEmbeddable": "src/plugins/image_embeddable",
     "kbn": "src/legacy/core_plugins/kibana",
-    "kbn-esql-validation-autocomplete": "packages/kbn-esql-validation-autocomplete/src",
+    "kbn-esql-validation-autocomplete": "src/platform/packages/shared/kbn-esql-validation-autocomplete/src",
     "kbnConfig": "packages/kbn-config/src",
     "kbnDocViews": "src/legacy/core_plugins/kbn_doc_views",
     "kibana_react": "src/legacy/core_plugins/kibana_react",
@@ -98,8 +98,8 @@
     "presentationUtil": "src/plugins/presentation_util",
     "randomSampling": "x-pack/packages/kbn-random-sampling",
     "reactPackages": "packages/react",
-    "esqlEditor": "packages/kbn-esql-editor",
-    "esqlUtils": "packages/kbn-esql-utils",
+    "esqlEditor": "src/platform/packages/private/kbn-esql-editor",
+    "esqlUtils": "src/platform/packages/shared/kbn-esql-utils",
     "reporting": "packages/kbn-reporting",
     "savedObjects": "src/plugins/saved_objects",
     "savedObjectsFinder": "src/plugins/saved_objects_finder",
@@ -119,9 +119,9 @@
     "serverlessPackages": "packages/serverless",
     "sse": ["packages/kbn-sse-utils"],
     "coloring": "packages/kbn-coloring/src",
-    "languageDocumentation": "packages/kbn-language-documentation/src",
-    "esql": "src/plugins/esql",
-    "esqlDataGrid": "src/plugins/esql_datagrid",
+    "languageDocumentation": "src/platform/packages/private/kbn-language-documentation/src",
+    "esql": "src/platform/plugins/shared/esql",
+    "esqlDataGrid": "src/platform/plugins/shared/esql_datagrid",
     "statusPage": "src/legacy/core_plugins/status_page",
     "telemetry": ["src/plugins/telemetry", "src/plugins/telemetry_management_section"],
     "timelion": ["src/plugins/vis_types/timelion"],

docs/developer/plugin-list.asciidoc

@@ -102,11 +102,11 @@ This API doesn't support angular, for registering angular dev tools, bootstrap a
 |Embeddables are React components that manage their own state, can be serialized and deserialized, and return an API that can be used to interact with them imperatively.
 
 
-|{kib-repo}blob/{branch}/src/plugins/esql/README.md[esql]
+|{kib-repo}blob/{branch}/src/platform/plugins/shared/esql/README.md[esql]
 |The editor accepts the following properties:
 
 
-|{kib-repo}blob/{branch}/src/plugins/esql_datagrid/README.md[esqlDataGrid]
+|{kib-repo}blob/{branch}/src/platform/plugins/shared/esql_datagrid/README.md[esqlDataGrid]
 |Contains a Discover-like table specifically for ES|QL queries:
 
 

examples/esql_ast_inspector/public/plugin.ts

@@ -42,7 +42,7 @@ export class ESQLASTInspectorPlugin
       links: [
         {
           label: 'README',
-          href: 'https://github.com/elastic/kibana/tree/main/packages/kbn-esql-validation-autocomplete/README.md',
+          href: 'https://github.com/elastic/kibana/tree/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md',
           iconType: 'logoGithub',
           size: 's',
           target: '_blank',

examples/esql_validation_example/public/plugin.ts

@@ -42,7 +42,7 @@ export class ESQLValidationExamplePlugin
       links: [
         {
           label: 'README',
-          href: 'https://github.com/elastic/kibana/tree/main/packages/kbn-esql-validation-autocomplete/README.md',
+          href: 'https://github.com/elastic/kibana/tree/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md',
           iconType: 'logoGithub',
           size: 's',
           target: '_blank',

package.json

@@ -489,13 +489,13 @@
     "@kbn/es-ui-shared-plugin": "link:src/plugins/es_ui_shared",
     "@kbn/eso-model-version-example": "link:examples/eso_model_version_example",
     "@kbn/eso-plugin": "link:x-pack/test/encrypted_saved_objects_api_integration/plugins/api_consumer_plugin",
-    "@kbn/esql": "link:src/plugins/esql",
-    "@kbn/esql-ast": "link:packages/kbn-esql-ast",
+    "@kbn/esql": "link:src/platform/plugins/shared/esql",
+    "@kbn/esql-ast": "link:src/platform/packages/shared/kbn-esql-ast",
     "@kbn/esql-ast-inspector-plugin": "link:examples/esql_ast_inspector",
-    "@kbn/esql-datagrid": "link:src/plugins/esql_datagrid",
-    "@kbn/esql-editor": "link:packages/kbn-esql-editor",
-    "@kbn/esql-utils": "link:packages/kbn-esql-utils",
-    "@kbn/esql-validation-autocomplete": "link:packages/kbn-esql-validation-autocomplete",
+    "@kbn/esql-datagrid": "link:src/platform/plugins/shared/esql_datagrid",
+    "@kbn/esql-editor": "link:src/platform/packages/private/kbn-esql-editor",
+    "@kbn/esql-utils": "link:src/platform/packages/shared/kbn-esql-utils",
+    "@kbn/esql-validation-autocomplete": "link:src/platform/packages/shared/kbn-esql-validation-autocomplete",
     "@kbn/esql-validation-example-plugin": "link:examples/esql_validation_example",
     "@kbn/eui-provider-dev-warning": "link:test/plugin_functional/plugins/eui_provider_dev_warning",
     "@kbn/event-annotation-common": "link:packages/kbn-event-annotation-common",
@@ -609,7 +609,7 @@
     "@kbn/kibana-utils-plugin": "link:src/plugins/kibana_utils",
     "@kbn/kubernetes-security-plugin": "link:x-pack/plugins/kubernetes_security",
     "@kbn/langchain": "link:x-pack/packages/kbn-langchain",
-    "@kbn/language-documentation": "link:packages/kbn-language-documentation",
+    "@kbn/language-documentation": "link:src/platform/packages/private/kbn-language-documentation",
     "@kbn/lens-config-builder-example-plugin": "link:x-pack/examples/lens_config_builder_example",
     "@kbn/lens-embeddable-utils": "link:packages/kbn-lens-embeddable-utils",
     "@kbn/lens-formula-docs": "link:packages/kbn-lens-formula-docs",

packages/kbn-monaco/BUILD.bazel

@@ -26,8 +26,8 @@ SRCS = glob(
 SHARED_DEPS = [
   "//packages/kbn-i18n",
   "//packages/kbn-ui-theme",
-  "//packages/kbn-esql-validation-autocomplete",
-  "//packages/kbn-esql-ast",
+  "//src/platform/packages/shared/kbn-esql-validation-autocomplete",
+  "//src/platform/packages/shared/kbn-esql-ast",
   "@npm//antlr4",
   "@npm//monaco-editor",
   "@npm//monaco-yaml",

packages/kbn-ui-shared-deps-src/BUILD.bazel

@@ -25,7 +25,7 @@ webpack_cli(
     "//packages/kbn-i18n",
     "//packages/kbn-i18n-react",
     "//packages/kbn-ebt-tools",
-    "//packages/kbn-esql-ast",
+    "//src/platform/packages/shared/kbn-esql-ast",
     "//packages/kbn-monaco",
     "//packages/kbn-datemath",
     "//packages/kbn-analytics",

src/dev/storybook/aliases.ts

@@ -19,7 +19,8 @@ export const storybookAliases = {
   cloud_security_posture_packages: 'x-pack/packages/kbn-cloud-security-posture/storybook/config',
   cloud: 'packages/cloud/.storybook',
   coloring: 'packages/kbn-coloring/.storybook',
-  language_documentation_popover: 'packages/kbn-language-documentation/.storybook',
+  language_documentation_popover:
+    'src/platform/packages/private/kbn-language-documentation/.storybook',
   chart_icons: 'packages/kbn-chart-icons/.storybook',
   content_management_examples: 'examples/content_management_examples/.storybook',
   custom_icons: 'packages/kbn-custom-icons/.storybook',
@@ -61,7 +62,7 @@ export const storybookAliases = {
   observability_slo: 'x-pack/plugins/observability_solution/slo/.storybook',
   presentation: 'src/plugins/presentation_util/storybook',
   random_sampling: 'x-pack/packages/kbn-random-sampling/.storybook',
-  esql_editor: 'packages/kbn-esql-editor/.storybook',
+  esql_editor: 'src/platform/packages/private/kbn-esql-editor/.storybook',
   security_solution: 'x-pack/plugins/security_solution/.storybook',
   security_solution_packages: 'x-pack/packages/security-solution/storybook/config',
   serverless: 'packages/serverless/storybook/config',

src/platform/packages/private/kbn-esql-editor/jest.config.js

@@ -9,7 +9,7 @@
 
 module.exports = {
   preset: '@kbn/test',
-  rootDir: '../..',
-  roots: ['<rootDir>/packages/kbn-esql-editor'],
-  setupFilesAfterEnv: ['<rootDir>/packages/kbn-esql-editor/setup_tests.ts'],
+  rootDir: '../../../../..',
+  roots: ['<rootDir>/src/platform/packages/private/kbn-esql-editor'],
+  setupFilesAfterEnv: ['<rootDir>/src/platform/packages/private/kbn-esql-editor/setup_tests.ts'],
 };

src/platform/packages/private/kbn-esql-editor/tsconfig.json

@@ -1,5 +1,5 @@
 {
-  "extends": "../../tsconfig.base.json",
+  "extends": "../../../../../tsconfig.base.json",
   "compilerOptions": {
     "outDir": "target/types",
     "types": [

src/platform/packages/private/kbn-language-documentation/jest.config.js

@@ -9,6 +9,6 @@
 
 module.exports = {
   preset: '@kbn/test',
-  rootDir: '../..',
-  roots: ['<rootDir>/packages/kbn-esql-utils'],
+  rootDir: '../../../../..',
+  roots: ['<rootDir>/src/platform/packages/private/kbn-language-documentation'],
 };

src/platform/packages/private/kbn-language-documentation/package.json

@@ -9,6 +9,6 @@
   "scripts": {
     "make:docs": "ts-node --transpileOnly scripts/generate_esql_docs.ts",
     "postmake:docs": "yarn run lint:fix",
-    "lint:fix": "cd ../.. && node ./scripts/eslint --fix ./packages/kbn-language-documentation/src/sections/generated"
+    "lint:fix": "cd ../../../../.. && node ./scripts/eslint --fix ./src/platform/packages/private/kbn-language-documentation/src/sections/generated"
   }
 }

src/platform/packages/private/kbn-language-documentation/tsconfig.json

@@ -1,5 +1,5 @@
 {
-  "extends": "../../tsconfig.base.json",
+  "extends": "../../../../../tsconfig.base.json",
   "compilerOptions": {
     "outDir": "target/types",
     "types": [

src/platform/packages/shared/kbn-esql-ast/jest.config.js

@@ -9,6 +9,6 @@
 
 module.exports = {
   preset: '@kbn/test',
-  rootDir: '../..',
-  roots: ['<rootDir>/packages/kbn-esql-validation-autocomplete'],
+  rootDir: '../../../../..',
+  roots: ['<rootDir>/src/platform/packages/shared/kbn-esql-ast'],
 };