# Backport
This will backport the following commits from `main` to `7.17`:
- [[ci] Add node scripts/yarn_deduplicate check
(main)(#169165)](https://github.com/elastic/kibana/pull/169165)
---------
Co-authored-by: Jon <jon@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
# Backport
This will backport the following commits from `main` to `7.17`:
- [chore(NA): remove usage of re2 and replace it with a non native
module (#188134)](https://github.com/elastic/kibana/pull/188134)
<!--- Backport version: 8.9.8 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Tiago
Costa","email":"tiago.costa@elastic.co"},"sourceCommit":{"committedDate":"2024-07-15T19:33:28Z","message":"chore(NA):
remove usage of re2 and replace it with a non native module
(#188134)\n\nThis PR removes the usage of the native module version of
`re2` and\r\nreplaces it with a js port called `re2js`.\r\nIt also ends
our usage of native node modules in production and it\r\nremoves the
task from the build as well. Further steps will be taken\r\nalong our
strategy to avoid future usages of native node modules in
prod\r\nenvironments.","sha":"3c338a82c2d6d59c91f0f3b74f725231c69c5b13","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["chore","Team:Operations","release_note:skip","backport:all-open","v8.16.0"],"number":188134,"url":"https://github.com/elastic/kibana/pull/188134","mergeCommit":{"message":"chore(NA):
remove usage of re2 and replace it with a non native module
(#188134)\n\nThis PR removes the usage of the native module version of
`re2` and\r\nreplaces it with a js port called `re2js`.\r\nIt also ends
our usage of native node modules in production and it\r\nremoves the
task from the build as well. Further steps will be taken\r\nalong our
strategy to avoid future usages of native node modules in
prod\r\nenvironments.","sha":"3c338a82c2d6d59c91f0f3b74f725231c69c5b13"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/188134","number":188134,"mergeCommit":{"message":"chore(NA):
remove usage of re2 and replace it with a non native module
(#188134)\n\nThis PR removes the usage of the native module version of
`re2` and\r\nreplaces it with a js port called `re2js`.\r\nIt also ends
our usage of native node modules in production and it\r\nremoves the
task from the build as well. Further steps will be taken\r\nalong our
strategy to avoid future usages of native node modules in
prod\r\nenvironments.","sha":"3c338a82c2d6d59c91f0f3b74f725231c69c5b13"}}]}]
BACKPORT-->
Closes https://github.com/elastic/kibana/issues/162695
# Backport
This will backport the following commits from `main` to `7.17`:
- [[Ops] Bump Node.js to version 18
(#160289)](https://github.com/elastic/kibana/pull/160289)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Alex
Szabo","email":"alex.szabo@elastic.co"},"sourceCommit":{"committedDate":"2023-07-27T12:12:48Z","message":"[Ops]
Bump Node.js to version 18 (#160289)\n\n## Summary\r\n\r\nBumps node.js
to 18.17.0 (replacement for PR #144012 which was
later\r\nreverted)\r\n\r\nAs a result, these categorical additions were
needed: \r\n- `node` evocations will need the
`--openssl-legacy-provider` flag,\r\nwherever it would use certain
crypto functionalities\r\n- tests required updating of the expected
HTTPS Agent call arguments,\r\n`noDelay` seems to be a default\r\n -
`window.[NAME]` fields cannot be written directly\r\n - some stricter
typechecks\r\n\r\nThis is using our in-house built node.js 18 versions
through the URLs\r\nthe proxy-cache. (built
with\r\nhttps://github.com/elastic/kibana-custom-nodejs-builds/pull/4)\r\n\r\nThese
urls are served from a bucket, where the RHEL7/Centos7
compatible\r\nnode distributables are.
(see:\r\nhttps://github.com/elastic/kibana-ci-proxy-cache/pull/7)\r\n\r\nFurther
todos: \r\n - [x] check docs wording and consistency\r\n - [ ] update
the dependency report\r\n - [x] explain custom builds in
documentation\r\n - [x] node_sass
prebuilts\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Tiago Costa <tiago.costa@elastic.co>\r\nCo-authored-by: Thomas Watson
<w@tson.dk>","sha":"8cf68dc6ba8f010e36538c1e7c92601a341efcf4","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","Team:uptime","backport:skip","release_note:feature","ci:all-cypress-suites","v8.10.0"],"number":160289,"url":"https://github.com/elastic/kibana/pull/160289","mergeCommit":{"message":"[Ops]
Bump Node.js to version 18 (#160289)\n\n## Summary\r\n\r\nBumps node.js
to 18.17.0 (replacement for PR #144012 which was
later\r\nreverted)\r\n\r\nAs a result, these categorical additions were
needed: \r\n- `node` evocations will need the
`--openssl-legacy-provider` flag,\r\nwherever it would use certain
crypto functionalities\r\n- tests required updating of the expected
HTTPS Agent call arguments,\r\n`noDelay` seems to be a default\r\n -
`window.[NAME]` fields cannot be written directly\r\n - some stricter
typechecks\r\n\r\nThis is using our in-house built node.js 18 versions
through the URLs\r\nthe proxy-cache. (built
with\r\nhttps://github.com/elastic/kibana-custom-nodejs-builds/pull/4)\r\n\r\nThese
urls are served from a bucket, where the RHEL7/Centos7
compatible\r\nnode distributables are.
(see:\r\nhttps://github.com/elastic/kibana-ci-proxy-cache/pull/7)\r\n\r\nFurther
todos: \r\n - [x] check docs wording and consistency\r\n - [ ] update
the dependency report\r\n - [x] explain custom builds in
documentation\r\n - [x] node_sass
prebuilts\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Tiago Costa <tiago.costa@elastic.co>\r\nCo-authored-by: Thomas Watson
<w@tson.dk>","sha":"8cf68dc6ba8f010e36538c1e7c92601a341efcf4"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/160289","number":160289,"mergeCommit":{"message":"[Ops]
Bump Node.js to version 18 (#160289)\n\n## Summary\r\n\r\nBumps node.js
to 18.17.0 (replacement for PR #144012 which was
later\r\nreverted)\r\n\r\nAs a result, these categorical additions were
needed: \r\n- `node` evocations will need the
`--openssl-legacy-provider` flag,\r\nwherever it would use certain
crypto functionalities\r\n- tests required updating of the expected
HTTPS Agent call arguments,\r\n`noDelay` seems to be a default\r\n -
`window.[NAME]` fields cannot be written directly\r\n - some stricter
typechecks\r\n\r\nThis is using our in-house built node.js 18 versions
through the URLs\r\nthe proxy-cache. (built
with\r\nhttps://github.com/elastic/kibana-custom-nodejs-builds/pull/4)\r\n\r\nThese
urls are served from a bucket, where the RHEL7/Centos7
compatible\r\nnode distributables are.
(see:\r\nhttps://github.com/elastic/kibana-ci-proxy-cache/pull/7)\r\n\r\nFurther
todos: \r\n - [x] check docs wording and consistency\r\n - [ ] update
the dependency report\r\n - [x] explain custom builds in
documentation\r\n - [x] node_sass
prebuilts\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by:
Tiago Costa <tiago.costa@elastic.co>\r\nCo-authored-by: Thomas Watson
<w@tson.dk>","sha":"8cf68dc6ba8f010e36538c1e7c92601a341efcf4"}}]}]
BACKPORT-->
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Thomas Watson <w@tson.dk>
Co-authored-by: Jonathan Budzenski <jon@elastic.co>
# Backport
This will backport the following commits from `main` to `7.17`:
- [chore(NA): updates from lmdb-store to lmdb
(#145891)](https://github.com/elastic/kibana/pull/145891)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Tiago
Costa","email":"tiago.costa@elastic.co"},"sourceCommit":{"committedDate":"2022-11-28T23:48:15Z","message":"chore(NA):
updates from lmdb-store to lmdb (#145891)\n\nThis PR upgrades from
`lbmd-store` into `lmdb` which is the new package\r\nand fully
compatible with node `v18`.\r\nSo far my tests shows the new
implementation is compatible with our\r\nusages and I'm actually seeing
a great performance boost when comparing\r\nwith main specially on
subsequent calls of the same command.\r\n\r\nThis can be tested by
running the following 2 times on main vs this\r\nbranch `time node
scripts/kibana
--config\r\nsrc/cli/serve/integration_tests/__fixtures__/invalid_config.yml\r\n--migrations.skip=true
--verbose`\r\n\r\nI verify the following on my
machine\r\n\r\n**main:**\r\n_1st run:_ 36s\r\n_2nd run:_
34s\r\n\r\n**this branch:**\r\n_1st run:_ 35s\r\n_2nd run:_
6s\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"e65c6526107670689c520bc20c973626c12c636d","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["chore","Team:Operations","release_note:skip","backport:all-open","v8.7.0"],"number":145891,"url":"https://github.com/elastic/kibana/pull/145891","mergeCommit":{"message":"chore(NA):
updates from lmdb-store to lmdb (#145891)\n\nThis PR upgrades from
`lbmd-store` into `lmdb` which is the new package\r\nand fully
compatible with node `v18`.\r\nSo far my tests shows the new
implementation is compatible with our\r\nusages and I'm actually seeing
a great performance boost when comparing\r\nwith main specially on
subsequent calls of the same command.\r\n\r\nThis can be tested by
running the following 2 times on main vs this\r\nbranch `time node
scripts/kibana
--config\r\nsrc/cli/serve/integration_tests/__fixtures__/invalid_config.yml\r\n--migrations.skip=true
--verbose`\r\n\r\nI verify the following on my
machine\r\n\r\n**main:**\r\n_1st run:_ 36s\r\n_2nd run:_
34s\r\n\r\n**this branch:**\r\n_1st run:_ 35s\r\n_2nd run:_
6s\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"e65c6526107670689c520bc20c973626c12c636d"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/145891","number":145891,"mergeCommit":{"message":"chore(NA):
updates from lmdb-store to lmdb (#145891)\n\nThis PR upgrades from
`lbmd-store` into `lmdb` which is the new package\r\nand fully
compatible with node `v18`.\r\nSo far my tests shows the new
implementation is compatible with our\r\nusages and I'm actually seeing
a great performance boost when comparing\r\nwith main specially on
subsequent calls of the same command.\r\n\r\nThis can be tested by
running the following 2 times on main vs this\r\nbranch `time node
scripts/kibana
--config\r\nsrc/cli/serve/integration_tests/__fixtures__/invalid_config.yml\r\n--migrations.skip=true
--verbose`\r\n\r\nI verify the following on my
machine\r\n\r\n**main:**\r\n_1st run:_ 36s\r\n_2nd run:_
34s\r\n\r\n**this branch:**\r\n_1st run:_ 35s\r\n_2nd run:_
6s\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"e65c6526107670689c520bc20c973626c12c636d"}}]}]
BACKPORT-->
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [docs] rewrite docs cli to show logs and use modern apis (#117767)
Co-authored-by: spalger <spalger@users.noreply.github.com>
# Conflicts:
# src/docs/docs_repo.js
* commit with @elastic.co
* remove unnecessary ts-ignore
* add context propagation to x-opaque-id header tests
* run tests on CI
* simplify logging. the action purpose follows from the context name
* extend tests with the assertion against execution_context from the Kibana logs
* split JSON log records only
* apply suggestions proposed by Spencer
# Conflicts:
# scripts/functional_tests.js
* [build_ts_refs] improve caches, allow building a subset of projects
* cleanup project def script and update refs in type check script
* rename browser_bazel config to avoid kebab-case
* remove execInProjects() helper
* list references for tsconfig.types.json for api-extractor workload
* disable composite features of tsconfig.types.json for api-extractor
* set declaration: true to avoid weird debug error
* fix jest tests
Co-authored-by: spalger <spalger@users.noreply.github.com>
# Conflicts:
# .gitignore
# examples/hello_world/tsconfig.json
# tsconfig.json
# tsconfig.refs.json
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* chore(NA): ensure scripts run with --preserve-symlinks
* chore(NA): run webpack configs with symlinks: false
* chore(NA): fix pkg json load on kbn test
* chore(NA): add script into setup node env
* chore(NA): fix kbn test for relative improt
* chore(NA): fix change on docs
* chore(NA): move ensure node preserve symlinks into setup node env
* chore(NA): update changed docs
* chore(NA): update jest unit test
* chore(NA): fix wrapper script exit code
* chore(NA): updated generated plugin list docs
* fix(NA): make functional test runner use kbn utils repo_root
* chore(NA): fix eslint imports
* chore(NA): missing react correct config on eslint package
* chore(NA): use correct value to make test pass locally
* chore(NA): fix jest tests
* chore(NA): try remove extra preserve symlinks
* chore(NA): fix windows environment
* chore(NA): fix kbn-optimizer to run with preserve-symlinks
* chore(NA): fix integration jest test for kbn/optimizer
* chore(NA): remove require.resolve from eslintrc.js
* chore(NA): avoid load json file
* chore(NA): move kbn/utils import into kbn/dev-utils
* chore(NA): use correct dependencies on eslint config package
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* script that finds the chromium commit
* use run from dev-utils
* update docs and inch towards build pipeline
* more docs updates
* move the scripts the root
* review feedback
* Updating the Licenses, except for applying eslint, building
* Applying ESLint rules,building @kbn/pm, regenerating api docs
Co-authored-by: kobelb <brandon.kobel@elastic.co>
* chore(NA): update gitignore to include first changes from moving into a single package.json
* chore(NA): update gitignore
* chore(NA): move all the dependencies into the single package.json and apply changes to bootstrap
* chore(NA): fix types problems after the single package json
* chore(NA): include code to find the dependencies used across the code
* chore(NA): introduce pure lockfile for install dependencies on build
* chore(NA): update clean task to not delete anything from xpack node_modules
* chore(NA): update gitignore to remove development temporary rules
* chore(NA): update notice file
* chore(NA): update jest snapshots
* chore(NA): fix whitelisted licenses to include a new specify form of an already included one
* chore(NA): remove check lockfile symlinks from child projects
* chore(NA): fix eslint and add missing declared deps on single pkg json
* chore(NA): correctly update notice
* chore(NA): fix failing jest test for storyshots.test.tsx
* chore(NA): fix cypress multi reporter path
* chore(NA): fix Project tests check
* chore(NA): fix problem with logic to detect used dependes on oss build
* chore(NA): include correct x-pack plugins dep discovery
* chore(NA): discover entries under dynamic requires on vis_type_timelion
* chore(NA): remove canvas
* chore(NA): add initial code to find circular deps
* chore(NA): ground work to integrate the circular deps scripts
* chore(NA): add correct filtering to find circular dependenices feature
* chore(NA): add ci mode flag into circular deps script
* chore(NA): feature complete circular dependencies detect script
* chore(NA): merge and solve conflicts with master
* chore(NA): remove unwanted changes
* chore(NA): remove unwanted changes on kbn storybook
* chore(NA): hook find circular deps tool into ci
* chore(NA): remove previous find plugin circular deps script
* chore(NA): add type for circular dep list
* chore(NA): add type for circular dep list for allowed list
* chore(NA): allow CI to fail check
* chore(NA): update deps allowed list
* chore(NA): run search circular deps script over examples too
* docs(NA): adds cli description
* chore(NA): use plugin search paths to build entries to find circular deps
* chore(NA): update allowed list
* chore(NA): snapshot update for kbn optimizer test
* chore(NA): update dpdm version
* chore(NA): remove thirdParty flag
* chore(NA): update docs to include info about the new tool
* docs(NA): update to link PR instead of the issue
* chore(NA): update debug logs to always output allowedList
* fix(NA): correctly list found differences number
* chore(NA): remove quiet flag
* fix(NA): correctly fail the CI if circular deps are found
* chore(NA): complete list of found circular deps
* chore(NA): used named capturing group into the regex
* docs(NA): update typescript best practices docs and styleguide
* chore(NA): introduce quick filter option flag
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Run prettier on the files that will be refactored under @kbn/test, in a follow-up pr.
* Fixup all paths to mocha,
under kbn-test, with needed
exports/imports.
* Fixed borked path (bad refactor).
* Fixup one more borked path.
* Fixup tsconfig for xml.ts
* Drop setup_node_env as it's already
in KIBANA/scripts/mocha.js.
Also, fixup cwd for globby as we are
exec-ing from a different directory.
Co-authored-by: Tre <wayne.seymour@elastic.co>
* chore(NA): update gitignore to include first changes from moving into a single package.json
* chore(NA): update gitignore
* chore(NA): move all the dependencies into the single package.json and apply changes to bootstrap
* chore(NA): fix types problems after the single package json
* chore(NA): include code to find the dependencies used across the code
* chore(NA): introduce pure lockfile for install dependencies on build
* chore(NA): update clean task to not delete anything from xpack node_modules
* chore(NA): update gitignore to remove development temporary rules
* chore(NA): update notice file
* chore(NA): update jest snapshots
* chore(NA): fix whitelisted licenses to include a new specify form of an already included one
* chore(NA): remove check lockfile symlinks from child projects
* chore(NA): fix eslint and add missing declared deps on single pkg json
* chore(NA): correctly update notice
* chore(NA): fix failing jest test for storyshots.test.tsx
* chore(NA): fix cypress multi reporter path
* chore(NA): fix Project tests check
* chore(NA): fix problem with logic to detect used dependes on oss build
* chore(NA): include correct x-pack plugins dep discovery
* chore(NA): discover entries under dynamic requires on vis_type_timelion
* chore(NA): remove canvas
* test(NA): fix jest unit tests
* chore(NA): remove double react declaration from storyshot test file
* chore(NA): try removing isOSS check
* chore(NA): support for plugin development
* chore(NA): update logic to fix unit tests and typechecking
* chore(NA): support to run npm scripts in child kbn projects across all envs
* chore(NA): support github checks reporter on x-pack and remove cpy types as the package correctly provides them
* chore(NA): update cpy version
* chore(NA): include last kbn pm changes
* chore(NA): update style on build_production_projects.ts
* chore(NA): remove any cast fom telemetry opt in stats
* chore(NA): remove del and re-use rm -rf again
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
# Conflicts:
# package.json
# packages/kbn-pm/dist/index.js
# x-pack/package.json
# yarn.lock
* move kbn-dev-utils plugin helpers under a dedicated folder
* use getPluginSearchPaths in kbn-config & kbn-optimizer
* add a script to find plugin dependencies not migrated to TS project refs
* update docs
* add a script reporting all circular deps between plugins
based on kibana.json declaration, so it doesn't provide all the cases
* fix optimizer scan logic. removed by mistake
* revert changes. fails on CI
* remove prod depenedency on kbn/dev-utils
* remove last export
* only run plugin discovery once to speed up circular dep detection
* address comments
* address comments
* update fixtures
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: spalger <spalger@users.noreply.github.com>
* move test_helpers to the core
* create base tsconfig
* all tsconfigs use the base one
* use test_helpers exposed from the src/core
* move getFieldFormatsRegistry to data plugin
* add test_utils project
* compile types after checkout
* add a stub for platform tsconfig.json
* fix broken import
* fix broken path to the base config
* set tsBuildInfoFile for project without outDir
* do not commit tsbuildinfo file
* do not check output d.ts files
* fix type error
* use separate config to build types
* rollback changes to include paths
* mute import zone error
* rename files to avoid references to tsd
* do not use tsd for type tests
* include all ts files in project
* run buildRefs before type check to ensure the latest version
* store tsbuildinfo locally
* update paths to base config
* comment out core/tsconfig.json
* remove ui path
* fix wrong tsbuildinfo path
