# Backport
This will backport the following commits from `main` to `8.6`:
- [Reduce startup time by skipping update mappings step when possible
(#145604)](https://github.com/elastic/kibana/pull/145604)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Gerard
Soldevila","email":"gerard.soldevila@elastic.co"},"sourceCommit":{"committedDate":"2022-11-28T14:34:58Z","message":"Reduce
startup time by skipping update mappings step when possible
(#145604)\n\nThe goal of this PR is to reduce the startup times of
Kibana server by\r\nimproving the migration logic.\r\n\r\nFixes
https://github.com/elastic/kibana/issues/145743\r\nRelatedhttps://github.com/elastic/kibana/issues/144035)\r\n\r\nThe migration
logic is run systematically at startup, whether the\r\ncustomers are
upgrading or not.\r\nHistorically, these steps have been very quick, but
we recently found\r\nout about some customers that have more than **one
million** Saved\r\nObjects stored, making the overall startup process
slow, even when there\r\nare no migrations to perform.\r\n\r\nThis PR
specifically targets the case where there are no migrations
to\r\nperform, aka a Kibana node is started against an ES cluster that
is\r\nalready up to date wrt stack version and list of
plugins.\r\n\r\nIn this scenario, we aim at skipping the
`UPDATE_TARGET_MAPPINGS` step\r\nof the migration logic, which
internally runs the\r\n`updateAndPickupMappings` method, which turns out
to be expensive if the\r\nsystem indices contain lots of
SO.\r\n\r\n\r\nI locally tested the following scenarios too:\r\n\r\n-
**Fresh install.** The step is not even run, as the `.kibana`
index\r\ndid not exist ✅\r\n- **Stack version + list of plugins up to
date.** Simply restarting\r\nKibana after the fresh install. The step is
run and leads to `DONE`, as\r\nthe md5 hashes match those stored in
`.kibana._mapping._meta` ✅\r\n- **Faking re-enabling an old plugin.** I
manually removed one of the\r\nMD5 hashes from the stored
.kibana._mapping._meta through `curl`, and\r\nthen restarted Kibana. The
step is run and leads to\r\n`UPDATE_TARGET_MAPPINGS` as it used to
before the PR ✅\r\n- **Faking updating a plugin.** Same as the previous
one, but altering\r\nan existing md5 stored in the metas. ✅\r\n\r\nAnd
that is the curl command used to tamper with the stored
_meta:\r\n```bash\r\ncurl -X PUT
\"kibana:changeme@localhost:9200/.kibana/_mapping?pretty\" -H
'Content-Type: application/json' -d'\r\n{\r\n \"_meta\": {\r\n
\"migrationMappingPropertyHashes\": {\r\n \"references\":
\"7997cf5a56cc02bdc9c93361bde732b0\",\r\n }\r\n
}\r\n}\r\n'\r\n```","sha":"b1e18a0414ed99456706119d15173b687c6e7366","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Core","enhancement","release_note:skip","Feature:Migrations","backport:prev-minor","v8.7.0"],"number":145604,"url":"https://github.com/elastic/kibana/pull/145604","mergeCommit":{"message":"Reduce
startup time by skipping update mappings step when possible
(#145604)\n\nThe goal of this PR is to reduce the startup times of
Kibana server by\r\nimproving the migration logic.\r\n\r\nFixes
https://github.com/elastic/kibana/issues/145743\r\nRelatedhttps://github.com/elastic/kibana/issues/144035)\r\n\r\nThe migration
logic is run systematically at startup, whether the\r\ncustomers are
upgrading or not.\r\nHistorically, these steps have been very quick, but
we recently found\r\nout about some customers that have more than **one
million** Saved\r\nObjects stored, making the overall startup process
slow, even when there\r\nare no migrations to perform.\r\n\r\nThis PR
specifically targets the case where there are no migrations
to\r\nperform, aka a Kibana node is started against an ES cluster that
is\r\nalready up to date wrt stack version and list of
plugins.\r\n\r\nIn this scenario, we aim at skipping the
`UPDATE_TARGET_MAPPINGS` step\r\nof the migration logic, which
internally runs the\r\n`updateAndPickupMappings` method, which turns out
to be expensive if the\r\nsystem indices contain lots of
SO.\r\n\r\n\r\nI locally tested the following scenarios too:\r\n\r\n-
**Fresh install.** The step is not even run, as the `.kibana`
index\r\ndid not exist ✅\r\n- **Stack version + list of plugins up to
date.** Simply restarting\r\nKibana after the fresh install. The step is
run and leads to `DONE`, as\r\nthe md5 hashes match those stored in
`.kibana._mapping._meta` ✅\r\n- **Faking re-enabling an old plugin.** I
manually removed one of the\r\nMD5 hashes from the stored
.kibana._mapping._meta through `curl`, and\r\nthen restarted Kibana. The
step is run and leads to\r\n`UPDATE_TARGET_MAPPINGS` as it used to
before the PR ✅\r\n- **Faking updating a plugin.** Same as the previous
one, but altering\r\nan existing md5 stored in the metas. ✅\r\n\r\nAnd
that is the curl command used to tamper with the stored
_meta:\r\n```bash\r\ncurl -X PUT
\"kibana:changeme@localhost:9200/.kibana/_mapping?pretty\" -H
'Content-Type: application/json' -d'\r\n{\r\n \"_meta\": {\r\n
\"migrationMappingPropertyHashes\": {\r\n \"references\":
\"7997cf5a56cc02bdc9c93361bde732b0\",\r\n }\r\n
}\r\n}\r\n'\r\n```","sha":"b1e18a0414ed99456706119d15173b687c6e7366"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/145604","number":145604,"mergeCommit":{"message":"Reduce
startup time by skipping update mappings step when possible
(#145604)\n\nThe goal of this PR is to reduce the startup times of
Kibana server by\r\nimproving the migration logic.\r\n\r\nFixes
https://github.com/elastic/kibana/issues/145743\r\nRelatedhttps://github.com/elastic/kibana/issues/144035)\r\n\r\nThe migration
logic is run systematically at startup, whether the\r\ncustomers are
upgrading or not.\r\nHistorically, these steps have been very quick, but
we recently found\r\nout about some customers that have more than **one
million** Saved\r\nObjects stored, making the overall startup process
slow, even when there\r\nare no migrations to perform.\r\n\r\nThis PR
specifically targets the case where there are no migrations
to\r\nperform, aka a Kibana node is started against an ES cluster that
is\r\nalready up to date wrt stack version and list of
plugins.\r\n\r\nIn this scenario, we aim at skipping the
`UPDATE_TARGET_MAPPINGS` step\r\nof the migration logic, which
internally runs the\r\n`updateAndPickupMappings` method, which turns out
to be expensive if the\r\nsystem indices contain lots of
SO.\r\n\r\n\r\nI locally tested the following scenarios too:\r\n\r\n-
**Fresh install.** The step is not even run, as the `.kibana`
index\r\ndid not exist ✅\r\n- **Stack version + list of plugins up to
date.** Simply restarting\r\nKibana after the fresh install. The step is
run and leads to `DONE`, as\r\nthe md5 hashes match those stored in
`.kibana._mapping._meta` ✅\r\n- **Faking re-enabling an old plugin.** I
manually removed one of the\r\nMD5 hashes from the stored
.kibana._mapping._meta through `curl`, and\r\nthen restarted Kibana. The
step is run and leads to\r\n`UPDATE_TARGET_MAPPINGS` as it used to
before the PR ✅\r\n- **Faking updating a plugin.** Same as the previous
one, but altering\r\nan existing md5 stored in the metas. ✅\r\n\r\nAnd
that is the curl command used to tamper with the stored
_meta:\r\n```bash\r\ncurl -X PUT
\"kibana:changeme@localhost:9200/.kibana/_mapping?pretty\" -H
'Content-Type: application/json' -d'\r\n{\r\n \"_meta\": {\r\n
\"migrationMappingPropertyHashes\": {\r\n \"references\":
\"7997cf5a56cc02bdc9c93361bde732b0\",\r\n }\r\n
}\r\n}\r\n'\r\n```","sha":"b1e18a0414ed99456706119d15173b687c6e7366"}}]}]
BACKPORT-->
* Bump `elastic/elasticsearch` to `8.4.0-canary.1`
* fixing package violations
* some xpack violations
* remove more unused ts-expect-error
* started muting errors
* start adapting watcher
* fix more usages
* fix more usages
* fixing more usages
* just one more
* more usages
* infra usages
* moar types
* fix unit test
* adapt more usages
* fixes in FTR tests
* try to fix kbn/es-types
* last fixes?
* [packages] add kibana.jsonc files
* auto-migrate to kibana.jsonc
* support interactive pkg id selection too
* remove old codeowners entry
* skip codeowners generation when .github/CODEOWNERS doesn't exist
* fall back to format validation if user is offline
* update question style
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
