# Backport
This will backport the following commits from `main` to `8.11`:
- [[build] Rename ubi9 image to ubi
(#171424)](https://github.com/elastic/kibana/pull/171424)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"Jon","email":"jon@elastic.co"},"sourceCommit":{"committedDate":"2023-11-29T17:50:22Z","message":"[build]
Rename ubi9 image to ubi (#171424)\n\nFuture ubi image names will be
version agnostic. Removal of ubi8
TBD.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"0796976a274a691221d79c067f9aef947dcf37e9","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","ci:build-os-packages","backport:all-open","v8.12.0"],"number":171424,"url":"https://github.com/elastic/kibana/pull/171424","mergeCommit":{"message":"[build]
Rename ubi9 image to ubi (#171424)\n\nFuture ubi image names will be
version agnostic. Removal of ubi8
TBD.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"0796976a274a691221d79c067f9aef947dcf37e9"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/171424","number":171424,"mergeCommit":{"message":"[build]
Rename ubi9 image to ubi (#171424)\n\nFuture ubi image names will be
version agnostic. Removal of ubi8
TBD.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"0796976a274a691221d79c067f9aef947dcf37e9"}}]}]
BACKPORT-->
Co-authored-by: Jon <jon@elastic.co>
# Backport
This will backport the following commits from `main` to `8.11`:
- [Remove CI Composite Storybook
(#171258)](https://github.com/elastic/kibana/pull/171258)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Brad
White","email":"Ikuni17@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-11-21T21:59:39Z","message":"Remove
CI Composite Storybook (#171258)\n\n## Summary\r\nCloses
#160803\r\n\r\nThis PR removes the `CI Composite` story because it has
been broken\r\nsince at least ac23dce29f
(and possibly\r\nsince b862a6c181). The
functionality is\r\ncovered by the generated `index.html`
in\r\ndda4498fee/.buildkite/scripts/steps/storybooks/build_and_upload.ts (L105-L120)\r\n\r\nTo
fix the composite story requires generating `stories.json` for
every\r\nstorybook, which requires migrating the repo off the
deprecated\r\n`storiesOf` API. That task is quite extensive and would be
better\r\nhandled alongside an upgrade to Storybook
7.x","sha":"1919c87b90c4f489c2027d71293631d89034f40f","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:all-open","ci:build-storybooks","v8.12.0"],"number":171258,"url":"https://github.com/elastic/kibana/pull/171258","mergeCommit":{"message":"Remove
CI Composite Storybook (#171258)\n\n## Summary\r\nCloses
#160803\r\n\r\nThis PR removes the `CI Composite` story because it has
been broken\r\nsince at least ac23dce29f
(and possibly\r\nsince b862a6c181). The
functionality is\r\ncovered by the generated `index.html`
in\r\ndda4498fee/.buildkite/scripts/steps/storybooks/build_and_upload.ts (L105-L120)\r\n\r\nTo
fix the composite story requires generating `stories.json` for
every\r\nstorybook, which requires migrating the repo off the
deprecated\r\n`storiesOf` API. That task is quite extensive and would be
better\r\nhandled alongside an upgrade to Storybook
7.x","sha":"1919c87b90c4f489c2027d71293631d89034f40f"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/171258","number":171258,"mergeCommit":{"message":"Remove
CI Composite Storybook (#171258)\n\n## Summary\r\nCloses
#160803\r\n\r\nThis PR removes the `CI Composite` story because it has
been broken\r\nsince at least ac23dce29f
(and possibly\r\nsince b862a6c181). The
functionality is\r\ncovered by the generated `index.html`
in\r\ndda4498fee/.buildkite/scripts/steps/storybooks/build_and_upload.ts (L105-L120)\r\n\r\nTo
fix the composite story requires generating `stories.json` for
every\r\nstorybook, which requires migrating the repo off the
deprecated\r\n`storiesOf` API. That task is quite extensive and would be
better\r\nhandled alongside an upgrade to Storybook
7.x","sha":"1919c87b90c4f489c2027d71293631d89034f40f"}}]}] BACKPORT-->
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
# Backport
This will backport the following commits from `main` to `8.11`:
- [[Security Solution][Endpoint][Cypress] Update endpoint cypress config
to include videos
(#170499)](https://github.com/elastic/kibana/pull/170499)
<!--- Backport version: 8.9.8 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"Ash","email":"1849116+ashokaditya@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-11-06T16:04:42Z","message":"[Security
Solution][Endpoint][Cypress] Update endpoint cypress config to include
videos (#170499)\n\n## Summary\r\n\r\nAdds videos to failed cypress
tests for
debugging.","sha":"1c2521705a1ece1ebe87740a4c99c27b254630f9","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","OLM
Sprint","v8.11.0","v8.12.0"],"number":170499,"url":"https://github.com/elastic/kibana/pull/170499","mergeCommit":{"message":"[Security
Solution][Endpoint][Cypress] Update endpoint cypress config to include
videos (#170499)\n\n## Summary\r\n\r\nAdds videos to failed cypress
tests for
debugging.","sha":"1c2521705a1ece1ebe87740a4c99c27b254630f9"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/170499","number":170499,"mergeCommit":{"message":"[Security
Solution][Endpoint][Cypress] Update endpoint cypress config to include
videos (#170499)\n\n## Summary\r\n\r\nAdds videos to failed cypress
tests for
debugging.","sha":"1c2521705a1ece1ebe87740a4c99c27b254630f9"}}]}]
BACKPORT-->
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
# Backport
This will backport the following commits from `main` to `8.11`:
- [FTR Api Integration with the Second Security Quality Gate
(#169422)](https://github.com/elastic/kibana/pull/169422)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"dkirchan","email":"55240027+dkirchan@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-11-09T15:27:02Z","message":"FTR
Api Integration with the Second Security Quality Gate (#169422)\n\n##
Summary\r\n\r\nFTR Api Integration tests for Security Solution are now
integrated with\r\nthe Second Security Quality Gate. The tests are
running in 5 scripts
in\r\n`x-pack/test/security_solution_api_integration/package.json`.\r\nEach
different script is creating its own task in buildkite so 5
scripts\r\ncould be running in parallel, depending on the agents
availability,\r\nhowever they all target the same environment for this
first integration.\r\n\r\nFuture needs: \r\n- The FTR runner serverless
is needed to be introduced. \r\n\r\n\r\n### Checklist\r\n\r\nDelete any
items that are not applicable to this PR.\r\n\r\n- [ ] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces—unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes—Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
Jon
<jon@elastic.co>","sha":"397061aebcca23b22d052cf240e566ca25dcc34d","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:
SecuritySolution","Team:Detection
Engine","v8.11.0","v8.12.0"],"number":169422,"url":"https://github.com/elastic/kibana/pull/169422","mergeCommit":{"message":"FTR
Api Integration with the Second Security Quality Gate (#169422)\n\n##
Summary\r\n\r\nFTR Api Integration tests for Security Solution are now
integrated with\r\nthe Second Security Quality Gate. The tests are
running in 5 scripts
in\r\n`x-pack/test/security_solution_api_integration/package.json`.\r\nEach
different script is creating its own task in buildkite so 5
scripts\r\ncould be running in parallel, depending on the agents
availability,\r\nhowever they all target the same environment for this
first integration.\r\n\r\nFuture needs: \r\n- The FTR runner serverless
is needed to be introduced. \r\n\r\n\r\n### Checklist\r\n\r\nDelete any
items that are not applicable to this PR.\r\n\r\n- [ ] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces—unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes—Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
Jon
<jon@elastic.co>","sha":"397061aebcca23b22d052cf240e566ca25dcc34d"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169422","number":169422,"mergeCommit":{"message":"FTR
Api Integration with the Second Security Quality Gate (#169422)\n\n##
Summary\r\n\r\nFTR Api Integration tests for Security Solution are now
integrated with\r\nthe Second Security Quality Gate. The tests are
running in 5 scripts
in\r\n`x-pack/test/security_solution_api_integration/package.json`.\r\nEach
different script is creating its own task in buildkite so 5
scripts\r\ncould be running in parallel, depending on the agents
availability,\r\nhowever they all target the same environment for this
first integration.\r\n\r\nFuture needs: \r\n- The FTR runner serverless
is needed to be introduced. \r\n\r\n\r\n### Checklist\r\n\r\nDelete any
items that are not applicable to this PR.\r\n\r\n- [ ] Any text added
follows [EUI's
writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\r\nsentence case text and includes
[i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n-
[
]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas
added for features that require explanation or tutorials\r\n- [ ] [Unit
or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] Any UI
touched in this PR is usable by keyboard only (learn more\r\nabout
[keyboard accessibility](https://webaim.org/techniques/keyboard/))\r\n-
[ ] Any UI touched in this PR does not create any new axe
failures\r\n(run axe in
browser:\r\n[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),\r\n[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))\r\n-
[ ] If a plugin configuration key changed, check if it needs to
be\r\nallowlisted in the cloud and added to the
[docker\r\nlist](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)\r\n-
[ ] This renders correctly on smaller devices using a
responsive\r\nlayout. (You can test this [in
your\r\nbrowser](https://www.browserstack.com/guide/responsive-testing-on-local-server))\r\n-
[ ] This was checked for
[cross-browser\r\ncompatibility](https://www.elastic.co/support/matrix#matrix_browsers)\r\n\r\n\r\n###
Risk Matrix\r\n\r\nDelete this section if it is not applicable to this
PR.\r\n\r\nBefore closing this PR, invite QA, stakeholders, and other
developers to\r\nidentify risks that should be tested prior to the
change/feature\r\nrelease.\r\n\r\nWhen forming the risk matrix, consider
some of the following examples\r\nand how they may potentially impact
the change:\r\n\r\n| Risk | Probability | Severity | Mitigation/Notes
|\r\n\r\n|---------------------------|-------------|----------|-------------------------|\r\n|
Multiple Spaces—unexpected behavior in non-default Kibana
Space.\r\n| Low | High | Integration tests will verify that all features
are still\r\nsupported in non-default Kibana Space and when user
switches between\r\nspaces. |\r\n| Multiple nodes—Elasticsearch
polling might have race conditions\r\nwhen multiple Kibana nodes are
polling for the same tasks. | High | Low\r\n| Tasks are idempotent, so
executing them multiple times will not result\r\nin logical error, but
will degrade performance. To test for this case we\r\nadd plenty of unit
tests around this logic and document manual testing\r\nprocedure. |\r\n|
Code should gracefully handle cases when feature X or plugin Y
are\r\ndisabled. | Medium | High | Unit tests will verify that any
feature flag\r\nor plugin combination still results in our service
operational. |\r\n| [See more potential
risk\r\nexamples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
|\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for
breaking API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
Jon
<jon@elastic.co>","sha":"397061aebcca23b22d052cf240e566ca25dcc34d"}}]}]
BACKPORT-->
Co-authored-by: dkirchan <55240027+dkirchan@users.noreply.github.com>
# Backport
This will backport the following commits from `main` to `8.11`:
- [[artifacts] Test mock building Iron Bank context
(#170682)](https://github.com/elastic/kibana/pull/170682)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"Jon","email":"jon@elastic.co"},"sourceCommit":{"committedDate":"2023-11-07T14:07:09Z","message":"[artifacts]
Test mock building Iron Bank context (#170682)\n\nThis uses our mirror
to fetch UBI instead of the upstream mirror. It's\r\nintended to
approximately test changes in development and to provide\r\nquicker
feedback before publishing pipelines are
run.\r\n\r\nhttps://buildkite.com/elastic/kibana-artifacts-snapshot/builds/3443","sha":"553cbd2c76c460596e7236075bc1a885621f5b58","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:all-open","v8.12.0"],"number":170682,"url":"https://github.com/elastic/kibana/pull/170682","mergeCommit":{"message":"[artifacts]
Test mock building Iron Bank context (#170682)\n\nThis uses our mirror
to fetch UBI instead of the upstream mirror. It's\r\nintended to
approximately test changes in development and to provide\r\nquicker
feedback before publishing pipelines are
run.\r\n\r\nhttps://buildkite.com/elastic/kibana-artifacts-snapshot/builds/3443","sha":"553cbd2c76c460596e7236075bc1a885621f5b58"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/170682","number":170682,"mergeCommit":{"message":"[artifacts]
Test mock building Iron Bank context (#170682)\n\nThis uses our mirror
to fetch UBI instead of the upstream mirror. It's\r\nintended to
approximately test changes in development and to provide\r\nquicker
feedback before publishing pipelines are
run.\r\n\r\nhttps://buildkite.com/elastic/kibana-artifacts-snapshot/builds/3443","sha":"553cbd2c76c460596e7236075bc1a885621f5b58"}}]}]
BACKPORT-->
Co-authored-by: Jon <jon@elastic.co>
# Backport
This will backport the following commits from `main` to `8.11`:
- [[ci/on-merge] Adjust check types queue
(#170026)](https://github.com/elastic/kibana/pull/170026)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"Jon","email":"jon@elastic.co"},"sourceCommit":{"committedDate":"2023-10-27T13:08:48Z","message":"[ci/on-merge]
Adjust check types queue (#170026)\n\nn2-2-spot preemption are frequent
enough that this is causing pipeline\r\ninstability. This moves the
instance size up to n2-4\r\n\r\nThis also removes the max-old-space-size
definition I previously set.\r\ndelanni helped me understand that we're
spawning a tsc process with a\r\nmemory limit already
defined.","sha":"fafec2839882b59b39080f6ebdfd8cd67c0f62cd","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:prev-minor","v8.12.0"],"number":170026,"url":"https://github.com/elastic/kibana/pull/170026","mergeCommit":{"message":"[ci/on-merge]
Adjust check types queue (#170026)\n\nn2-2-spot preemption are frequent
enough that this is causing pipeline\r\ninstability. This moves the
instance size up to n2-4\r\n\r\nThis also removes the max-old-space-size
definition I previously set.\r\ndelanni helped me understand that we're
spawning a tsc process with a\r\nmemory limit already
defined.","sha":"fafec2839882b59b39080f6ebdfd8cd67c0f62cd"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/170026","number":170026,"mergeCommit":{"message":"[ci/on-merge]
Adjust check types queue (#170026)\n\nn2-2-spot preemption are frequent
enough that this is causing pipeline\r\ninstability. This moves the
instance size up to n2-4\r\n\r\nThis also removes the max-old-space-size
definition I previously set.\r\ndelanni helped me understand that we're
spawning a tsc process with a\r\nmemory limit already
defined.","sha":"fafec2839882b59b39080f6ebdfd8cd67c0f62cd"}}]}]
BACKPORT-->
Co-authored-by: Jon <jon@elastic.co>
# Backport
This will backport the following commits from `main` to `8.11`:
- [[ci] Remove --fix from precommit_hook
(#169978)](https://github.com/elastic/kibana/pull/169978)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT
[{"author":{"name":"Jon","email":"jon@elastic.co"},"sourceCommit":{"committedDate":"2023-10-27T12:39:44Z","message":"[ci]
Remove --fix from precommit_hook (#169978)\n\nCurrently on CI, the
precommit_hook will fix lint warnings against the\r\nmost recent commit.
When multiple commits are pushed up at once, lint\r\nwarnings on earlier
commits may go unnoticed until they are squashed and\r\nmerged. The
on-merge pipeline will fail due to changed files, and pull\r\nrequests
based on the squashed commit will see unrelated
changes\r\nauto-fixed.\r\n\r\nThis removes the --fix flag from the
precommit hook check.\r\n\r\nCloses
https://github.com/elastic/kibana/issues/169966","sha":"134532384ec5a3419c2ef08bcef3562574da97e5","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:prev-minor","v8.12.0"],"number":169978,"url":"https://github.com/elastic/kibana/pull/169978","mergeCommit":{"message":"[ci]
Remove --fix from precommit_hook (#169978)\n\nCurrently on CI, the
precommit_hook will fix lint warnings against the\r\nmost recent commit.
When multiple commits are pushed up at once, lint\r\nwarnings on earlier
commits may go unnoticed until they are squashed and\r\nmerged. The
on-merge pipeline will fail due to changed files, and pull\r\nrequests
based on the squashed commit will see unrelated
changes\r\nauto-fixed.\r\n\r\nThis removes the --fix flag from the
precommit hook check.\r\n\r\nCloses
https://github.com/elastic/kibana/issues/169966","sha":"134532384ec5a3419c2ef08bcef3562574da97e5"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/169978","number":169978,"mergeCommit":{"message":"[ci]
Remove --fix from precommit_hook (#169978)\n\nCurrently on CI, the
precommit_hook will fix lint warnings against the\r\nmost recent commit.
When multiple commits are pushed up at once, lint\r\nwarnings on earlier
commits may go unnoticed until they are squashed and\r\nmerged. The
on-merge pipeline will fail due to changed files, and pull\r\nrequests
based on the squashed commit will see unrelated
changes\r\nauto-fixed.\r\n\r\nThis removes the --fix flag from the
precommit hook check.\r\n\r\nCloses
https://github.com/elastic/kibana/issues/169966","sha":"134532384ec5a3419c2ef08bcef3562574da97e5"}}]}]
BACKPORT-->
Co-authored-by: Jon <jon@elastic.co>
# Backport
This will backport the following commits from `main` to `8.11`:
- [[EDR Workflows] Flaky CY e2e tests
(#167870)](https://github.com/elastic/kibana/pull/167870)
<!--- Backport version: 8.9.8 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Konrad
Szwarc","email":"konrad.szwarc@elastic.co"},"sourceCommit":{"committedDate":"2023-10-09T09:21:02Z","message":"[EDR
Workflows] Flaky CY e2e tests (#167870)\n\nThis pull request tackles
flaky `artifacts.cy.ts` e2e test. Ran 7+ times\r\nwithout flakiness in
CI.\r\n\r\nAdded Test Burning to Defend Workflows
pipeline.\r\n\r\nChanged:\r\n1. Removed `.first()` selectors in favour
of explicit element getters\r\n2. Removed method chain\r\n3. Moved url
change from `before` to test body.\r\n4. In `beforeAll` now we first
fetch current revision number, then run\r\n`Promise.all` on
`removeAllArtifacts` which resolves with `true/false`\r\nbased on
whether they actually removed an artifact. Its being counted\r\nthen and
with this number we know what next revision should be
(revision\r\nfetched before removal + times removal was successful). We
query API\r\nrecursively until revisions match and only then we can be
sure that\r\ndisplayed in the UI revision is up to date. This fixes the
main reason\r\nof the flakiness of this test and makes it a pure test
that should\r\nalways yield the same result no matter the number of
runs.\r\n\r\n---------\r\n\r\nCo-authored-by: Patryk Kopyciński
<contact@patrykkopycinski.com>","sha":"be26b7d7bcad0f01efc6eddd8544958dd1e7ea5a","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Defend
Workflows","v8.11.0","v8.12.0"],"number":167870,"url":"https://github.com/elastic/kibana/pull/167870","mergeCommit":{"message":"[EDR
Workflows] Flaky CY e2e tests (#167870)\n\nThis pull request tackles
flaky `artifacts.cy.ts` e2e test. Ran 7+ times\r\nwithout flakiness in
CI.\r\n\r\nAdded Test Burning to Defend Workflows
pipeline.\r\n\r\nChanged:\r\n1. Removed `.first()` selectors in favour
of explicit element getters\r\n2. Removed method chain\r\n3. Moved url
change from `before` to test body.\r\n4. In `beforeAll` now we first
fetch current revision number, then run\r\n`Promise.all` on
`removeAllArtifacts` which resolves with `true/false`\r\nbased on
whether they actually removed an artifact. Its being counted\r\nthen and
with this number we know what next revision should be
(revision\r\nfetched before removal + times removal was successful). We
query API\r\nrecursively until revisions match and only then we can be
sure that\r\ndisplayed in the UI revision is up to date. This fixes the
main reason\r\nof the flakiness of this test and makes it a pure test
that should\r\nalways yield the same result no matter the number of
runs.\r\n\r\n---------\r\n\r\nCo-authored-by: Patryk Kopyciński
<contact@patrykkopycinski.com>","sha":"be26b7d7bcad0f01efc6eddd8544958dd1e7ea5a"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/167870","number":167870,"mergeCommit":{"message":"[EDR
Workflows] Flaky CY e2e tests (#167870)\n\nThis pull request tackles
flaky `artifacts.cy.ts` e2e test. Ran 7+ times\r\nwithout flakiness in
CI.\r\n\r\nAdded Test Burning to Defend Workflows
pipeline.\r\n\r\nChanged:\r\n1. Removed `.first()` selectors in favour
of explicit element getters\r\n2. Removed method chain\r\n3. Moved url
change from `before` to test body.\r\n4. In `beforeAll` now we first
fetch current revision number, then run\r\n`Promise.all` on
`removeAllArtifacts` which resolves with `true/false`\r\nbased on
whether they actually removed an artifact. Its being counted\r\nthen and
with this number we know what next revision should be
(revision\r\nfetched before removal + times removal was successful). We
query API\r\nrecursively until revisions match and only then we can be
sure that\r\ndisplayed in the UI revision is up to date. This fixes the
main reason\r\nof the flakiness of this test and makes it a pure test
that should\r\nalways yield the same result no matter the number of
runs.\r\n\r\n---------\r\n\r\nCo-authored-by: Patryk Kopyciński
<contact@patrykkopycinski.com>","sha":"be26b7d7bcad0f01efc6eddd8544958dd1e7ea5a"}}]}]
BACKPORT-->
## Summary
This is hopefully the last batch of typescript issues to be fixed,
related to https://github.com/elastic/kibana/pull/166813.
It's also re-enabling full typecheck, with this, we should be back in a
clean, typechecked main branch.
Blocked by #167428
---------
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Co-authored-by: Brad White <brad.white@elastic.co>
Co-authored-by: Thomas Watson <watson@elastic.co>
Co-authored-by: Patryk Kopyciński <contact@patrykkopycinski.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
### Error 1: Strict mode throws an error on non-filled variables. I
forgot to add this when creating the script
```
.buildkite/scripts/steps/es_serverless/promote_es_serverless_image.sh: line 40: UPLOAD_MANIFEST: unbound variable
```
### Error 2: Uploading multi-arch images
https://elastic.slack.com/archives/C5UDAFZQU/p1695725623585409
We've noticed that the downloaded `latest-verified` images are slower to
start up than normal. After inspecting the manifests, it seems we were
getting `linux/amd64` on our arm devices as well.
The solution is to grab and upload both platform variants. (using this
blog:
https://www.docker.com/blog/multi-arch-build-and-images-the-simple-way/)
## Summary
This PR is the core part of #166813. The original work seems to grow
large, and we'd like to enable a preventive check beforehand to prevent
more errors from entering the codebase.
The idea is to have a selective type check that would only check changed
files' projects.
- [x] when there's no extra label, run the selective type check only on
the diffing files' projects (success:
https://buildkite.com/elastic/kibana-pull-request/builds/161837)
- [x] when the label `ci:hard-typecheck` is present, run the regular
(but now, working) full typecheck (expected to fail: )
cc: @watson
---------
Co-authored-by: Brad White <brad.white@elastic.co>
Co-authored-by: Thomas Watson <w@tson.dk>
Co-authored-by: Thomas Watson <watson@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Prepares the serverless FTR tests to be runnable with a custom ES image.
(`--esServerlessImage` cli arg)
Creates a pipeline for testing and promoting ES Serverless docker
releases.
The job can be triggered here:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote
The three main env variables it takes:
- BUILDKITE_BRANCH: the kibana branch to test with (maybe not as
important)
- BUILDKITE_COMMIT: the kibana commit to test with
- ES_SERVERLESS_IMAGE: the elasticsearch serverless image, or tag to use
from this repo:
`docker.elastic.co/elasticsearch-ci/elasticsearch-serverless`
## TODOS:
- [x] set `latest_verified` with full img path as default
- [x] ~~find other CLIs that might need the `esServerlessImage` argument
(if the docker runner has multiple usages)~~ | I confused the `yarn es
docker` with this, because I thought we only run ES serverless in a
docker container, but `elasticsearch` can also be run in docker.
- [x] set `latest-compatible` or similar flag in a manifest in gcs for
Elastic's use-case
- [ ] ensure we can only verify "forward" (ie.: to avoid a
parameterization on old versions to set our pointers back) [on a second
thought, this might be kept as a feature to roll back (if we should ever
need that)]
There are two confusing things I couldn't sort out just yet:
#### Ambiguity in --esServerlessImage
We can either have 2 CLI args: one for an image tag, one for an image
repo/image url, or we can have one (like I have it now) and interpret
that in the code, it can be either the image url, or the tag. It's more
flexible, but it's two things in one. Is it ok this way, or is it too
confusing?
e.g.:
```
node scripts/functional_tests --esFrom serverless --esServerlessImage docker.elastic.co/elasticsearch-ci/elasticsearch-serverless:git-8fc8f941bd4d --bail --config x-pack/test_serverless/functional/test_suites/security/config.ts
# or
node scripts/functional_tests --esFrom serverless --esServerlessImage latest --bail --config x-pack/test_serverless/functional/test_suites/security/config.ts
```
#### Ambiguity in the default image path
The published ES Serverless images will sit on this image path:
`docker.elastic.co/elasticsearch-ci/elasticsearch-serverless`, however,
our one exception is the `latest-verified` which we will be tagging
under a different path, where we have write rights:
`docker.elastic.co/kibana-ci/elasticsearch-serverless:latest-verified`.
Is it okay, that by default, we're searching in the `elasticsearch-ci`
images for any tags as parameters (after all, all the new images will be
published there), however our grand default will ultimately be
`docker.elastic.co/kibana-ci/elasticsearch-serverless:latest-verified`.
## Links
Buildkite:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote
eg.:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote/builds/24
Closes: https://github.com/elastic/kibana/issues/162931
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
- Combines the `endpoint` and `mocked_data` tests suites so that they
run from the same cypress configuration/run buildkite setup
- Moved test files from the `endpoint/` and `mocked_data/` directories
into new sub-directories that more closely describe the set of tests
they contain
- The `security_solution/package.json` file was updated so that the
following `scripts` will now output a warning indicating that command is
no longer valid:
- `cypress:dw:endpoint`
- `cypress:dw:endpoint:run`
- `cypress:dw:endpoint:open`
The following npm/yarn commands remain available for running tests
locally:
```shell
yarn --cwd=x-pack/plugins/security_solution cypress:dw:open
```
```shell
yarn --cwd=x-pack/plugins/security_solution cypress:dw:run
```
New test file struncture:
<img width="415" alt="image"
src="0cb4bc76-b434-4219-b73e-508645201a81">
## Summary
- introduces tags for [Defend Workflows] cypress tests (similarly to
https://github.com/elastic/kibana/pull/162698)
- adds scripts to Security Solution:
- `cypress:dw:serverless:open` and `:run`
- `cypress:dw:endpoint:serverless:open` and `:run`
- adds CI jobs to run these scripts
- so far most of the expected tests got both `@serverless` and
`@brokenInServerless` tests, because of other issues to be solved,
- one test is able to run against serverless:
`x-pack/plugins/security_solution/public/management/cypress/e2e/mocked_data/policy_details.cy.ts`
This moves all FTR and cypress based serverless tests into the default
test pipeline. By moving these tests, failures will
now fail pull request and on-merge pipelines.
## Summary
Fixes failed build steps when there are no tests to execute and hence no
reports.
## Details
Despite this [PR](https://github.com/elastic/kibana/pull/165824) fixes
tests failure when there are no reports but `yarn junit:merge` tries to
convert them to junit format the problem is partially stays as `Failed
Test Reporter` runs for every build step meaning also for successful
build steps. We don't need to handle failed test reports for successful
build steps. There are cases when there are no tests to run so Cypress
doesn't produce reports and nothing is converted to junit format so
`Failed Test Reporter` fails and causes a build step to fails. It could
be solved by defining an environment variable
`DISABLE_MISSING_TEST_REPORT_ERRORS=true` but we need to make sure
reports exist for failed tests. Taking this into account we can rely on
`BUILDKITE_COMMAND_EXIT_STATUS` to avoid running `Failed Test Reporter`
if the build step successed.
One may ask why don't skip `Upload Artifacts` too. We may need some
build artifacts for successful build steps.
---------
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
## Summary
This PR implements a temporary fix to prevent `yarn junit:merge` command
from breaking a build step by returning non zero code. `yarn
junit:merge` fails in case if it can't find reports or folders it
operates on weren't created beforehand.
## Summary
It fixes Osquery Cypress CI build step by rolling back the changes.
## Details
https://github.com/elastic/kibana/pull/165541 broke Osquery build step
by running junit transformation command while Osquery Cypress tests
don't produce any reports. The fix just removes junit transformation
command.
**Fixes:** https://github.com/elastic/kibana/issues/165546
## Summary
This PR fixes junit report transformation for Security Solution build steps (Cypress writes reports in `mochawesome` format and `yarn junit:transform` transforms it to junit format).
## Details
After refactoring it turned out `yarn junit:merge` was moved from `cypress:*` yarn scripts to build step `*.sh` files in `.buildkite/scripts/steps/functional` folder. This way a command to run Cypress tests changed from
```sh
yarn cypress:run:ess
```
to
```sh
yarn cypress:run:ess; status=$?; yarn junit:merge && exit $status
```
In first case any test failure do not lead to early exist and all following commands are executed as yarn runs scripts without preserving shell settings. In the second case failing tests lead to `yarn cypress:run:ess` exit with non-zero exit code so the shell script exits immediately without giving a chance for `yarn junit:merge` to execute.
This problem is solved by disabling early exit on error via `set +e`.
On top of that using of `artifact_paths` config option is redundant as [post command script](https://github.com/elastic/kibana/blob/main/.buildkite/scripts/lifecycle/post_command.sh#L16) upload build artefact via a bildkite agent.
#### [Failed build example](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3049#_)
## Summary
Kibana's build jobs work on a different subset of executors than the
buildkite pipelines defined in `catalog-info.yaml`.
The former jobs require a set of `pre_command` / `post_command` steps to
prepare the jobs for building/testing kibana.
The latter don't have access rights to certain vault secrets (and
possibly other missing config from the Kibana world), but for now,
they're also not building Kibana, they just trigger other jobs, so we
can just skip the problematic hooks.
~~A probably good indicator I found for deciding whether we need the
kibana-related `pre_command` is the
`BUILDKITE_AGENT_META_DATA_AGENT_MANAGER` flag, that's set to `"kibana"`
in the case of the kibana executors.~~
We can try to match on the agent names for the CI-systems agents. They
seem to be starting with `bk-agent`.
This should allow for the
[kibana-tests](https://buildkite.com/elastic/kibana-tests) job to run.
Split from: https://github.com/elastic/kibana/pull/165346
---------
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
When buildkite-agent is uploading a pipeline we can skip setting up our
node environment. This stricter check avoids matching on similarly named
steps, e.g. our storybooks upload.
## Summary
This PR calls the pipeline introduced on
https://github.com/elastic/gpctl/pull/167 which calls the e2e tests
before updating serverless gitops.
The previous change which was introduced on the commits below, used
gpctl to update the image tags for kibana on the controller. This change
resulted in skipping the part where the e2e tests were being run 😬bf148fb35f3cfbf24190
The present PR fixes this, by using the new pipeline which includes the
e2e tests introduced on https://github.com/elastic/gpctl/pull/167
## Summary
The config does not propagate to downstream workflow jobs in all cases
so things break if we don't use the default. Therefore we will make dev
the exception.
Update the gpctl config we reference in the gitops repo. The default
config needs to be the promotion workflow and dev needs to be separate
otherwise other promotion infrastructure breaks since it does not know
to look for the overridden config.
https://github.com/elastic/serverless-gitops/pull/688
Slack: https://elastic.slack.com/archives/C9PPG4EJH/p1693480365606699
Closes#162593Closes#163939Closes#162625
The original intention of this PR was to add FTR support for ESS.
However the scope increased as that also required adding SSL support due
to tests failing from disabled `security` and no authentication.
Additionally, after using serverless in `kbn/es` extensively for this,
there was a bit of friction in regards to DX.
## Summary
- Switch `x-pack/test_serverless` FTR to use ES serverless instead of
(stateful) snapshot
- Adds SSL support to Docker and Serverless in `kbn/es`
- Adds `port` option override
- Adds `teardown` option to kill running nodes if the process exits
without shutdown
- Adds `kill` option to kill running nodes on startup if detected
- Adds `--esFrom serverless` to FTR CLI
- Adds `files` option to mount extra files into containers
- For serverless, automatically attach to first node with `docker logs
-f es01` on startup for better DX.
- Added `background` flag to not attach `logs`.
- Adds graceful shutdown for ESS cluster
- Separate `docker pull` from `run` for better logging, ensures latest
image and stops multiple pulls of the same image occurring in parallel
- Align (most) default settings for ES serverless with `gradlew`
[settings](https://github.com/elastic/elasticsearch-serverless/blob/main/serverless-build-tools/src/main/kotlin/elasticsearch.serverless-run.gradle.kts#L8)
- Fixes Docker bind mount permissions in CI
- Fixes issue where `esFrom` would default to `snapshot` and override
FTR config settings.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Related Issues for Skipped Tests
Security Threat Hunting: #165135
Observability: #165138
Response Ops: #165145
---------
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
## Summary
The FTR serverless pipeline's output can be confusing, because in case
any of the configs fail, the error log is always attached to the last
collapsing block (thus making devs think it was the last block that was
erroneous).
This PR adds an output message after the configs are ran, and separates
the log from the last run's logs.
- chore: Add result logging for serverless_ftr.sh with a new collapsing
block to avoid confusion
Was:
<img width="1086" alt="Screenshot 2023-08-28 at 16 08 43"
src="46721879-8add-49a2-8448-b9ed40ac8bc0">
Would be:
<img width="1079" alt="Screenshot 2023-08-28 at 16 08 20"
src="3eaf7f2b-7fc8-4777-81e3-1c21ec57faaa">
**Part of: https://github.com/elastic/security-team/issues/6726**
## Summary
Migrates the prebuilt rules and timelines status API route schema to
OpenAPI. This is exploratory work to assess the level of effort required
to migrate API route schemas from `io-ts` to `zod` generated by OpenAPI
codegen.
**Summary of the changes:**
- Added a CI job that runs code generation in Security Solution and
comments change if there are any.
- Migrated the `/api/detection_engine/rules/prepackaged/_status` route
to use generated `zod` schemas
- Updated schema tests
- Adjusted the code generator templates to handle `strict` schemas,
i.e., schemas that do not allow any extra params
- Updated the error transformation code to work with zod errors.
Validation errors are converted to string representations, like the
following:
<img width="627" alt="image"
src="93002573-972f-42e1-901d-01a19937f568">
Currently some on merge steps fetch labels from the merged pull request
by parsing the first line of the commit message. This convention usually
works, but it overlaps with the convention of skipping test suites
linking to an issue instead of a pull request. Errors are thrown when a
step attempts to pull metadata from an unset value.
This sets a default value of an empty string if the label metadata is
unset and cleans up the log message when an error occurs.
Example error:
https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/5821#018a1d62-fbd2-486d-afe3-9a448449bbf7/257-258
## Summary
It seems beneficial to have feature flag tests in a separate test config
file
- tests are still run on Kibana CI automatically
- tests are not run on MKI projects automatically, but you can deploy
custom project and run tests via feature flags config
All the feature flags within the same project should be places in the
same config to make sure there is no arguments conflict.
When the flag is moved to the yml configuration, we can rely on Kibana
CI and manually triggered deployment to make sure projects are
functioning correctly.
---------
Co-authored-by: Robert Oskamp <robert.oskamp@elastic.co>
