# Backport
This will backport the following commits from `main` to `8.6`:
- [[APM] Add parallel builds for cypress tests
(#142598)](https://github.com/elastic/kibana/pull/142598)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Giorgos
Bamparopoulos","email":"georgios.bamparopoulos@elastic.co"},"sourceCommit":{"committedDate":"2022-12-21T15:36:08Z","message":"[APM]
Add parallel builds for cypress tests (#142598)\n\n- Tests runs
initiated by the flaky test runner are not recorded
or\r\nparallelized","sha":"042403e606e59468024b9a822ab7a9c199a487f5","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:APM","release_note:skip","v8.6.0","v8.7.0"],"number":142598,"url":"https://github.com/elastic/kibana/pull/142598","mergeCommit":{"message":"[APM]
Add parallel builds for cypress tests (#142598)\n\n- Tests runs
initiated by the flaky test runner are not recorded
or\r\nparallelized","sha":"042403e606e59468024b9a822ab7a9c199a487f5"}},"sourceBranch":"main","suggestedTargetBranches":["8.6"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/142598","number":142598,"mergeCommit":{"message":"[APM]
Add parallel builds for cypress tests (#142598)\n\n- Tests runs
initiated by the flaky test runner are not recorded
or\r\nparallelized","sha":"042403e606e59468024b9a822ab7a9c199a487f5"}}]}]
BACKPORT-->
Co-authored-by: Giorgos Bamparopoulos <georgios.bamparopoulos@elastic.co>
# Backport
This will backport the following commits from `main` to `8.6`:
- [Add custom Buildkite pipeline for @kbn/handlebars
(#146964)](https://github.com/elastic/kibana/pull/146964)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Thomas
Watson","email":"watson@elastic.co"},"sourceCommit":{"committedDate":"2022-12-08T17:28:33Z","message":"Add
custom Buildkite pipeline for @kbn/handlebars (#146964)\n\nCloses
#146963","sha":"39d27bb868ba5e0d544a0fa17afb19abf650c7bf","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:prev-minor","v8.7.0"],"number":146964,"url":"https://github.com/elastic/kibana/pull/146964","mergeCommit":{"message":"Add
custom Buildkite pipeline for @kbn/handlebars (#146964)\n\nCloses
#146963","sha":"39d27bb868ba5e0d544a0fa17afb19abf650c7bf"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/146964","number":146964,"mergeCommit":{"message":"Add
custom Buildkite pipeline for @kbn/handlebars (#146964)\n\nCloses
#146963","sha":"39d27bb868ba5e0d544a0fa17afb19abf650c7bf"}}]}]
BACKPORT-->
Co-authored-by: Thomas Watson <watson@elastic.co>
## Summary
Last week, our team has dicovered that e2e tests are not executed on CI,
this PR is an attempt to fix that.
Threat intel pipeline should be run whenever the `threat_intelligence`
plugin source or related tests config has changed.
This removes the success requirement in the cloud deployment step for
pull requests.
Creating a cloud deployment from a pull request is optional and not
providing test coverage. Failures are typically upstream and not
something that can be fixed from the Kibana repository.
* [ftr] add first-class support for playwrite journeys
* [CI] Auto-commit changed files from 'node scripts/generate codeowners'
* fix jest test
* remove ability to customize kibana server args, if we need it we can add it back
* remove dev dir that doesn't exist
* fix typo
* prevent duplicated array converstion logic by sharing flag reader
* remove destructuring of option
* fix scalability config and config_path import
* fix start_servers args and tests
* include simple readme
* fix jest tests and support build re-use when changes are just to jest tests
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* add scalability pipeline to run scenarios from APM traces
* update runner description
* remove retry-all-errors curl flag
* add flag for curl to ignore connection refused
* override ES_HOST for gatling runner
* exclude config from regular CI, remove bail flag
* fix uploaded scalability traces
* fix text and remove space
* add validation in config
* use functions instead of scripts
* renaming var in loop
* add step timeout
* define functions before call
* use trap for stopping ES
* fix path for artifacts extraction
* update serverArgs
* add pre-build step
* add pre-build step
* use default pre-build step
* delete step
* print BUILDKITE_PIPELINE_SLUG
* disable telemetry
* remove log
* enable telemetry
* add step to upload test results
* move trap after pid
* upload test reports to gcs
* fix script
* Revert "fix script"
This reverts commit 1c6bc3f45c.
* Revert "upload test reports to gcs"
This reverts commit c957a31c32.
* [perf-tests/scalability] improve logging when uploading traces
* restore including kibana artifacts in scalability dataset
* create parent dirs too
* support skipping builds too
* [ci] Conditional storybook builds
This moves storybook builds to run conditionally, either if there's a
storybook change or the `ci:build-storybooks` label is set.
These builds don't seem to be providing test coverage, and may not be utilized
in scenarios such as automated backports or unrelated changes.
* test regex
* formatting
* Revert "test regex"
This reverts commit dcef465414.
* fix revert
* move to spot instance
* fix merge
* add scalability setup for login and promotion_tracking_dashboard journeys, re-enable step on ci
* update query to get single ftr transaction
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [QA][Code Coverage] Add a link to discover pinned to the current build.
When personnel other than myself wish to visually verify code coverage,
this link will be available in the ci output.
* Drop single quotes.
* Add annotation per J Budz.
* Fixup text.
* Drop debug.
## Summary
Fixes issue with security solution cypress flaky test runner
After introducing [dynamic split for cypress tests in Security Solution](https://github.com/elastic/kibana/pull/125986), there was discovered an issue with flaky test runner, which is using parallelism in slightly different manner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/697#_: tests were evenly split between N jobs, instead of repeating tests N times(as N jobs)
So, for flaky runner, I introducing a new ENV variable that would disable split for tests between parallel jobs and instead would run all test per each job.
Here is link to flaky runner build from this PR, which shows that all test runs per job
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/736
This test was added as a verification step for daily releasable
artifacts. We had originally planned on the release manager consuming
this context. The expected workflow has changed now to us building and
publishing the docker image instead, which already has coverage in the
Build and Deploy to Cloud step.
* [QA][Code Coverage] fixup for auto config handling pr
## Summary
Embed buildkite pipeline definition.
Follow up pr to change cc per auto config handling.
Also, resolves https://github.com/elastic/kibana/issues/132706
Increase worker count for `node scripts/build_kibana_platform_plugins` to 4 workers.
Normalize file names within coverage files such that nyc correctly builds the combined summaries.
_Ci runs this on myriad servers, so the paths are different, which "breaks" nyc's output_
Split the final merge of functional coverage into 2 passes due to [nyc issue](https://github.com/istanbuljs/nyc/issues/1263)
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- adds `parallelism: 4` for security_solution cypress buildkite pipeline
- added parsing /integrations folder with cypress tests, to retrieve paths to individual test files using `glob` utility
- list of test files split equally between agents(there are approx 70+ tests files, split ~20 per job with **parallelism=4**)
- small refactoring of existing cypress runners for `security_solution`
Old metrics(before @MadameSheema https://github.com/elastic/kibana/pull/127558 performance improvements):
before split: average time of completion ~ 1h 40m for tests, 1h 55m for Kibana build
after split in 4 chunks: chunk completion between 20m - 30m, Kibana build 1h 20m
**Current metrics:**
before split: average time of completion ~ 1h for tests, 1h 10m for Kibana build
after split in 4 chunks: each chunk completion between 10m - 20m, 1h Kibana build 1h
* [artifacts] Test cloud context
This artifact currently has no consumers, but in the near future the
release-manager will produce Cloud images using the context. This
ensures builds are working before we publish artifacts.
* update permissions
* fix if statement
* DOCKER_CONTEXT -> KIBANA_DOCKER_CONTEXT
