[8.6] [APM] Add parallel builds for cypress tests (#142598) (#147930)

# Backport

This will backport the following commits from `main` to `8.6`:
- [[APM] Add parallel builds for cypress tests
(#142598)](https://github.com/elastic/kibana/pull/142598)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Giorgos
Bamparopoulos","email":"georgios.bamparopoulos@elastic.co"},"sourceCommit":{"committedDate":"2022-12-21T15:36:08Z","message":"[APM]
Add parallel builds for cypress tests (#142598)\n\n- Tests runs
initiated by the flaky test runner are not recorded
or\r\nparallelized","sha":"042403e606e59468024b9a822ab7a9c199a487f5","branchLabelMapping":{"^v8.7.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:APM","release_note:skip","v8.6.0","v8.7.0"],"number":142598,"url":"https://github.com/elastic/kibana/pull/142598","mergeCommit":{"message":"[APM]
Add parallel builds for cypress tests (#142598)\n\n- Tests runs
initiated by the flaky test runner are not recorded
or\r\nparallelized","sha":"042403e606e59468024b9a822ab7a9c199a487f5"}},"sourceBranch":"main","suggestedTargetBranches":["8.6"],"targetPullRequestStates":[{"branch":"8.6","label":"v8.6.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.7.0","labelRegex":"^v8.7.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/142598","number":142598,"mergeCommit":{"message":"[APM]
Add parallel builds for cypress tests (#142598)\n\n- Tests runs
initiated by the flaky test runner are not recorded
or\r\nparallelized","sha":"042403e606e59468024b9a822ab7a9c199a487f5"}}]}]
BACKPORT-->

Co-authored-by: Giorgos Bamparopoulos <georgios.bamparopoulos@elastic.co>

.buildkite/pipelines/pull_request/apm_cypress.yml

@@ -5,6 +5,7 @@ steps:
       queue: ci-group-6
     depends_on: build
     timeout_in_minutes: 120
+    parallelism: 4
     retry:
       automatic:
         - exit_status: '*'

.buildkite/scripts/steps/functional/apm_cypress.sh

@@ -10,6 +10,14 @@ APM_CYPRESS_RECORD_KEY="$(retry 5 5 vault read -field=CYPRESS_RECORD_KEY secret/
 .buildkite/scripts/download_build_artifacts.sh
 
 export JOB=kibana-apm-cypress
+IS_FLAKY_TEST_RUNNER=${CLI_COUNT:-0}
+
+# Disable parallel tests and dashboard recording when running them in the flaky test runner
+if [[ "$IS_FLAKY_TEST_RUNNER" -ne 1 ]]; then
+  CYPRESS_ARGS="--record --key "$APM_CYPRESS_RECORD_KEY" --parallel --ci-build-id "${BUILDKITE_BUILD_ID}""
+else
+  CYPRESS_ARGS=""
+fi
 
 echo "--- APM Cypress Tests"
 
@@ -17,5 +25,4 @@ cd "$XPACK_DIR"
 
 node plugins/apm/scripts/test/e2e.js \
   --kibana-install-dir "$KIBANA_BUILD_LOCATION" \
-  --record \
-  --key "$APM_CYPRESS_RECORD_KEY"
+  $CYPRESS_ARGS

x-pack/plugins/apm/dev_docs/testing.md

@@ -70,7 +70,16 @@ node scripts/test/api --runner --basic --updateSnapshots
 
 ## E2E Tests (Cypress)
 
-The E2E tests are located in [`x-pack/plugins/apm/ftr_e2e`](../ftr_e2e)
+The E2E tests are located in [`x-pack/plugins/apm/ftr_e2e`](../ftr_e2e).
+
+Test runs are recorded to the [Cypress Dashboard](https://dashboard.cypress.io). Tests run on buildkite PR pipeline are parallelized (4 parallel jobs) and are orchestrated by the Cypress dashboard service. It can be configured in [.buildkite/pipelines/pull_request/apm_cypress.yml](https://github.com/elastic/kibana/blob/main/.buildkite/pipelines/pull_request/apm_cypress.yml) with the property `parallelism`.
+
+```yml
+    ...
+    depends_on: build
+    parallelism: 4
+    ...
+```
 
 [Test tips and best practices](../ftr_e2e/README.md)