## Summary
Fixes#181684
This PR creates scripts required to run threat hunting tests API tests
in ESS and Serverless mode for Quality Gate periodic run pipeline.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Gloria Hornero <gloria.hornero@elastic.co>
## Summary
This PR is addressing the following issues:
- The pipelines defined in
`.buildkite/pipeline-resource-definitions/security-solution-quality-gate/`
folder were skipping intermediate builds. We need to be able to run more
than one build in the same time for these pipelines.
- As part of the refactoring / optimization of the
`.buildkite/scripts/pipelines/security_solution_quality_gate/api_integration/api-integration-tests.sh`
script, it now executes a TS script in order to handle the projects for
serverless and execute the yarn script provided.
- As part of this refactoring, the methods and worfklow defined in the
`x-pack/plugins/security_solution/scripts/run_cypress/parallel_serverless.ts`
is now followed in order to reduce code duplication and maintenance.
- Fixed an issue in
`x-pack/test/security_solution_api_integration/scripts/index.js`. This
issue was causing false green test executions in buildkite. The exit
code was not actually returned from the child process so the exit code
of this script was 0, even though the child process (test execution) was
failing giving back an exit code 1.
- Parameterized
`.buildkite/pipelines/security_solution/api_integration.yml` to be
running the correct test suite (release or periodic) depending on
whether the environment variable `QUALITY_GATE=1` is passed or not.
The last bullet was misleading the test results interpretation, reading
as successful test runtime scripts which had one or more test failures.
E.g: [Buildkite Test Execution being green with failing
tests.](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-api-integration/builds/307#018f3409-c062-4edf-9663-3ba785823a6c/294-757)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Continues https://github.com/elastic/kibana/pull/179737 effort we are
aligning the tags on Cypress and API to have a unified experience.
## Summary
We want to start integrating our Cypress tests with the serverless
Kibana quality gate. However, not all the teams feel comfortable
enabling all the tests, to facilitate the effort of enabling tests in
the quality gate we are adding the `@serverlessQA` tag, now on API tests
as well.
We use tags to select which tests we want to execute on each environment
and pipeline.
`ess` - runs in ESS env
`serverless` - runs in serverless env and periodic pipeline (failures
don't block release)
`serverlessQA` - runs in kibana release process (failures block release)
`skipInEss` - skipped for ESS env
`skipInServerless` - skipped for all serverless related environments
`skipInServerlessMKI` - skipped for MKI environments
### Description
**Tests tagged as `@serverless`**
All the tests tagged as `@serverless` will be executed as part of the PR
validation process using the serverless FTR environment (not a real
one). That tests will be executed as well as part of the periodic
pipeline using a real serverless project. QA environment is used to do
so using the latest available commit in main at the time of the
execution.
**Tests tagged as `@serverlessQA`**
All the tests tagged as `@serverlessQA` will be executed as part of the
kibana release process using a real serverless project with the latest
image available in the QA environment
**Tests tagged as `@ess`**
All the tests tagged as `@ess` will be executed as part of the PR
validation process using an on-prem ESS environment.
**Tests tagged as `@skipInServerless`**
All the tests tagged as `@skipInServerless` will be excluded from the PR
validation process using the serverless FTR environment, the periodic
pipeline and kibana release process for Serverless.
**Tests tagged as `@skipInEss`**
All the tests tagged as `skipInEss` will be excluded from the PR
validation process using an on-prem ESS environment.
---------
Co-authored-by: Ryland Herrick <ryalnd@gmail.com>
## Summary
While it touches a large number of files, this PR is purely folder
restructure. It is not intended to change any test functionality or
enabled/disable any tests. Updates the D&R FTRs to follow a similar
domain based folder structure like that of cypress. This should greatly
simplify code owners and also allow us to quickly understand different
licensing coverage by looking at the folders.
Co-authored-by: Wafaa Nasr <wafaa.nasr@elastic.co>
