[Security Solution] Integrates security solution tests with Kibana QA quality gate (#187266)

## Summary

Once this PR is merged, when the Kibana QA quality gate is triggered the
following tests are going to be executed:

- API tests tagged with `@serverlessQA`
- Cypress tests tagged with `@serverlessQA`

Currently there are only a few that have been extensively tested and are
passing on MKI environments.

The current and future tests:
- Cover critical paths of the Security Solution application
- Test behaviours that if broken SDH or blocker issues might be reported
by customers.

We have created a run book that has been shared with all the team
members of Security Solution and can be found in our internal docs with
the title `Security Serverless Kibana QA Quality Gate Failure Run Book`

Some of the most important points mentioned in the run book:

Below are the channels/groups to be contacted should a test fail:
* Detection Engine tests:
  * #security-detection-engine
  * @security-detection-engine-team
  
* Entity Analytics tests:
  *  #security-entity-analytics 
  *  @security-entity-analytics-team
  
* Explore tests:
  * #threat-hunting-explore
  * @security-threat-hunting-explore
  
* Investigations tests: 
  * #security-threat-hunting-investigations 
  * @threat-hunting-investigations
  
* Gen AI tests:
  *  #security-gen-ai-sharing
  *  @security-generative-ai-team
  
* Rule Management tests:
  * #security-detection-rule-management  
  * @security-detection-rule-management-team
  
* Defend Workflows tests:
  * #security-defend-workflows
  * @defend-workflows
  
* Geneve detection rules tests: 
  * #sec-eng-productivity
  * @sec-eng-prod & `@Dom`
  
If nobody responds within 30 minutes: Post the issue in the
#security-unified-app channel with the message that team X is not
responding and assistance is needed. Tag @sec-eng-prod as well.

Should a failure occur, the Release Manager (RM) will post in the team
channell and @ the team responsible. The team, unless in the middle of
responding to an SDH Sev1-2, must respond immediately and begin
investigation with the RM. Serverless test failures in the Production
Quality Release Pipeline are BLOCKING for the entire Kibana Serverless
release.

---------

Co-authored-by: Robert Oskamp <traeluki@gmail.com>
This commit is contained in:
Gloria Hornero 2024-08-29 16:45:32 +02:00 committed by GitHub
parent 2d81ab694e
commit 7d54e4e026
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -14,13 +14,13 @@ steps:
RETRY_TESTS_ON_FAIL: "true"
message: "${BUILDKITE_MESSAGE} (triggered by pipeline.tests-qa.yaml)"
# TODO: Uncomment this code when the integration is ready.
# - label: ":pipeline::female-detective::seedling: Trigger Security Solution quality gate script"
# trigger: security-serverless-quality-gate # https://buildkite.com/elastic/security-serverless-quality-gate
# build:
# env:
# ENVIRONMENT: ${ENVIRONMENT}
# message: "${BUILDKITE_MESSAGE} (triggered by pipeline.tests-qa.yaml)"
- label: ":pipeline::female-detective::seedling: Trigger Security Solution quality gate script"
trigger: security-serverless-quality-gate # https://buildkite.com/elastic/security-serverless-quality-gate
soft_fail: true
build:
env:
ENVIRONMENT: ${ENVIRONMENT}
message: "${BUILDKITE_MESSAGE} (triggered by pipeline.tests-qa.yaml)"
- label: ":ship: Fleet serverless smoke tests for ${ENVIRONMENT}"
trigger: fleet-smoke-tests # https://buildkite.com/elastic/fleet-smoke-tests