github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-21 16:29:04 -04:00
Code Issues Projects Releases Packages Wiki Activity
kibana/.buildkite
History
Exact
Konrad Szwarc f96d55a4f5
[EDR Workflows] MKI API tests (#187560) 
This pull request introduces two changes to our existing API integration
tests:
1. It restructures the files to follow the security solution-wide
standard.
2. It adds our API integration tests to the periodic MKI pipeline.
[Example
build](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/818)

**Change of Structure:**
All tests have been moved to
`x-pack/test/security_solution_api_integration/test_suites/edr_workflows`
and are grouped by feature and then by licensing.
![Screenshot 2024-07-10 at 11 52
42](223c9138-8702-42f2-a801-a35be87304cb)

**MKI:**
Due to the nature of our tests – their dependence on switching users
and/or modifying internal indices – only 3 out of 7 test suites qualify
to be run in MKI. I've added all test suites to
`.buildkite/pipelines/security_solution_quality_gate/mki_periodic/mki_periodic_defend_workflows.yml`.
However, the ones that would be skipped are commented out to avoid
consuming resources without providing any value.

**Testing for Regression:**
I've noticed that the `@skipInServerlessMKI` tag is not working as
expected. Tests tagged with `@serverless @skipInServerlessMKI Test Name`
were not being run in the PR pipelines. The grep pattern we were using
in individual configs and in
`x-pack/test/security_solution_api_integration/scripts/index.js`
(`'/^(?!.*@skipInServerless).*@serverless.*/'`) would also match
`@skipInServerlessMKI`.

I've modified the pattern to look for a full word, expecting it to be at
the beginning or end of a string, and to be followed or not followed by
a whitespace. We could use unit tests for these grep patterns 😄

Here is a screenshot of the new regex being tested:

![Screenshot 2024-07-10 at 12 09
28](8b9dd49a-3ca5-458d-9567-ad938847f169)

This led me to double-check whether all our API integration tests are
being executed in both PR and MKI pipelines, all seems to be in place:

**MKI:**
1. Artifacts -
[buildkite](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/817#01909bfb-81ae-47c3-a867-b16de4bfa20e/262-380)
- 0 tests executed due to `@skipInServerlessMKI` present in all top
describe of each test file
2. Authentication -
[buildkite](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/817#01909bfb-81b0-4824-a658-3a881607eb56)
- 0 tests executed due to `@skipInServerlessMKI` present in all top
describe of each test file
3. Metadata -
[buildkite](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/817#01909bfb-81b1-4730-9b51-512b1b554f64/261-386)
- 0 tests executed due to `@skipInServerlessMKI` present in all top
describe of each test file
4. Package -
[buildkite](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/817#01909bfb-81b3-418c-ad33-0cd7dd68ad46/261-370)
- 0 tests executed due to `@skipInServerlessMKI` present in all top
describe of each test file
5. Policy Response -
[buildkite](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/817#01909bfb-81b4-4034-a575-3ddbdde42e24/261-422)
- all tests were executed
6. Resolver -
[buildkite](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/817#01909bfb-81b5-482d-8023-e1f819d3c56e/261-711)
- all but the tests with `@skipInServerless` were executed
7. Response actions -
[buildkite](https://buildkite.com/elastic/kibana-serverless-security-solution-quality-gate-defend-workflows/builds/817#01909bfb-81b7-4561-83a3-6896523cff8f/262-403)
- only one file was executed due to the second one being tagged as
`@skipInServerlessMKI`

**PR:**

All tests are accounted for and executed as expected, no regression.
package suite was never executed since it's `.skip`


policy_response/serverless
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-675b-4fab-a787-e5e472711fb0/3394)
policy_response/ess
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-678f-49c6-ae4f-aee3738713c2/3446)
authentication/serverless
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-6768-4330-9b6c-8328a46a5a99/2352)
authentication/ess
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-67b3-4b9b-ba31-110f737a1f3f/1970)
resolver/ess
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-6759-49a3-8cb7-4b0097cf8975/6266)
resolver/serverless
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-676a-4dfe-bc19-6fd50e42980a/3302)
metadata/serverless
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-676c-49ff-b0a5-cf7acc9c5506/4827)
metadata/ess
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-679e-45e8-aa52-5672baf344df/3000)
response_actions/ess
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-67ad-4826-bb58-4b6330fef338/2760)
response_actions/serverless
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-67b7-4a6a-a37d-d138a7054a41/9654)
artifacts/ess
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-67d6-4926-a93f-b193ab2859be/1158)
artifacts/serverless
[buildkite](https://buildkite.com/elastic/kibana-pull-request/builds/220548#01909c0b-672e-4350-8820-c7fd8d7ef010/2328)

---------

Co-authored-by: Angela Chuang <yi-chun.chuang@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Angela Chuang <6295984+angorayc@users.noreply.github.com>
2024-07-12 14:41:41 +02:00
..
hooks [Ops] Refactor env loading & fix agent-targeting rewrite (#178320) 2024-03-12 16:31:26 +01:00
pipeline-resource-definitions [BK] Migrate es-forward (+add versions.json dependent triggering) (#184018) 2024-07-10 12:08:16 +02:00
pipeline-utils [CI] Remove kme leftovers (take 2) (#187947) 2024-07-10 16:01:06 +02:00
pipelines [EDR Workflows] MKI API tests (#187560) 2024-07-12 14:41:41 +02:00
scripts Revert vault-related changes from #187762 (#187990) 2024-07-10 16:36:45 +02:00
.mocharc.json [CI] Move kibana-buildkite-library into kibana (#134787) 2022-06-28 13:19:05 -04:00
.npmrc [CI] Move kibana-buildkite-library into kibana (#134787) 2022-06-28 13:19:05 -04:00
disabled_jest_configs.json [Index Management] Fix broken jest tests (#173029) 2023-12-18 17:21:28 +01:00
ftr_configs.yml [EDR Workflows] MKI API tests (#187560) 2024-07-12 14:41:41 +02:00
package-lock.json [.buildkite] Upgrade braces to 3.0.3 (#186421) 2024-06-24 06:39:18 -05:00
package.json [ci] Upgrade mocha (#178456) 2024-03-12 09:49:02 -05:00
pull_requests.json [CI] Remove kme leftovers (take 2) (#187947) 2024-07-10 16:01:06 +02:00
README.md [CI] Move kibana-buildkite-library into kibana (#134787) 2022-06-28 13:19:05 -04:00
tsconfig.json fix(NA): adds pipeline-resource-definitions/fix-location-collection.ts into a tsconfig 2024-04-04 15:16:01 +01:00
tsconfig.test.json [CI] Move kibana-buildkite-library into kibana (#134787) 2022-06-28 13:19:05 -04:00

README.md

Kibana / Buildkite

Directory Structure

  • hooks - special directory used by Buildkite agents for hooks
  • pipeline-utils - Shared TypeScript utils for use in pipeline scripts
  • pipelines - contains pipeline definitions
  • scripts/common - scripts that get sourced by other scripts to set environment variables or import shared functions
  • scripts/lifecycle - general scripts for tasks that run before or after individual steps or the entire build
  • scripts/steps - scripts that define something that will run for a step defined in a pipeline
  • scripts/* - all other scripts are building blocks that make up the tasks in pipelines. They may be run by other scripts, but should not be sourced