mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 17:59:23 -04:00
[Security Solution] Fix flakiness in prebuilt_rules_preview.cy.ts
(#211034)
## Summary This PR fixes Rule Management Prebuilt Rules preview Cypress tests flakiness. The flakiness was localized to `prebuilt_rules_preview.cy.ts`. ## Problem details Quite recently Rule Management Prebuilt Cypress tests group started failing due to exceeding 1 hour execution limit. In normal conditions the group takes up to 45 minutes to run all the tests. Investigation revealed the problem. It turned out the real prebuilt rules get installed while it's not expected. The absolute majority of the tests interact with a few prebuilt rule assets mocks to avoid heavy prebuilt rules package installation and installing more than 1K rules from the package. In particular `/internal/detection_engine/prebuilt_rules/_bootstrap` endpoint is invoked upon loading any Security Solution plugin's page and leads to installing a prebuilt rules package. The Cypress test code was organized in way that first the Rule Management page is opened and then API calls interception is set up. Since page loading may vary sometimes real calls to `/internal/detection_engine/prebuilt_rules/_bootstrap` went through. Tests set up prebuilt rule assets mocks but real prebuilt rules package installation wiped out the mocks leading to failing tests. Since Cypress reruns failed tests execution time increases and exceeds the limit.  *`installPrebuiltRuleAssets()` sets up `/internal/detection_engine/prebuilt_rules/_bootstrap` calls interception. ## Flaky test runs **Before:** - `prebuilt_rules_preview.cy.ts` was run in Flaky test runner with 100 iterations. The CI is green but it's easy to notice some jobs took approximately 1 hour to run. 🔴 https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7870 **After:** - `prebuilt_rules_preview.cy.ts` with the fix was run in Flaky test runner with 100 iterations. Execution time approximately 15 minutes for each job. ✅ https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7873 - Rule Management Prebuilt Cypress tests group was run with 100 iterations ✅ https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7875
This commit is contained in:
parent
1ee13cae70
commit
67163cb802
2 changed files with 6 additions and 0 deletions
|
@ -20,6 +20,7 @@ import {
|
|||
installAllPrebuiltRulesRequest,
|
||||
installPrebuiltRuleAssets,
|
||||
createAndInstallMockedPrebuiltRules,
|
||||
preventPrebuiltRulesPackageInstallation,
|
||||
} from '../../../../tasks/api_calls/prebuilt_rules';
|
||||
import { resetRulesTableState } from '../../../../tasks/common';
|
||||
import { login } from '../../../../tasks/login';
|
||||
|
@ -35,6 +36,8 @@ describe(
|
|||
{ tags: ['@ess', '@serverless', '@skipInServerlessMKI'] },
|
||||
() => {
|
||||
beforeEach(() => {
|
||||
preventPrebuiltRulesPackageInstallation();
|
||||
|
||||
login();
|
||||
/* Make sure persisted rules table state is cleared */
|
||||
resetRulesTableState();
|
||||
|
|
|
@ -26,6 +26,7 @@ import { RULE_MANAGEMENT_PAGE_BREADCRUMB } from '../../../../screens/breadcrumbs
|
|||
import {
|
||||
installPrebuiltRuleAssets,
|
||||
createAndInstallMockedPrebuiltRules,
|
||||
preventPrebuiltRulesPackageInstallation,
|
||||
} from '../../../../tasks/api_calls/prebuilt_rules';
|
||||
import { createSavedQuery, deleteSavedQueries } from '../../../../tasks/api_calls/saved_queries';
|
||||
import { fetchMachineLearningModules } from '../../../../tasks/api_calls/machine_learning';
|
||||
|
@ -374,6 +375,8 @@ describe(
|
|||
};
|
||||
|
||||
beforeEach(() => {
|
||||
preventPrebuiltRulesPackageInstallation();
|
||||
|
||||
login();
|
||||
resetRulesTableState();
|
||||
deleteAlertsAndRules();
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue