[8.9] [Security Solution] Unskip rules table auto-refresh Cypress tests (#163451) (#164062)

# Backport

This will backport the following commits from `main` to `8.9`:
- [[Security Solution] Unskip rules table auto-refresh Cypress tests
(#163451)](https://github.com/elastic/kibana/pull/163451)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2023-08-11T12:07:35Z","message":"[Security
Solution] Unskip rules table auto-refresh Cypress tests
(#163451)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/154694\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules table auto-refresh tests.\r\n\r\n##
Details\r\n\r\nRules table auto-refresh Cypress tests were outdated so
it was hard to determine the original flakiness reason. To make it
working the following has been done\r\n\r\n- `mockGlobalClock()` moved
above `visit()` to mock the time before visiting the page. It didn't
look like working so the test had to wait for 60 seconds (our refresh
interval)\r\n- removed unnecessary waiting
`waitForRulesTableToBeLoaded()` as the test should work without it\r\n-
removed `setRowsPerPageTo(5)` as the test doesn't look related to the
page size\r\n- `AutoRefreshButtonComponent ` was refactored in attempt
to fix popover closing related flakiness but it doesn't seem to help but
the refactoring looks improving readability so leaving as a part of this
PR\r\n- auto-refresh popover's behavior has been changed. The trigger
button gets disabled if there is a rule selected. Clicking the disabled
button won't make the auto-refresh popover to appear so this part was
changed.\r\n\r\n### Encountered issues\r\n\r\n- `EuiPopover` should
close by clicking the trigger button, clicking outside or pressing `Esc`
button. The latter two cause flakiness for some reason. I spent quite
some time to investigate the reason but without success. It looks like
there is a race condition related to
[EuiFocusTrap](https://github.com/elastic/eui/blob/main/src/components/focus_trap/focus_trap.tsx)
but it doesn't look to be an issue outside Cypress.\r\n-
`waitForRulesTableToBeLoaded()` basically does nothing after changes in
the rules table. Looking at tis contents\r\n\r\n```ts\r\nexport const
waitForRulesTableToBeLoaded = () => {\r\n // Wait up to 5 minutes for
the rules to load as in CI containers this can be very slow\r\n
cy.get(RULES_TABLE_INITIAL_LOADING_INDICATOR, { timeout: 300000
}).should('not.exist');\r\n};\r\n```\r\n\r\nreveals that
`RULES_TABLE_INITIAL_LOADING_INDICATOR` defined as
`[data-test-subj=\"initialLoadingPanelAllRulesTable\"]` doesn't exist
anymore in our codebase so this function instantly returns. The selector
will be fixed in a separate PR.\r\n\r\n- invoking `mockGlobalClock()`
during the test leads to appearing a lot of `Executing a cancelled
action` error messages.\r\n\r\n\r\n### Flaky test runner\r\n\r\n[50 runs
succeeded](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2842)","sha":"589ef228fb9c2525a49d6abc5db64eb98b2c381e","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","skipped-test","release_note:skip","impact:high","Team:Detections
and Resp","Team: SecuritySolution","Feature:Rule
Management","Team:Detection Rule
Management","backport:prev-minor","v8.10.0"],"number":163451,"url":"https://github.com/elastic/kibana/pull/163451","mergeCommit":{"message":"[Security
Solution] Unskip rules table auto-refresh Cypress tests
(#163451)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/154694\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules table auto-refresh tests.\r\n\r\n##
Details\r\n\r\nRules table auto-refresh Cypress tests were outdated so
it was hard to determine the original flakiness reason. To make it
working the following has been done\r\n\r\n- `mockGlobalClock()` moved
above `visit()` to mock the time before visiting the page. It didn't
look like working so the test had to wait for 60 seconds (our refresh
interval)\r\n- removed unnecessary waiting
`waitForRulesTableToBeLoaded()` as the test should work without it\r\n-
removed `setRowsPerPageTo(5)` as the test doesn't look related to the
page size\r\n- `AutoRefreshButtonComponent ` was refactored in attempt
to fix popover closing related flakiness but it doesn't seem to help but
the refactoring looks improving readability so leaving as a part of this
PR\r\n- auto-refresh popover's behavior has been changed. The trigger
button gets disabled if there is a rule selected. Clicking the disabled
button won't make the auto-refresh popover to appear so this part was
changed.\r\n\r\n### Encountered issues\r\n\r\n- `EuiPopover` should
close by clicking the trigger button, clicking outside or pressing `Esc`
button. The latter two cause flakiness for some reason. I spent quite
some time to investigate the reason but without success. It looks like
there is a race condition related to
[EuiFocusTrap](https://github.com/elastic/eui/blob/main/src/components/focus_trap/focus_trap.tsx)
but it doesn't look to be an issue outside Cypress.\r\n-
`waitForRulesTableToBeLoaded()` basically does nothing after changes in
the rules table. Looking at tis contents\r\n\r\n```ts\r\nexport const
waitForRulesTableToBeLoaded = () => {\r\n // Wait up to 5 minutes for
the rules to load as in CI containers this can be very slow\r\n
cy.get(RULES_TABLE_INITIAL_LOADING_INDICATOR, { timeout: 300000
}).should('not.exist');\r\n};\r\n```\r\n\r\nreveals that
`RULES_TABLE_INITIAL_LOADING_INDICATOR` defined as
`[data-test-subj=\"initialLoadingPanelAllRulesTable\"]` doesn't exist
anymore in our codebase so this function instantly returns. The selector
will be fixed in a separate PR.\r\n\r\n- invoking `mockGlobalClock()`
during the test leads to appearing a lot of `Executing a cancelled
action` error messages.\r\n\r\n\r\n### Flaky test runner\r\n\r\n[50 runs
succeeded](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2842)","sha":"589ef228fb9c2525a49d6abc5db64eb98b2c381e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/163451","number":163451,"mergeCommit":{"message":"[Security
Solution] Unskip rules table auto-refresh Cypress tests
(#163451)\n\n**Addresses:**
https://github.com/elastic/kibana/issues/154694\r\n\r\n##
Summary\r\n\r\nThis PR unskips rules table auto-refresh tests.\r\n\r\n##
Details\r\n\r\nRules table auto-refresh Cypress tests were outdated so
it was hard to determine the original flakiness reason. To make it
working the following has been done\r\n\r\n- `mockGlobalClock()` moved
above `visit()` to mock the time before visiting the page. It didn't
look like working so the test had to wait for 60 seconds (our refresh
interval)\r\n- removed unnecessary waiting
`waitForRulesTableToBeLoaded()` as the test should work without it\r\n-
removed `setRowsPerPageTo(5)` as the test doesn't look related to the
page size\r\n- `AutoRefreshButtonComponent ` was refactored in attempt
to fix popover closing related flakiness but it doesn't seem to help but
the refactoring looks improving readability so leaving as a part of this
PR\r\n- auto-refresh popover's behavior has been changed. The trigger
button gets disabled if there is a rule selected. Clicking the disabled
button won't make the auto-refresh popover to appear so this part was
changed.\r\n\r\n### Encountered issues\r\n\r\n- `EuiPopover` should
close by clicking the trigger button, clicking outside or pressing `Esc`
button. The latter two cause flakiness for some reason. I spent quite
some time to investigate the reason but without success. It looks like
there is a race condition related to
[EuiFocusTrap](https://github.com/elastic/eui/blob/main/src/components/focus_trap/focus_trap.tsx)
but it doesn't look to be an issue outside Cypress.\r\n-
`waitForRulesTableToBeLoaded()` basically does nothing after changes in
the rules table. Looking at tis contents\r\n\r\n```ts\r\nexport const
waitForRulesTableToBeLoaded = () => {\r\n // Wait up to 5 minutes for
the rules to load as in CI containers this can be very slow\r\n
cy.get(RULES_TABLE_INITIAL_LOADING_INDICATOR, { timeout: 300000
}).should('not.exist');\r\n};\r\n```\r\n\r\nreveals that
`RULES_TABLE_INITIAL_LOADING_INDICATOR` defined as
`[data-test-subj=\"initialLoadingPanelAllRulesTable\"]` doesn't exist
anymore in our codebase so this function instantly returns. The selector
will be fixed in a separate PR.\r\n\r\n- invoking `mockGlobalClock()`
during the test leads to appearing a lot of `Executing a cancelled
action` error messages.\r\n\r\n\r\n### Flaky test runner\r\n\r\n[50 runs
succeeded](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2842)","sha":"589ef228fb9c2525a49d6abc5db64eb98b2c381e"}}]}]
BACKPORT-->

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Maxim Palenov 2023-08-21 17:08:33 +02:00 committed by GitHub
parent ad228feeb1
commit 6a21870c4e
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 89 additions and 82 deletions

View file

@ -8,20 +8,19 @@
import {
RULE_CHECKBOX,
REFRESH_RULES_STATUS,
REFRESH_SETTINGS_SWITCH,
REFRESH_SETTINGS_SELECTION_NOTE,
RULES_TABLE_AUTOREFRESH_INDICATOR,
RULES_MANAGEMENT_TABLE,
} from '../../screens/alerts_detection_rules';
import {
checkAutoRefresh,
waitForRulesTableToBeLoaded,
selectAllRules,
openRefreshSettingsPopover,
clearAllRuleSelection,
selectNumberOfRules,
mockGlobalClock,
disableAutoRefresh,
checkAutoRefreshIsDisabled,
checkAutoRefreshIsEnabled,
expectAutoRefreshIsDisabled,
expectAutoRefreshIsEnabled,
expectAutoRefreshIsDeactivated,
expectNumberOfRules,
} from '../../tasks/alerts_detection_rules';
import { login, visit, visitWithoutDateRange } from '../../tasks/login';
@ -29,16 +28,16 @@ import { DETECTIONS_RULE_MANAGEMENT_URL } from '../../urls/navigation';
import { createRule } from '../../tasks/api_calls/rules';
import { cleanKibana } from '../../tasks/common';
import { getNewRule } from '../../objects/rule';
import { setRowsPerPageTo } from '../../tasks/table_pagination';
const DEFAULT_RULE_REFRESH_INTERVAL_VALUE = 60000;
const NUM_OF_TEST_RULES = 6;
// TODO: See https://github.com/elastic/kibana/issues/154694
describe.skip('Alerts detection rules table auto-refresh', () => {
describe('Rules table: auto-refresh', () => {
before(() => {
cleanKibana();
login();
for (let i = 1; i < 7; i += 1) {
for (let i = 1; i <= NUM_OF_TEST_RULES; ++i) {
createRule(getNewRule({ name: `Test rule ${i}`, rule_id: `${i}` }));
}
});
@ -48,31 +47,31 @@ describe.skip('Alerts detection rules table auto-refresh', () => {
});
it('Auto refreshes rules', () => {
mockGlobalClock();
visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
// ensure rules have rendered. As there is no user interaction in this test,
// rules were not rendered before test completes
cy.get(RULE_CHECKBOX).should('have.length', 6);
expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
// // mock 1 minute passing to make sure refresh is conducted
mockGlobalClock();
checkAutoRefresh(DEFAULT_RULE_REFRESH_INTERVAL_VALUE, 'be.visible');
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
cy.tick(DEFAULT_RULE_REFRESH_INTERVAL_VALUE);
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('be.visible');
cy.contains(REFRESH_RULES_STATUS, 'Updated now');
});
it('should prevent table from rules refetch if any rule selected', () => {
mockGlobalClock();
visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
selectNumberOfRules(1);
// mock 1 minute passing to make sure refresh is not conducted
mockGlobalClock();
checkAutoRefresh(DEFAULT_RULE_REFRESH_INTERVAL_VALUE, 'not.exist');
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
cy.tick(DEFAULT_RULE_REFRESH_INTERVAL_VALUE);
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
// ensure rule is still selected
cy.get(RULE_CHECKBOX).first().should('be.checked');
@ -82,50 +81,37 @@ describe.skip('Alerts detection rules table auto-refresh', () => {
it('should disable auto refresh when any rule selected and enable it after rules unselected', () => {
visit(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
setRowsPerPageTo(5);
expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
// check refresh settings if it's enabled before selecting
openRefreshSettingsPopover();
checkAutoRefreshIsEnabled();
expectAutoRefreshIsEnabled();
selectAllRules();
// auto refresh should be disabled after rules selected
openRefreshSettingsPopover();
checkAutoRefreshIsDisabled();
// if any rule selected, refresh switch should be disabled and help note to users should displayed
cy.get(REFRESH_SETTINGS_SWITCH).should('be.disabled');
cy.contains(
REFRESH_SETTINGS_SELECTION_NOTE,
'Note: Refresh is disabled while there is an active selection.'
);
// auto refresh should be deactivated (which means disabled without an ability to enable it) after rules selected
expectAutoRefreshIsDeactivated();
clearAllRuleSelection();
// after all rules unselected, auto refresh should renew
openRefreshSettingsPopover();
checkAutoRefreshIsEnabled();
// after all rules unselected, auto refresh should be reset to its previous state
expectAutoRefreshIsEnabled();
});
it('should not enable auto refresh after rules were unselected if auto refresh was disabled', () => {
visit(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
setRowsPerPageTo(5);
openRefreshSettingsPopover();
expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
disableAutoRefresh();
selectAllRules();
openRefreshSettingsPopover();
checkAutoRefreshIsDisabled();
expectAutoRefreshIsDeactivated();
clearAllRuleSelection();
// after all rules unselected, auto refresh should still be disabled
openRefreshSettingsPopover();
checkAutoRefreshIsDisabled();
expectAutoRefreshIsDisabled();
});
});

View file

@ -169,7 +169,7 @@ export const RULES_SELECTED_TAG = '.euiSelectableListItem[aria-checked="true"]';
export const SELECTED_RULES_NUMBER_LABEL = '[data-test-subj="selectedRules"]';
export const REFRESH_SETTINGS_POPOVER = '[data-test-subj="refreshSettings-popover"]';
export const AUTO_REFRESH_POPOVER_TRIGGER_BUTTON = '[data-test-subj="autoRefreshButton"]';
export const REFRESH_RULES_TABLE_BUTTON = '[data-test-subj="refreshRulesAction-linkIcon"]';

View file

@ -42,7 +42,6 @@ import {
RULES_TAGS_POPOVER_WRAPPER,
INTEGRATIONS_POPOVER,
SELECTED_RULES_NUMBER_LABEL,
REFRESH_SETTINGS_POPOVER,
REFRESH_SETTINGS_SWITCH,
ELASTIC_RULES_BTN,
BULK_EXPORT_ACTION_BTN,
@ -65,6 +64,7 @@ import {
DUPLICATE_WITH_EXCEPTIONS_WITHOUT_EXPIRED_OPTION,
TOASTER_CLOSE_ICON,
ADD_ELASTIC_RULES_EMPTY_PROMPT_BTN,
AUTO_REFRESH_POPOVER_TRIGGER_BUTTON,
} from '../screens/alerts_detection_rules';
import type { RULES_MONITORING_TABLE } from '../screens/alerts_detection_rules';
import { EUI_CHECKBOX } from '../screens/common/controls';
@ -506,22 +506,45 @@ export const testMultipleSelectedRulesLabel = (rulesCount: number) => {
cy.get(SELECTED_RULES_NUMBER_LABEL).should('have.text', `Selected ${rulesCount} rules`);
};
export const openRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_POPOVER).click();
const openRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_SWITCH).should('not.exist');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).click();
cy.get(REFRESH_SETTINGS_SWITCH).should('be.visible');
};
export const checkAutoRefreshIsDisabled = () => {
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'false');
const closeRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_SWITCH).should('be.visible');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).click();
cy.get(REFRESH_SETTINGS_SWITCH).should('not.exist');
};
export const checkAutoRefreshIsEnabled = () => {
export const expectAutoRefreshIsDisabled = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.enabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'Off');
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'false');
closeRefreshSettingsPopover();
};
export const expectAutoRefreshIsEnabled = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.enabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'On');
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'true');
closeRefreshSettingsPopover();
};
// Expects the auto refresh to be deactivated which means it's disabled without an ability to enable it
// so it's even impossible to open the popover
export const expectAutoRefreshIsDeactivated = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.disabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'Off');
};
export const disableAutoRefresh = () => {
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).click();
checkAutoRefreshIsDisabled();
expectAutoRefreshIsDisabled();
};
export const mockGlobalClock = () => {

View file

@ -41,9 +41,14 @@ const AutoRefreshButtonComponent = ({
setIsRefreshOn,
}: AutoRefreshButtonProps) => {
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
const closePopover = useCallback(() => setIsPopoverOpen(false), [setIsPopoverOpen]);
const togglePopover = useCallback(
() => setIsPopoverOpen((prevState) => !prevState),
[setIsPopoverOpen]
);
const handleAutoRefreshSwitch = useCallback(
(closePopover: () => void) => (e: EuiSwitchEvent) => {
(e: EuiSwitchEvent) => {
const refreshOn = e.target.checked;
if (refreshOn) {
reFetchRules();
@ -51,18 +56,35 @@ const AutoRefreshButtonComponent = ({
setIsRefreshOn(refreshOn);
closePopover();
},
[reFetchRules, setIsRefreshOn]
[reFetchRules, setIsRefreshOn, closePopover]
);
const handleGetRefreshSettingsPopoverContent = useCallback(
(closePopover: () => void) => (
return (
<EuiPopover
isOpen={isPopoverOpen}
closePopover={closePopover}
button={
<EuiButtonEmpty
data-test-subj="autoRefreshButton"
color={'text'}
iconType={'timeRefresh'}
onClick={togglePopover}
disabled={isDisabled}
css={css`
margin-left: 10px;
`}
>
{isRefreshOn ? 'On' : 'Off'}
</EuiButtonEmpty>
}
>
<EuiContextMenuPanel
items={[
<EuiSwitch
key="allRulesAutoRefreshSwitch"
label={i18n.REFRESH_RULE_POPOVER_DESCRIPTION}
checked={isRefreshOn ?? false}
onChange={handleAutoRefreshSwitch(closePopover)}
onChange={handleAutoRefreshSwitch}
compressed
disabled={isDisabled}
data-test-subj="refreshSettingsSwitch"
@ -82,30 +104,6 @@ const AutoRefreshButtonComponent = ({
: []),
]}
/>
),
[isRefreshOn, handleAutoRefreshSwitch, isDisabled]
);
return (
<EuiPopover
isOpen={isPopoverOpen}
closePopover={() => setIsPopoverOpen(false)}
button={
<EuiButtonEmpty
data-test-subj="autoRefreshButton"
color={'text'}
iconType={'timeRefresh'}
onClick={() => setIsPopoverOpen(!isPopoverOpen)}
disabled={isDisabled}
css={css`
margin-left: 10px;
`}
>
{isRefreshOn ? 'On' : 'Off'}
</EuiButtonEmpty>
}
>
{handleGetRefreshSettingsPopoverContent(() => setIsPopoverOpen(false))}
</EuiPopover>
);
};