[Security Solution] Unskip rules table auto-refresh Cypress tests (#163451)

**Addresses:** https://github.com/elastic/kibana/issues/154694

## Summary

This PR unskips rules table auto-refresh tests.

## Details

Rules table auto-refresh Cypress tests were outdated so it was hard to determine the original flakiness reason. To make it working the following has been done

- `mockGlobalClock()` moved above `visit()` to mock the time before visiting the page. It didn't look like working so the test had to wait for 60 seconds (our refresh interval)
- removed unnecessary waiting `waitForRulesTableToBeLoaded()` as the test should work without it
- removed `setRowsPerPageTo(5)` as the test doesn't look related to the page size
- `AutoRefreshButtonComponent ` was refactored in attempt to fix popover closing related flakiness but it doesn't seem to help but the refactoring looks improving readability so leaving as a part of this PR
- auto-refresh popover's behavior has been changed. The trigger button gets disabled if there is a rule selected. Clicking the disabled button won't make the auto-refresh popover to appear so this part was changed.

### Encountered issues

- `EuiPopover` should close by clicking the trigger button, clicking outside or pressing `Esc` button. The latter two cause flakiness for some reason. I spent quite some time to investigate the reason but without success. It looks like there is a race condition related to [EuiFocusTrap](https://github.com/elastic/eui/blob/main/src/components/focus_trap/focus_trap.tsx) but it doesn't look to be an issue outside Cypress.
- `waitForRulesTableToBeLoaded()` basically does nothing after changes in the rules table. Looking at tis contents

```ts
export const waitForRulesTableToBeLoaded = () => {
  // Wait up to 5 minutes for the rules to load as in CI containers this can be very slow
  cy.get(RULES_TABLE_INITIAL_LOADING_INDICATOR, { timeout: 300000 }).should('not.exist');
};
```

reveals that `RULES_TABLE_INITIAL_LOADING_INDICATOR` defined as `[data-test-subj="initialLoadingPanelAllRulesTable"]` doesn't exist anymore in our codebase so this function instantly returns. The selector will be fixed in a separate PR.

- invoking `mockGlobalClock()` during the test leads to appearing a lot of `Executing a cancelled action` error messages.


### Flaky test runner

[50 runs succeeded](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2842)
This commit is contained in:
Maxim Palenov 2023-08-11 14:07:35 +02:00 committed by GitHub
parent 26e5c20238
commit 589ef228fb
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 89 additions and 89 deletions

View file

@ -8,20 +8,19 @@
import { import {
RULE_CHECKBOX, RULE_CHECKBOX,
REFRESH_RULES_STATUS, REFRESH_RULES_STATUS,
REFRESH_SETTINGS_SWITCH, RULES_TABLE_AUTOREFRESH_INDICATOR,
REFRESH_SETTINGS_SELECTION_NOTE, RULES_MANAGEMENT_TABLE,
} from '../../../../screens/alerts_detection_rules'; } from '../../../../screens/alerts_detection_rules';
import { import {
checkAutoRefresh,
waitForRulesTableToBeLoaded,
selectAllRules, selectAllRules,
openRefreshSettingsPopover,
clearAllRuleSelection, clearAllRuleSelection,
selectNumberOfRules, selectNumberOfRules,
mockGlobalClock, mockGlobalClock,
disableAutoRefresh, disableAutoRefresh,
checkAutoRefreshIsDisabled, expectAutoRefreshIsDisabled,
checkAutoRefreshIsEnabled, expectAutoRefreshIsEnabled,
expectAutoRefreshIsDeactivated,
expectNumberOfRules,
} from '../../../../tasks/alerts_detection_rules'; } from '../../../../tasks/alerts_detection_rules';
import { login, visit, visitWithoutDateRange } from '../../../../tasks/login'; import { login, visit, visitWithoutDateRange } from '../../../../tasks/login';
@ -29,16 +28,16 @@ import { DETECTIONS_RULE_MANAGEMENT_URL } from '../../../../urls/navigation';
import { createRule } from '../../../../tasks/api_calls/rules'; import { createRule } from '../../../../tasks/api_calls/rules';
import { cleanKibana } from '../../../../tasks/common'; import { cleanKibana } from '../../../../tasks/common';
import { getNewRule } from '../../../../objects/rule'; import { getNewRule } from '../../../../objects/rule';
import { setRowsPerPageTo } from '../../../../tasks/table_pagination';
const DEFAULT_RULE_REFRESH_INTERVAL_VALUE = 60000; const DEFAULT_RULE_REFRESH_INTERVAL_VALUE = 60000;
const NUM_OF_TEST_RULES = 6;
// TODO: See https://github.com/elastic/kibana/issues/154694 describe('Rules table: auto-refresh', () => {
describe.skip('Rules table: auto-refresh', () => {
before(() => { before(() => {
cleanKibana(); cleanKibana();
login(); login();
for (let i = 1; i < 7; i += 1) {
for (let i = 1; i <= NUM_OF_TEST_RULES; ++i) {
createRule(getNewRule({ name: `Test rule ${i}`, rule_id: `${i}` })); createRule(getNewRule({ name: `Test rule ${i}`, rule_id: `${i}` }));
} }
}); });
@ -48,31 +47,31 @@ describe.skip('Rules table: auto-refresh', () => {
}); });
it('Auto refreshes rules', () => { it('Auto refreshes rules', () => {
mockGlobalClock();
visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL); visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded(); expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
// ensure rules have rendered. As there is no user interaction in this test,
// rules were not rendered before test completes
cy.get(RULE_CHECKBOX).should('have.length', 6);
// // mock 1 minute passing to make sure refresh is conducted // // mock 1 minute passing to make sure refresh is conducted
mockGlobalClock(); cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
checkAutoRefresh(DEFAULT_RULE_REFRESH_INTERVAL_VALUE, 'be.visible'); cy.tick(DEFAULT_RULE_REFRESH_INTERVAL_VALUE);
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('be.visible');
cy.contains(REFRESH_RULES_STATUS, 'Updated now'); cy.contains(REFRESH_RULES_STATUS, 'Updated now');
}); });
it('should prevent table from rules refetch if any rule selected', () => { it('should prevent table from rules refetch if any rule selected', () => {
mockGlobalClock();
visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL); visitWithoutDateRange(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded(); expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
selectNumberOfRules(1); selectNumberOfRules(1);
// mock 1 minute passing to make sure refresh is not conducted // mock 1 minute passing to make sure refresh is not conducted
mockGlobalClock(); cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
checkAutoRefresh(DEFAULT_RULE_REFRESH_INTERVAL_VALUE, 'not.exist'); cy.tick(DEFAULT_RULE_REFRESH_INTERVAL_VALUE);
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
// ensure rule is still selected // ensure rule is still selected
cy.get(RULE_CHECKBOX).first().should('be.checked'); cy.get(RULE_CHECKBOX).first().should('be.checked');
@ -82,50 +81,37 @@ describe.skip('Rules table: auto-refresh', () => {
it('should disable auto refresh when any rule selected and enable it after rules unselected', () => { it('should disable auto refresh when any rule selected and enable it after rules unselected', () => {
visit(DETECTIONS_RULE_MANAGEMENT_URL); visit(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
setRowsPerPageTo(5); expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
// check refresh settings if it's enabled before selecting // check refresh settings if it's enabled before selecting
openRefreshSettingsPopover(); expectAutoRefreshIsEnabled();
checkAutoRefreshIsEnabled();
selectAllRules(); selectAllRules();
// auto refresh should be disabled after rules selected // auto refresh should be deactivated (which means disabled without an ability to enable it) after rules selected
openRefreshSettingsPopover(); expectAutoRefreshIsDeactivated();
checkAutoRefreshIsDisabled();
// if any rule selected, refresh switch should be disabled and help note to users should displayed
cy.get(REFRESH_SETTINGS_SWITCH).should('be.disabled');
cy.contains(
REFRESH_SETTINGS_SELECTION_NOTE,
'Note: Refresh is disabled while there is an active selection.'
);
clearAllRuleSelection(); clearAllRuleSelection();
// after all rules unselected, auto refresh should renew // after all rules unselected, auto refresh should be reset to its previous state
openRefreshSettingsPopover(); expectAutoRefreshIsEnabled();
checkAutoRefreshIsEnabled();
}); });
it('should not enable auto refresh after rules were unselected if auto refresh was disabled', () => { it('should not enable auto refresh after rules were unselected if auto refresh was disabled', () => {
visit(DETECTIONS_RULE_MANAGEMENT_URL); visit(DETECTIONS_RULE_MANAGEMENT_URL);
waitForRulesTableToBeLoaded();
setRowsPerPageTo(5);
openRefreshSettingsPopover(); expectNumberOfRules(RULES_MANAGEMENT_TABLE, NUM_OF_TEST_RULES);
disableAutoRefresh(); disableAutoRefresh();
selectAllRules(); selectAllRules();
openRefreshSettingsPopover(); expectAutoRefreshIsDeactivated();
checkAutoRefreshIsDisabled();
clearAllRuleSelection(); clearAllRuleSelection();
// after all rules unselected, auto refresh should still be disabled // after all rules unselected, auto refresh should still be disabled
openRefreshSettingsPopover(); expectAutoRefreshIsDisabled();
checkAutoRefreshIsDisabled();
}); });
}); });

View file

@ -158,7 +158,7 @@ export const RULES_SELECTED_TAG = '.euiSelectableListItem[aria-checked="true"]';
export const SELECTED_RULES_NUMBER_LABEL = '[data-test-subj="selectedRules"]'; export const SELECTED_RULES_NUMBER_LABEL = '[data-test-subj="selectedRules"]';
export const REFRESH_SETTINGS_POPOVER = '[data-test-subj="refreshSettings-popover"]'; export const AUTO_REFRESH_POPOVER_TRIGGER_BUTTON = '[data-test-subj="autoRefreshButton"]';
export const REFRESH_RULES_TABLE_BUTTON = '[data-test-subj="refreshRulesAction-linkIcon"]'; export const REFRESH_RULES_TABLE_BUTTON = '[data-test-subj="refreshRulesAction-linkIcon"]';

View file

@ -12,7 +12,6 @@ import {
DELETE_RULE_ACTION_BTN, DELETE_RULE_ACTION_BTN,
RULES_SELECTED_TAG, RULES_SELECTED_TAG,
RULES_TABLE_INITIAL_LOADING_INDICATOR, RULES_TABLE_INITIAL_LOADING_INDICATOR,
RULES_TABLE_AUTOREFRESH_INDICATOR,
RULE_CHECKBOX, RULE_CHECKBOX,
RULE_NAME, RULE_NAME,
RULE_SWITCH, RULE_SWITCH,
@ -37,7 +36,6 @@ import {
RULES_TAGS_POPOVER_WRAPPER, RULES_TAGS_POPOVER_WRAPPER,
INTEGRATIONS_POPOVER, INTEGRATIONS_POPOVER,
SELECTED_RULES_NUMBER_LABEL, SELECTED_RULES_NUMBER_LABEL,
REFRESH_SETTINGS_POPOVER,
REFRESH_SETTINGS_SWITCH, REFRESH_SETTINGS_SWITCH,
ELASTIC_RULES_BTN, ELASTIC_RULES_BTN,
TOASTER_ERROR_BTN, TOASTER_ERROR_BTN,
@ -57,6 +55,7 @@ import {
TOASTER_CLOSE_ICON, TOASTER_CLOSE_ICON,
ADD_ELASTIC_RULES_EMPTY_PROMPT_BTN, ADD_ELASTIC_RULES_EMPTY_PROMPT_BTN,
CONFIRM_DELETE_RULE_BTN, CONFIRM_DELETE_RULE_BTN,
AUTO_REFRESH_POPOVER_TRIGGER_BUTTON,
} from '../screens/alerts_detection_rules'; } from '../screens/alerts_detection_rules';
import type { RULES_MONITORING_TABLE } from '../screens/alerts_detection_rules'; import type { RULES_MONITORING_TABLE } from '../screens/alerts_detection_rules';
import { EUI_CHECKBOX } from '../screens/common/controls'; import { EUI_CHECKBOX } from '../screens/common/controls';
@ -305,12 +304,6 @@ export const waitForRuleToUpdate = () => {
cy.get(RULE_SWITCH_LOADER, { timeout: 300000 }).should('not.exist'); cy.get(RULE_SWITCH_LOADER, { timeout: 300000 }).should('not.exist');
}; };
export const checkAutoRefresh = (ms: number, condition: string) => {
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should('not.exist');
cy.tick(ms);
cy.get(RULES_TABLE_AUTOREFRESH_INDICATOR).should(condition);
};
export const importRules = (rulesFile: string) => { export const importRules = (rulesFile: string) => {
cy.get(RULE_IMPORT_MODAL).click(); cy.get(RULE_IMPORT_MODAL).click();
cy.get(INPUT_FILE).click({ force: true }); cy.get(INPUT_FILE).click({ force: true });
@ -459,22 +452,45 @@ export const testMultipleSelectedRulesLabel = (rulesCount: number) => {
cy.get(SELECTED_RULES_NUMBER_LABEL).should('have.text', `Selected ${rulesCount} rules`); cy.get(SELECTED_RULES_NUMBER_LABEL).should('have.text', `Selected ${rulesCount} rules`);
}; };
export const openRefreshSettingsPopover = () => { const openRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_POPOVER).click(); cy.get(REFRESH_SETTINGS_SWITCH).should('not.exist');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).click();
cy.get(REFRESH_SETTINGS_SWITCH).should('be.visible'); cy.get(REFRESH_SETTINGS_SWITCH).should('be.visible');
}; };
export const checkAutoRefreshIsDisabled = () => { const closeRefreshSettingsPopover = () => {
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'false'); cy.get(REFRESH_SETTINGS_SWITCH).should('be.visible');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).click();
cy.get(REFRESH_SETTINGS_SWITCH).should('not.exist');
}; };
export const checkAutoRefreshIsEnabled = () => { export const expectAutoRefreshIsDisabled = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.enabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'Off');
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'false');
closeRefreshSettingsPopover();
};
export const expectAutoRefreshIsEnabled = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.enabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'On');
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'true'); cy.get(REFRESH_SETTINGS_SWITCH).should('have.attr', 'aria-checked', 'true');
closeRefreshSettingsPopover();
};
// Expects the auto refresh to be deactivated which means it's disabled without an ability to enable it
// so it's even impossible to open the popover
export const expectAutoRefreshIsDeactivated = () => {
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('be.disabled');
cy.get(AUTO_REFRESH_POPOVER_TRIGGER_BUTTON).should('have.text', 'Off');
}; };
export const disableAutoRefresh = () => { export const disableAutoRefresh = () => {
openRefreshSettingsPopover();
cy.get(REFRESH_SETTINGS_SWITCH).click(); cy.get(REFRESH_SETTINGS_SWITCH).click();
checkAutoRefreshIsDisabled(); expectAutoRefreshIsDisabled();
}; };
export const mockGlobalClock = () => { export const mockGlobalClock = () => {

View file

@ -41,9 +41,14 @@ const AutoRefreshButtonComponent = ({
setIsRefreshOn, setIsRefreshOn,
}: AutoRefreshButtonProps) => { }: AutoRefreshButtonProps) => {
const [isPopoverOpen, setIsPopoverOpen] = useState(false); const [isPopoverOpen, setIsPopoverOpen] = useState(false);
const closePopover = useCallback(() => setIsPopoverOpen(false), [setIsPopoverOpen]);
const togglePopover = useCallback(
() => setIsPopoverOpen((prevState) => !prevState),
[setIsPopoverOpen]
);
const handleAutoRefreshSwitch = useCallback( const handleAutoRefreshSwitch = useCallback(
(closePopover: () => void) => (e: EuiSwitchEvent) => { (e: EuiSwitchEvent) => {
const refreshOn = e.target.checked; const refreshOn = e.target.checked;
if (refreshOn) { if (refreshOn) {
reFetchRules(); reFetchRules();
@ -51,18 +56,35 @@ const AutoRefreshButtonComponent = ({
setIsRefreshOn(refreshOn); setIsRefreshOn(refreshOn);
closePopover(); closePopover();
}, },
[reFetchRules, setIsRefreshOn] [reFetchRules, setIsRefreshOn, closePopover]
); );
const handleGetRefreshSettingsPopoverContent = useCallback( return (
(closePopover: () => void) => ( <EuiPopover
isOpen={isPopoverOpen}
closePopover={closePopover}
button={
<EuiButtonEmpty
data-test-subj="autoRefreshButton"
color={'text'}
iconType={'timeRefresh'}
onClick={togglePopover}
disabled={isDisabled}
css={css`
margin-left: 10px;
`}
>
{isRefreshOn ? 'On' : 'Off'}
</EuiButtonEmpty>
}
>
<EuiContextMenuPanel <EuiContextMenuPanel
items={[ items={[
<EuiSwitch <EuiSwitch
key="allRulesAutoRefreshSwitch" key="allRulesAutoRefreshSwitch"
label={i18n.REFRESH_RULE_POPOVER_DESCRIPTION} label={i18n.REFRESH_RULE_POPOVER_DESCRIPTION}
checked={isRefreshOn ?? false} checked={isRefreshOn ?? false}
onChange={handleAutoRefreshSwitch(closePopover)} onChange={handleAutoRefreshSwitch}
compressed compressed
disabled={isDisabled} disabled={isDisabled}
data-test-subj="refreshSettingsSwitch" data-test-subj="refreshSettingsSwitch"
@ -82,30 +104,6 @@ const AutoRefreshButtonComponent = ({
: []), : []),
]} ]}
/> />
),
[isRefreshOn, handleAutoRefreshSwitch, isDisabled]
);
return (
<EuiPopover
isOpen={isPopoverOpen}
closePopover={() => setIsPopoverOpen(false)}
button={
<EuiButtonEmpty
data-test-subj="autoRefreshButton"
color={'text'}
iconType={'timeRefresh'}
onClick={() => setIsPopoverOpen(!isPopoverOpen)}
disabled={isDisabled}
css={css`
margin-left: 10px;
`}
>
{isRefreshOn ? 'On' : 'Off'}
</EuiButtonEmpty>
}
>
{handleGetRefreshSettingsPopoverContent(() => setIsPopoverOpen(false))}
</EuiPopover> </EuiPopover>
); );
}; };