[Cloud Posture] Fix for Flaky test Column Sort (#154877)

## Summary
This PR for fixing the 'Flakiness' for our FTR Column Sort test cases.
The fix includes adding a 1 second delay or sleep between the test
clicking on the column its trying to sort and the test getting values
for that column.

There seems to be a problem from time to time where the values on that
column is not correct for a split second after clicking on the column to
sort it. Adding 1 second delay allows enough time for the column to sort
and show the correct value before we get the value for comparison with
the sorted (using our sorting function on the test)

This delay also fix the issue where the test fails because it checks for
an element while the page is still loading. I ran this 'fix' on our
flaky test runner for over 1000 runs and it hasn't fail so far, whereas
back then It would fail 3-4 times out of 450 runs

### Some other stuff that I tried before choosing this fix includes:

- Breaking the Column sort test cases into smaller test, currently we
test all Column sort in 1 test thus when 1 fail it might fail the next
one too.
- wait until loading element doesn't exist but I seem to be encountering
some issue when i use waitFor. Thus, I switch it to check if loading
element don't exist but It doesnt seem to help that much on fixing
column sort issue

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Rickyanto Ang 2023-04-17 15:18:27 -07:00 committed by GitHub
parent ce912a9231
commit 28b1f96d47
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -175,8 +175,7 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
});
});
// FLAKY: https://github.com/elastic/kibana/issues/152913
describe.skip('Table Sort', () => {
describe('Table Sort', () => {
type SortingMethod = (a: string, b: string) => number;
type SortDirection = 'asc' | 'desc';
// Sort by lexical order will sort by the first character of the string (case-sensitive)
@ -187,6 +186,11 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
return a.localeCompare(b);
};
/* This sleep or delay is added to allow some time for the column to settle down before we get the value and to prevent the test from getting the wrong value*/
const sleep = (num: number) => {
return new Promise((res) => setTimeout(res, num));
};
it('sorts by a column, should be case sensitive/insensitive depending on the column', async () => {
type TestCase = [string, SortDirection, SortingMethod];
const testCases: TestCase[] = [
@ -201,13 +205,19 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) {
];
for (const [columnName, dir, sortingMethod] of testCases) {
await latestFindingsTable.toggleColumnSort(columnName, dir);
/* This sleep or delay is added to allow some time for the column to settle down before we get the value and to prevent the test from getting the wrong value*/
await sleep(1000);
const values = (await latestFindingsTable.getColumnValues(columnName)).filter(Boolean);
expect(values).to.not.be.empty();
const sorted = values
.slice()
.sort((a, b) => (dir === 'asc' ? sortingMethod(a, b) : sortingMethod(b, a)));
values.forEach((value, i) => expect(value).to.be(sorted[i]));
values.forEach((value, i) => {
expect(value).to.be.eql(
sorted[i],
`Row number ${i + 1} missmatch, expected value: ${value}. Instead got: ${sorted[i]}`
);
});
}
});
});