github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-23 17:28:26 -04:00
Code Issues Projects Releases Packages Wiki Activity

[ResponseOps] Stabilize flaky summary actions tests for MKI (#170511)

Browse source 
## Summary

This PR stabilizes the alerting summary actions test `should schedule
actions for summary of alerts on a custom interval` for MKI runs.

## Details

This test has been reported to be flaky in MKI runs in #169204 and the
fix attempts so far didn't remove the flakiness completely.
I was able to reproduce and added some debug logging to find the root
cause of the flakiness. It turns out, that `waitForDocumentInIndex`
sometimes comes back with three documents (instead of the two expected):
```
[
  {
    _index: 'alert-action-es-query',
    _id: 'ukTylIsBgjaJ6Di2KTpT',
    _score: null,
    _source: {
      all: '1',
      new: '0',
      newIds: '[]',
      ongoing: '1',
      ongoingIds: '[query matched,]',
      recovered: '0',
      recoveredIds: '[]',
      date: '2023-11-03T11:29:38.058Z',
      ruleId: '1f288dc5-ec93-44e4-8508-24ca31bac52e'
    },
    sort: [ 1699010978058 ]
  },
  {
    _index: 'alert-action-es-query',
    _id: 'tUTxlIsBgjaJ6Di2Pzoz',
    _score: null,
    _source: {
      all: '1',
      new: '0',
      newIds: '[]',
      ongoing: '1',
      ongoingIds: '[query matched,]',
      recovered: '0',
      recoveredIds: '[]',
      date: '2023-11-03T11:28:37.578Z',
      ruleId: '1f288dc5-ec93-44e4-8508-24ca31bac52e'
    },
    sort: [ 1699010917578 ]
  },
  {
    _index: 'alert-action-es-query',
    _id: 'eAzwlIsBqzjBaGCRSxxs',
    _score: null,
    _source: {
      all: '1',
      new: '1',
      newIds: '[query matched,]',
      ongoing: '0',
      ongoingIds: '[]',
      recovered: '0',
      recoveredIds: '[]',
      date: '2023-11-03T11:27:34.977Z',
      ruleId: '1f288dc5-ec93-44e4-8508-24ca31bac52e'
    },
    sort: [ 1699010854977 ]
  }
]
```
So it seems due to a search delay, there are already 2 `ongoing` entries
logged, which pushes the `new` entry to index 2 in that array and the
assertion which expects it at index 1 fails.

I think in this test, we don't really care about the number of `ongoing`
items in that array, we just want to make sure that we have a `new`
entry first followed by an `ongoing` entry and we don't care about the
rest of the entries.
With that, I've introduced an optional `sort` parameter to the
`waitForDocumentInIndex` helper that allows to sort `asc` (instead of
the hard coded `desc` so far). That way the test could expect the `new`
entry on index 0, an `ongoing` entry on index 1 and pass no matter how
many `ongoing` entries are following.
This commit is contained in:
Robert Oskamp 2023-11-03 14:32:26 +01:00 • committed by GitHub
parent d89631e7d3
commit 9f3e1f9a01
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 6 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

4
x-pack/test_serverless/api_integration/test_suites/common/alerting/helpers/alerting_wait_for_helpers.ts
View file

@ -19,17 +19,19 @@ export async function waitForDocumentInIndex({
indexName,
ruleId,
num = 1,
sort = 'desc',
}: {
esClient: Client;
indexName: string;
ruleId: string;
num?: number;
sort?: 'asc' | 'desc';
}): Promise<SearchResponse> {
return await pRetry(
async () => {
const response = await esClient.search({
index: indexName,
sort: 'date:desc',
sort: `date:${sort}`,
body: {
query: {
bool: {

5
x-pack/test_serverless/api_integration/test_suites/common/alerting/summary_actions.ts
View file

@ -466,6 +466,7 @@ export default function ({ getService }: FtrProviderContext) {
indexName: ALERT_ACTION_INDEX,
ruleId,
num: 2,
sort: 'asc',
});
const resp2 = await waitForAlertInIndex({
@ -477,7 +478,7 @@ export default function ({ getService }: FtrProviderContext) {
});
expect(resp2.hits.hits.length).to.be(1);
const document = resp.hits.hits[1];
const document = resp.hits.hits[0];
expect(omit(document, '_source.date')._source).to.eql({
all: '1',
new: '1',
@ -489,7 +490,7 @@ export default function ({ getService }: FtrProviderContext) {
ruleId,
});
const document1 = resp.hits.hits[0];
const document1 = resp.hits.hits[1];
expect(omit(document1, '_source.date')._source).to.eql({
all: '1',
new: '0',