[scout] Remove perf stats validation for Discover (#215130)

## Summary

With #212397 we added 2 tests for Discover app (loading `/app/discover`)
to track:
- js bundles loaded on page
- perf metrics like CPU time, Layout time and Script time fetched with
CDP Performance Domain API

While the first test for bundles _didn't report any failures_, second
test to validate Perf metrics fails periodically:


https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34729#0195a4de-6cd5-4d1e-be11-5d02be6de2b0
```
Error: CPU time (seconds) usage during page navigation should not exceed 1.5 seconds

expect(received).toBeLessThan(expected)

Expected: < 1.5
Received:   1.591343
```


https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34877
```
Error: Additional time spent executing JS scripts should not exceed 0.5 second

expect(received).toBeLessThan(expected)

Expected: < 0.5
Received:   0.601434
```


https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34899
```
Error: Total layout computation time should not exceed 0.06 second

expect(received).toBeLessThan(expected)

Expected: < 0.06
Received:   0.061723
```


https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34912#0195adb8-4536-42b7-ab4d-524535fdad9a
```
Error: Additional time spent executing JS scripts should not exceed 0.5 second

expect(received).toBeLessThan(expected)

Expected: < 0.5
Received:   0.561259
```

It was worth an experiment, but due to flakiness we decided to keep only
bundles limits validation for now and see if it is stable in the long
run.

If Data-Discovery team has interest in collecting Perf metrics without
strict validation in PRs, we can discuss the options. Alternatively we
can wait for Scout GA and you can deep dive into your own performance
testing with Playwright/CDP.
This commit is contained in:
Dzmitry Lemechko 2025-03-22 19:31:54 +01:00 committed by GitHub
parent 5c05fb9ad7
commit 4dc27ba4aa
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -95,6 +95,7 @@ test.describe(
page,
pageObjects,
perfTracker,
log,
}) => {
const beforeMetrics = await perfTracker.capturePagePerformanceMetrics(cdp);
@ -113,18 +114,7 @@ test.describe(
afterMetrics
);
expect(
perfStats.cpuTime.diff,
'CPU time (seconds) usage during page navigation should not exceed 2 seconds'
).toBeLessThan(2);
expect(
perfStats.scriptTime.diff,
'Additional time spent executing JS scripts should not exceed 0.5 second'
).toBeLessThan(0.5);
expect(
perfStats.layoutTime.diff,
'Total layout computation time should not exceed 0.1 second'
).toBeLessThan(0.06);
log.info(`Performance Metrics for Discover app: ${JSON.stringify(perfStats, null, 2)}`);
});
}
);