mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 01:38:56 -04:00
[scout] Remove perf stats validation for Discover (#215130)
## Summary With #212397 we added 2 tests for Discover app (loading `/app/discover`) to track: - js bundles loaded on page - perf metrics like CPU time, Layout time and Script time fetched with CDP Performance Domain API While the first test for bundles _didn't report any failures_, second test to validate Perf metrics fails periodically: https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34729#0195a4de-6cd5-4d1e-be11-5d02be6de2b0 ``` Error: CPU time (seconds) usage during page navigation should not exceed 1.5 seconds expect(received).toBeLessThan(expected) Expected: < 1.5 Received: 1.591343 ``` https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34877 ``` Error: Additional time spent executing JS scripts should not exceed 0.5 second expect(received).toBeLessThan(expected) Expected: < 0.5 Received: 0.601434 ``` https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34899 ``` Error: Total layout computation time should not exceed 0.06 second expect(received).toBeLessThan(expected) Expected: < 0.06 Received: 0.061723 ``` https://buildkite.com/elastic/kibana-on-merge-unsupported-ftrs/builds/34912#0195adb8-4536-42b7-ab4d-524535fdad9a ``` Error: Additional time spent executing JS scripts should not exceed 0.5 second expect(received).toBeLessThan(expected) Expected: < 0.5 Received: 0.561259 ``` It was worth an experiment, but due to flakiness we decided to keep only bundles limits validation for now and see if it is stable in the long run. If Data-Discovery team has interest in collecting Perf metrics without strict validation in PRs, we can discuss the options. Alternatively we can wait for Scout GA and you can deep dive into your own performance testing with Playwright/CDP.
This commit is contained in:
parent
5c05fb9ad7
commit
4dc27ba4aa
1 changed files with 2 additions and 12 deletions
|
@ -95,6 +95,7 @@ test.describe(
|
|||
page,
|
||||
pageObjects,
|
||||
perfTracker,
|
||||
log,
|
||||
}) => {
|
||||
const beforeMetrics = await perfTracker.capturePagePerformanceMetrics(cdp);
|
||||
|
||||
|
@ -113,18 +114,7 @@ test.describe(
|
|||
afterMetrics
|
||||
);
|
||||
|
||||
expect(
|
||||
perfStats.cpuTime.diff,
|
||||
'CPU time (seconds) usage during page navigation should not exceed 2 seconds'
|
||||
).toBeLessThan(2);
|
||||
expect(
|
||||
perfStats.scriptTime.diff,
|
||||
'Additional time spent executing JS scripts should not exceed 0.5 second'
|
||||
).toBeLessThan(0.5);
|
||||
expect(
|
||||
perfStats.layoutTime.diff,
|
||||
'Total layout computation time should not exceed 0.1 second'
|
||||
).toBeLessThan(0.06);
|
||||
log.info(`Performance Metrics for Discover app: ${JSON.stringify(perfStats, null, 2)}`);
|
||||
});
|
||||
}
|
||||
);
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue