mirror of
https://github.com/elastic/kibana.git
synced 2025-04-22 08:49:27 -04:00
1054799f9c
## Summary This PR attempts (I have lost count of how many there have been so far) to stabilize the flakiness of cases tests. ## Approach Based on our investigations, I believe a common factor that causes all tests to time out is how we wrap the tests with the needed providers. Instead of figuring out why they time out (which is very difficult), I tried a different approach. I rewrote how we initialize the testing dependencies, mocks, and providers on tests. To test my theory, I created a VM instance in GCloud with the same configuration as the one running in the CI, specifically the n2-standard-4 (4 vCPUs, 16 GB Memory) machine type. I ran the tests 100 times, which took almost two days. In 10 of the runs, a random test was timeouted. I noticed that the machine was stressed while running the tests, and the CPU frequently spiked to 100%, especially at the beginning of each test. Then, I increased the timeout for all cases tests to 10 seconds and ran the tests again 100 times. No timeouts occurred. Lastly, I created a VM instance in GCloud with the same configuration as the one running in the CI, specifically the n2-standard-8 (8 vCPUs, 32 GB Memory) machine type. I ran the tests again 100 times. In 1 of the runs, a random test was timeouted. The machine on the CI cannot handle the cases tests. I believe the work in this PR is a step in the right direction either way, and we will benefit from it. I also believe increasing the timeout is a good decision as we need it based on the experiments and the research we have done in the last months. CPU usage on n2-standard-4 https://github.com/user-attachments/assets/36b035df-310f-4906-98ba-688d57b97c7e CPU usage on n2-standard-8 https://github.com/user-attachments/assets/3b1b6351-d48e-41da-a413-a56e52b54b82 ## RLT eslint rules I enabled the [RLT eslint rules](https://testing-library.com/docs/ecosystem-eslint-plugin-testing-library/) for Cases and resolved any eslint errors. The process revealed small bugs in the tests, which I fixed them. ## Testing utils I removed the `appMockRender` and any usage in favor of the new `renderWithTestingProviders` utility function and the improved `TestProviders` component. The `renderWithTestingProviders` follows the principals [suggested](https://github.com/testing-library/react-testing-library/issues/780#issuecomment-689053441) by the RLT team. Specifically: - The `renderWithTestingProviders` is a wrapper of the `render` function of the RTL library. - The `renderWithTestingProviders` does not create the services or any component inside it. - The `renderWithTestingProviders` cannot be used in `beforeEach` functions. It should be called separately on each test. - The `renderWithTestingProviders` accepts props to override the default mocks. - The `renderWithTestingProviders` passes the `TestProviders` in the `wrapper` argument of the RLT `render` function. - The `TestProviders` component initializes and memoizes all services and dependencies. It accepts props to override the default mocks. - Mock overrides (like core services) should be created and passed to `renderWithTestingProviders` on each test, even if it means duplication. We favor test isolation. ### Checklist Check the PR satisfies the following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ## Issues <details> <summary>List</summary> Fixes: https://github.com/elastic/kibana/issues/207712 Fixes: https://github.com/elastic/kibana/issues/192739 Fixes: https://github.com/elastic/kibana/issues/174682 Fixes: https://github.com/elastic/kibana/issues/206366 Fixes: https://github.com/elastic/kibana/issues/207427 Fixes: https://github.com/elastic/kibana/issues/175239 Fixes: https://github.com/elastic/kibana/issues/177334 Fixes: https://github.com/elastic/kibana/issues/208443 Fixes: https://github.com/elastic/kibana/issues/187526 Fixes: https://github.com/elastic/kibana/issues/208310 Fixes: https://github.com/elastic/kibana/issues/192640 Fixes: https://github.com/elastic/kibana/issues/207077 Fixes: https://github.com/elastic/kibana/issues/197304 Fixes: https://github.com/elastic/kibana/issues/207249 Fixes: https://github.com/elastic/kibana/issues/202761 Fixes: https://github.com/elastic/kibana/issues/202115 Fixes: https://github.com/elastic/kibana/issues/193026 Fixes: https://github.com/elastic/kibana/issues/177304 Fixes: https://github.com/elastic/kibana/issues/208415 Fixes: https://github.com/elastic/kibana/issues/174661 Fixes: https://github.com/elastic/kibana/issues/201611 Fixes: https://github.com/elastic/kibana/issues/182364 Fixes: https://github.com/elastic/kibana/issues/175841 Fixes: https://github.com/elastic/kibana/issues/207907 Fixes: https://github.com/elastic/kibana/issues/171177 Fixes: https://github.com/elastic/kibana/issues/196628 Fixes: https://github.com/elastic/kibana/issues/194703 Fixes: https://github.com/elastic/kibana/issues/207241 Fixes: https://github.com/elastic/kibana/issues/206056 Fixes: https://github.com/elastic/kibana/issues/207328 Fixes: https://github.com/elastic/kibana/issues/205953 Fixes: https://github.com/elastic/kibana/issues/176524 Fixes: https://github.com/elastic/kibana/issues/176335 Fixes: https://github.com/elastic/kibana/issues/207404 Fixes: https://github.com/elastic/kibana/issues/207384 Fixes: https://github.com/elastic/kibana/issues/208380 Fixes: https://github.com/elastic/kibana/issues/207248 Fixes: https://github.com/elastic/kibana/issues/207444 Fixes: https://github.com/elastic/kibana/issues/175240 Fixes: https://github.com/elastic/kibana/issues/178001 </details> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| hooks | ||
| pipeline-resource-definitions | ||
| pipeline-utils | ||
| pipelines | ||
| scripts | ||
| .mocharc.json | ||
| .npmrc | ||
| disabled_jest_configs.json | ||
| ftr_base_serverless_configs.yml | ||
| ftr_configs_manifests.json | ||
| ftr_oblt_serverless_configs.yml | ||
| ftr_oblt_stateful_configs.yml | ||
| ftr_platform_stateful_configs.yml | ||
| ftr_search_serverless_configs.yml | ||
| ftr_search_stateful_configs.yml | ||
| ftr_security_serverless_configs.yml | ||
| ftr_security_stateful_configs.yml | ||
| package-lock.json | ||
| package.json | ||
| pull_requests.json | ||
| README.md | ||
| tsconfig.json | ||
| tsconfig.test.json | ||
Kibana / Buildkite
Directory Structure
hooks- special directory used by Buildkite agents for hookspipeline-utils- Shared TypeScript utils for use in pipeline scriptspipelines- contains pipeline definitionsscripts/common- scripts that getsourced by other scripts to set environment variables or import shared functionsscripts/lifecycle- general scripts for tasks that run before or after individual steps or the entire buildscripts/steps- scripts that define something that will run for a step defined in a pipelinescripts/*- all other scripts are building blocks that make up the tasks in pipelines. They may be run by other scripts, but should not besourced