mirror of
https://github.com/elastic/kibana.git
synced 2025-06-28 19:13:14 -04:00
## UPDATE It has been removed the execution of the playwright tests on buildkite, the execution will be re-enabled as soon as we are ready and as described below in the PR, there are still steps pending to be done. ## Motivation **Cypress is not performing well lately.** * We have been facing significant performance issues with Cypress. For instance, it takes a long time to open the visual interface and start executing tests. **Teams are finding it increasingly challenging to write new tests and debug existing ones.** * The time and effort required to create new tests or troubleshoot existing ones have become burdensome. **Concern about the impact this could have on our testing practices.** * Lose motivation to write tests or, worse, skip writing crucial tests. ## Why Playwright? * Compared to Cypress, Playwright seems to be known for its faster execution times and lower resource consumption. What could have a positive impact by having faster feedback during development and execution of new tests as well as more efficient use of CI resources. * Provides powerful debugging tools which can make easier to write, debug and execute tests. * Seems to provide the same capabilities we currently use in our Cypress tests. * Given Playwright's active development and backing by Microsoft, it is likely to continue evolving rapidly, making it a safe long-term choice. Considering all the above, Playwright seems to be a strong candidate to replace Cypress and address all the issues we are facing lately regarding UI test automation. ## Objective of this POC To write in Playwright a couple of tests we currently have on Cypress to check the performance of the tool as well as the development experience. The tests selected have been: - [enable_risk_score_redirect.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/entity_analytics/dashboards/enable_risk_score_redirect.cy.ts) - Owned by Entity Analytics team and selected by its simplicity since it does not need any special setup to be executed and is short. - [manual_rule_run.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_gaps/manual_rule_run.cy.ts) - Owned by Detection Engine team and selected because is short and adds a bit more of complexity due to it needs of clean-up and setting up initial data through the API. ## How to execute the tests ### Visual mode - Navigate to: `x-pack/test/security_solution_playwright` - Execute: `yarn open:ess` for ESS environment or `yarn open:serverless` for serverless environment. ### Headless mode - Navigate to: `x-pack/test/security_solution_playwright` - Execute: `yarn run:ess` for ESS environment or `yarn run:serverless` for serverless environment. ### From VScode - Install `Playwright Test for VScode` extension by Microsoft - Navigate to: `x-pack/test/security_solution_playwright` - Execute: `yarn open:ess` for ESS environment or `yarn open:serverless` for serverless environment. - Open your IDE - Click on the `Testing` icon - On the `Test Explorer` click on the three dots to select the profile you are going to execute `ess` or `serverless` - Click on the test you want to execute or navigate to the spec file of the test and execute it from the same spec. ## My experience - Tests are way easier to implement than with Cypress. - Playwright does not rely on chainable commands. Chainable commands on Cypress can lead to confusing code. - Without chainable commands, the flow of the tests is more explicit and easier to understand. - You can notice that the tool has been designed with Typescript in mind. - Is super easy to implement the Page Object Model pattern (POM). - With POM the test code is clean and focused on "what" rather than "how". - Love the fact that you can execute the tests from the same IDE without having to switch windows during test development. - The visual mode execution gives you lots of information out of the box. ## The scope of this PR - Sets the initial infrastructure to write and execute tests with Playwright. - Has examples and set a basis about how to write tests using the POM. - Allows the execution of the tests in ESS and serverless (just stateless environment). - Integrates the execution of the tests with buildkite. ## Pending to be done/investigate - Proper readme - How to split tests and PO between the different teams - Good reports on CI - Upload screenshots on CI - Flaky test suite runner - Complete the labeling - Execution of the tests on MKI environments ## FAQ **Can I start adding tests to playwright?** Currently, you can explore and experiment with Playwright, but there is still work pending to be done to make the tool officially usable. **Why security engineering productivity is the owner of the playwright folder?** This is something temporary to make sure that good practices are followed. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: dkirchan <diamantis.kirchantzoglou@elastic.co> Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com> Co-authored-by: Jon <jon@budzenski.me>
68 lines
2.1 KiB
TypeScript
68 lines
2.1 KiB
TypeScript
/*
|
|
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
|
|
* or more contributor license agreements. Licensed under the Elastic License
|
|
* 2.0; you may not use this file except in compliance with the Elastic License
|
|
* 2.0.
|
|
*/
|
|
|
|
import { resolve } from 'path';
|
|
import { REPO_ROOT as KIBANA_ROOT } from '@kbn/repo-info';
|
|
import { FtrConfigProviderContext, findTestPluginPaths } from '@kbn/test';
|
|
import { services } from './services';
|
|
import { pageObjects } from './page_objects';
|
|
|
|
// the default export of config files must be a config provider
|
|
// that returns an object with the projects config values
|
|
|
|
export default async function ({ readConfigFile }: FtrConfigProviderContext) {
|
|
const xpackFunctionalConfig = await readConfigFile(
|
|
require.resolve('../functional/config.base.js')
|
|
);
|
|
|
|
return {
|
|
// list paths to the files that contain your plugins tests
|
|
testFiles: [
|
|
resolve(__dirname, './test_suites/resolver'),
|
|
resolve(__dirname, './test_suites/global_search'),
|
|
],
|
|
|
|
services,
|
|
pageObjects,
|
|
|
|
servers: xpackFunctionalConfig.get('servers'),
|
|
|
|
esTestCluster: xpackFunctionalConfig.get('esTestCluster'),
|
|
|
|
kbnTestServer: {
|
|
...xpackFunctionalConfig.get('kbnTestServer'),
|
|
serverArgs: [
|
|
...xpackFunctionalConfig.get('kbnTestServer.serverArgs'),
|
|
`--plugin-path=${resolve(
|
|
KIBANA_ROOT,
|
|
'test/plugin_functional/plugins/core_provider_plugin'
|
|
)}`,
|
|
...findTestPluginPaths(resolve(__dirname, 'plugins')),
|
|
],
|
|
},
|
|
uiSettings: xpackFunctionalConfig.get('uiSettings'),
|
|
// the apps section defines the urls that
|
|
// `PageObjects.common.navigateTo(appKey)` will use.
|
|
// Merge urls for your plugin with the urls defined in
|
|
// Kibana's config in order to use this helper
|
|
apps: {
|
|
...xpackFunctionalConfig.get('apps'),
|
|
resolverTest: {
|
|
pathname: '/app/resolverTest',
|
|
},
|
|
},
|
|
|
|
// choose where screenshots should be saved
|
|
screenshots: {
|
|
directory: resolve(__dirname, 'screenshots'),
|
|
},
|
|
|
|
junit: {
|
|
reportName: 'Chrome X-Pack UI Plugin Functional Tests',
|
|
},
|
|
};
|
|
}
|