kibana/x-pack/test/load
Gloria Hornero 4041d274b3
[ON-WEEK][POC] Playwright (#190803)
## UPDATE
It has been removed the execution of the playwright tests on buildkite,
the execution will be re-enabled as soon as we are ready and as
described below in the PR, there are still steps pending to be done.

## Motivation

**Cypress is not performing well lately.**
* We have been facing significant performance issues with Cypress. For
instance, it takes a long time to open the visual interface and start
executing tests.

**Teams are finding it increasingly challenging to write new tests and
debug existing ones.**
* The time and effort required to create new tests or troubleshoot
existing ones have become burdensome.

**Concern about the impact this could have on our testing practices.**
* Lose motivation to write tests or, worse, skip writing crucial tests.

## Why Playwright?

* Compared to Cypress, Playwright seems to be known for its faster
execution times and lower resource consumption. What could have a
positive impact by having faster feedback during development and
execution of new tests as well as more efficient use of CI resources.

* Provides powerful debugging tools which can make easier to write,
debug and execute tests.

* Seems to provide the same capabilities we currently use in our Cypress
tests.

* Given Playwright's active development and backing by Microsoft, it is
likely to continue evolving rapidly, making it a safe long-term choice.

Considering all the above, Playwright seems to be a strong candidate to
replace Cypress and address all the issues we are facing lately
regarding UI test automation.

## Objective of this POC

To write in Playwright a couple of tests we currently have on Cypress to
check the performance of the tool as well as the development experience.

The tests selected have been:
-
[enable_risk_score_redirect.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/entity_analytics/dashboards/enable_risk_score_redirect.cy.ts)
- Owned by Entity Analytics team and selected by its simplicity since it
does not need any special setup to be executed and is short.

-
[manual_rule_run.cy.ts](https://github.com/elastic/kibana/blob/main/x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine/rule_gaps/manual_rule_run.cy.ts)
- Owned by Detection Engine team and selected because is short and adds
a bit more of complexity due to it needs of clean-up and setting up
initial data through the API.
  
## How to execute the tests

### Visual mode
- Navigate to: `x-pack/test/security_solution_playwright`
- Execute: `yarn open:ess` for ESS environment or `yarn open:serverless`
for serverless environment.

### Headless mode
- Navigate to: `x-pack/test/security_solution_playwright`
- Execute: `yarn run:ess` for ESS environment or `yarn run:serverless`
for serverless environment.

### From VScode
- Install `Playwright Test for VScode` extension by Microsoft
- Navigate to: `x-pack/test/security_solution_playwright`
- Execute: `yarn open:ess` for ESS environment or `yarn open:serverless`
for serverless environment.
- Open your IDE
- Click on the `Testing` icon
- On the `Test Explorer` click on the three dots to select the profile
you are going to execute `ess` or `serverless`
- Click on the test you want to execute or navigate to the spec file of
the test and execute it from the same spec.

## My experience
- Tests are way easier to implement than with Cypress.
- Playwright does not rely on chainable commands. Chainable commands on
Cypress can lead to confusing code.
- Without chainable commands, the flow of the tests is more explicit and
easier to understand.
- You can notice that the tool has been designed with Typescript in
mind.
- Is super easy to implement the Page Object Model pattern (POM).
- With POM the test code is clean and focused on "what" rather than
"how".
- Love the fact that you can execute the tests from the same IDE without
having to switch windows during test development.
- The visual mode execution gives you lots of information out of the
box.

## The scope of this PR
- Sets the initial infrastructure to write and execute tests with
Playwright.
- Has examples and set a basis about how to write tests using the POM.
- Allows the execution of the tests in ESS and serverless (just
stateless environment).
- Integrates the execution of the tests with buildkite.

## Pending to be done/investigate
- Proper readme
- How to split tests and PO between the different teams
- Good reports on CI
- Upload screenshots on CI
- Flaky test suite runner 
- Complete the labeling
- Execution of the tests on MKI environments

## FAQ
**Can I start adding tests to playwright?**
Currently, you can explore and experiment with Playwright, but there is
still work pending to be done to make the tool officially usable.

**Why security engineering productivity is the owner of the playwright
folder?**
This is something temporary to make sure that good practices are
followed.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: dkirchan <diamantis.kirchantzoglou@elastic.co>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Jon <jon@budzenski.me>
2024-09-06 13:09:18 +02:00
..
config.ts Minimize shared-common everywhere (#188606) 2024-07-29 12:47:46 -06:00
runner.ts [ON-WEEK][POC] Playwright (#190803) 2024-09-06 13:09:18 +02:00