## Summary
in #174284 we split serverless roles into individual files per project.
This PR updates mock-idp-plugin we use to simulate SAML auth in "local"
setup to fetch roles based on `projectType` from the respective yml
file:
- since we read roles from a yml file, new end point `GET
/mock_idp/supported_roles` was added to handle it.
- loginPage & rolesSwitcher hooks are updated to fetch roles via http
- hardcoded roles removed
Few other changes were made:
- move `readRolesFromResource` from `kbn-test` to `kbn-es` for
convenience: it is used to read ES resource files.
- move `system_indices_superuser` to the bottom of the file so it won't
be the first choice in UI role selector (+ we plan to remove the value
in the future)
- Change MOCK_IDP_REALM_NAME to `cloud-saml-kibana` as it is the actual
provider name we use in setup.
<img width="1026" alt="image"
src="060b3c21-9c1a-4210-8b5f-2d574a8f78e7">
<img width="1593" alt="image"
src="4afde9df-5a7e-4ae5-a9f5-ab18b871fc1b">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
To make local/CI testing env closer to MKI experience this PR updates
serverless roles for each project to be defined in its own yml file.
When FTR tests are run, only project specific roles defined in the file
are provisioned. I use `kbnTestServer.serverArgs` from FTR config to get
projectType from `--serverless=<projectType>` and pass it to `kbn-es`.
If you start ES using `es serverless [<args>]` it will provision roles
defined for 'es' project by default, you need to use `--resources` flag
to override it with roles for other project types.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR adds a script that determines GitHub code ownership for
functional test files in the Kibana repository.
### Why do we need this?
We want to be able to determine test ownership to allow teams to get a
better overview of their tests (number of tests, number of skipped
tests, number of failures in the last x days, etc).
### What does this PR bring?
This PR is a first step on closing the test ownership gaps. It adds
functionality to determine the GitHub code owner for a given file (in
the `@kbn/code-owners` package) and adds a script that makes use of this
to check if all functional test files have a code owner, reporting the
gaps.
### Future plans
The idea is to include the test ownership information in our ingested
test results, such that we can create dashboards, reports, etc based on
it.
At some point (once all ownership gaps are closed), we might consider
running this check on CI to prevent new test files without owners.
### How to run?
```
node scripts/check_ftr_code_owners.js
```
The script lists the functional test files that are not covered by code
owners and also gives a summary like this:
```
ERROR Found 2592 test files without code owner (checked 7550 test files in 12.73 s)
```
## Summary
This PR moves SAML session creation from FTR service to `@kbn/test`. It
should simplify its adoption in non-FTR context, e.g. Cypress tests or
jest integration tests:
```
import { SamlSessionManager } from '@kbn/test';
// create instance in your setup file
const sessionManager = new SamlSessionManager({
hostOptions: {
protocol,
hostname,
port,
username,
password,
},
log,
isCloud
});
```
use it in your tests
```
sessionManager.getSessionCookieForRole('viewer');
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Closes#162593Closes#163939Closes#162625
The original intention of this PR was to add FTR support for ESS.
However the scope increased as that also required adding SSL support due
to tests failing from disabled `security` and no authentication.
Additionally, after using serverless in `kbn/es` extensively for this,
there was a bit of friction in regards to DX.
## Summary
- Switch `x-pack/test_serverless` FTR to use ES serverless instead of
(stateful) snapshot
- Adds SSL support to Docker and Serverless in `kbn/es`
- Adds `port` option override
- Adds `teardown` option to kill running nodes if the process exits
without shutdown
- Adds `kill` option to kill running nodes on startup if detected
- Adds `--esFrom serverless` to FTR CLI
- Adds `files` option to mount extra files into containers
- For serverless, automatically attach to first node with `docker logs
-f es01` on startup for better DX.
- Added `background` flag to not attach `logs`.
- Adds graceful shutdown for ESS cluster
- Separate `docker pull` from `run` for better logging, ensures latest
image and stops multiple pulls of the same image occurring in parallel
- Align (most) default settings for ES serverless with `gradlew`
[settings](https://github.com/elastic/elasticsearch-serverless/blob/main/serverless-build-tools/src/main/kotlin/elasticsearch.serverless-run.gradle.kts#L8)
- Fixes Docker bind mount permissions in CI
- Fixes issue where `esFrom` would default to `snapshot` and override
FTR config settings.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Related Issues for Skipped Tests
Security Threat Hunting: #165135
Observability: #165138
Response Ops: #165145
---------
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
This PR addresses Cypress parallelisation in Security Solution scope. It
is a first step in improving e2e tests reliability and gives us a solid
foundation for further work on flakiness and run times. With this PR
each CI job starts 3 instances of ES, kibana and Cypress runner in
isolation.
Other issues addressed in this PR:
- Use click() instead of trigger('click') in all Cypress tests.
- Use testIsolation in all Cypress tests.
- login before each test to ensure that the correct user is always
logged in. Also, with test isolation enabled, login should be required
before each test.
- use visit() instead of cy.visit() in some cases so that tests wait for
the page to fully load before executing.
- Cypress e2e tests are no longer compatible with Firefox (or any
non-Chromium based browser) due to using cypress-real-events
- Removes uses of cypress-pipe. Tests that used cypress-pipe in order to
retry click events will no longer retry click events.
- Fixes an error related to test files being overwritten by Webpack file
processing
---------
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/149344
This PR migrates all plugins to packages automatically. It does this
using `node scripts/lint_packages` to automatically migrate
`kibana.json` files to `kibana.jsonc` files. By doing this automatically
we can simplify many build and testing procedures to only support
packages, and not both "packages" and "synthetic packages" (basically
pointers to plugins).
The majority of changes are in operations related code, so we'll be
having operations review this before marking it ready for review. The
vast majority of the code owners are simply pinged because we deleted
all `kibana.json` files and replaced them with `kibana.jsonc` files, so
we plan on leaving the PR ready-for-review for about 24 hours before
merging (after feature freeze), assuming we don't have any blockers
(especially from @elastic/kibana-core since there are a few core
specific changes, though the majority were handled in #149370).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* [ftr] add first-class support for playwrite journeys
* [CI] Auto-commit changed files from 'node scripts/generate codeowners'
* fix jest test
* remove ability to customize kibana server args, if we need it we can add it back
* remove dev dir that doesn't exist
* fix typo
* prevent duplicated array converstion logic by sharing flag reader
* remove destructuring of option
* fix scalability config and config_path import
* fix start_servers args and tests
* include simple readme
* fix jest tests and support build re-use when changes are just to jest tests
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
