## Summary
Should fix TS check error `Project references may not form a circular
graph` by removing `@kbn/test-suites-xpack` from `kbn-scout` dependency
list.
Since dockerImage for Fleet package registry is just a constant, that is
used across different FTR and Scout configurations, it makes sense to
export it from `kbn-test`
Closes#194933Closes#192282
## Summary
This PR updates the serverless project yml files to
- enable custom roles for Elasticsearch and Security projects
- enable multiple spaces (max 100) for all serverless project types
### Tests
Additionally, this PR adjust the serverless test suites. Originally,
testing of roles and spaces endpoints was achieved from the feature flag
test config. Now that these features are enabled by default, the tests
have been migrated to the standard serverless test configs.
Affected tests:
-
x-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts
-
x-pack/test_serverless/functional/test_suites/common/platform_security/navigation/management_nav_cards.ts
-
x-pack/test_serverless/functional/test_suites/common/platform_security/roles.ts
-
x-pack/test_serverless/functional/test_suites/common/spaces/spaces_management.ts
-
x-pack/test_serverless/functional/test_suites/common/spaces/spaces_selection.ts
- Feature flag configs/indices
- Project specific configs/indices
- Base serverless config
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Closes https://github.com/elastic/kibana/issues/190343.
This PR aims to use a dockerized package registry version for testing.
In order to test it locally you have to set the value of
`FLEET_PACKAGE_REGISTRY_PORT` env var in your terminal, and you also
need to have a docker daemon running.
For example, you can open a terminal and start the server
```
yarn test:ftr:server --config ./x-pack/test/functional/apps/observability_logs_explorer/config.ts
```
then open a new terminal set the var value and start the runner with the
specific test using this configuration
```
export set FLEET_PACKAGE_REGISTRY_PORT=12345
yarn test:ftr:runner --config ./x-pack/test/functional/apps/observability_logs_explorer/config.ts --include ./x-pack/test/functional/apps/observability_logs_explorer/data_source_selector
```
If you want to test again without the dockerized version, you should
remove the value of the var
```
unset FLEET_PACKAGE_REGISTRY_PORT
```
Closes#188714
## Summary
Adds security response header report-to configuration in serverless test
config to mimic MKI. Adjusts the expected value in the test suite to
capture the expected MKI configuration.
### Tests
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/response_headers.ts
## Summary
Revert of https://github.com/elastic/kibana/pull/178705: `API keys`
request strict validation is already enabled by default in production,
and the decision regarding enabling `Has Privileges` request strict
validation has been reverted and replaced with warnings in response
headers.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR disables TLS mode for Kibana run in serverless.
Related to #170417 enabling serverless roles testing
Blocked by #171513
PR is created in cooperation with @azasypkin and intended to simplify
the automated testing process for serverless:
starting Kibana with TLS enabled adds unnecessary complexity to the
process of getting session cookie and overall Kibana APIs calling with
Dev certificate in the tests.
The selected approach is to disable TLS for Kibana and simply rely on
#171513 to configure mocked idp realm for Serverless ES with TLS
enabled.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR removes the `systemIndicesSuperuser` auth in the `esSupertest`
service for serverless test runs, adds `certificateAuthorities` to the
Elasticsearch server config in serverless and adds a tiny test to verify
functionality of this fix.
### Details
Issues before this PR when using the `esSupertest` service in serverless
tests (can be reproduced by running the added `elasticsearch_api` test
without the supertest and config changes):
1. Running against a local `functional_tests_server`, `esSupertest`
produces an error: `Error: self-signed certificate in certificate chain`
2. Running against an MKI project produces an error: `unable to
authenticate user [system_indices_superuser] for REST request [/]`,
because the `system_indices_superuser` doesn't exist in MKI.
How this PR addresses the issues:
1. Add `certificateAuthorities` to `servers.elasticsearch` in
`x-pack/test_serverless/shared/config.base.ts` in the same way we
already have it for `servers.kibana`
2. Modify the `esSUpertest` service to not override the default ES auth
when running in serverless, instead go with the default auth (regular
superuser), which is the best we can get.
### Additional information
It has been considered to add a serverless specific version of
`esSupertest` in order to avoid the `config.get('serverless') ?` code.
The fact that a number of stateful services are planned to be re-used in
serverless and rely on `esSupertest` in combination with the very small
change in a central service made it seem worth to make an exception
here.
Note that service methods or tests that use `esSupertest` can still run
into issues on serverless if they actually try to modify system indices.
This should be avoided anyways and particularly for serverless tests.
## Summary
Since we're planning to disable login selector in MKI soon, we should
adapt our tests for this new configuration. This PR disables Login
Selector and pushes `Basic` authentication providder down the
authentication chain in Serverless tests.
## Summary
Due to most users needing security plugin enabled by default, this
adjusts to that. Which requires transport SSL to be enabled as well. The
`--ssl` flag now will enable HTTP SSL only.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Closes#162593Closes#163939Closes#162625
The original intention of this PR was to add FTR support for ESS.
However the scope increased as that also required adding SSL support due
to tests failing from disabled `security` and no authentication.
Additionally, after using serverless in `kbn/es` extensively for this,
there was a bit of friction in regards to DX.
## Summary
- Switch `x-pack/test_serverless` FTR to use ES serverless instead of
(stateful) snapshot
- Adds SSL support to Docker and Serverless in `kbn/es`
- Adds `port` option override
- Adds `teardown` option to kill running nodes if the process exits
without shutdown
- Adds `kill` option to kill running nodes on startup if detected
- Adds `--esFrom serverless` to FTR CLI
- Adds `files` option to mount extra files into containers
- For serverless, automatically attach to first node with `docker logs
-f es01` on startup for better DX.
- Added `background` flag to not attach `logs`.
- Adds graceful shutdown for ESS cluster
- Separate `docker pull` from `run` for better logging, ensures latest
image and stops multiple pulls of the same image occurring in parallel
- Align (most) default settings for ES serverless with `gradlew`
[settings](https://github.com/elastic/elasticsearch-serverless/blob/main/serverless-build-tools/src/main/kotlin/elasticsearch.serverless-run.gradle.kts#L8)
- Fixes Docker bind mount permissions in CI
- Fixes issue where `esFrom` would default to `snapshot` and override
FTR config settings.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Related Issues for Skipped Tests
Security Threat Hunting: #165135
Observability: #165138
Response Ops: #165145
---------
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
## Summary
Enable ZDT migrations to run on `migrator` node roles (the default).
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Issues when rolling out Kibana's | Med. | Low | It's possible this
change could lead to interruption in internal Elastic environments |
## Summary
Allow Kibana to restrict the usage of JWT for a predefined set of routes
only in Serverless environment by default. This capability is not
available in non-Serverless environment.
Any route that needs to be accessed in Serverless environemnt using JWT
as a means of authentication should include `security:acceptJWT` tag.
## How to test
If you'd like to generate your own JWT to test the PR, please follow the
steps outlined in
https://github.com/elastic/kibana/pull/159117#issue-1743796706 or just
run functional test server and use static JWT from the Serverless test.
This PR also generated a Serverless Docker image that you can use in
your Dev/QA MKI cluster.
- [x] Implementation functionality and add unit tests
- [x] Update metrics/status routes to include new `security:acceptJWT`
tag
- [x] Update serverless test suite to include a test for
`security:acceptJWT`
__Fixes: https://github.com/elastic/kibana/issues/162632__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#161337
## Summary
Uses build flavor(see #161930) to disable specific Kibana security,
spaces, and encrypted saved objects HTTP API routes in serverless (see
details in #161337). HTTP APIs that will be public in serverless have
been handled in #162523.
**IMPORTANT: This PR leaves login, user, and role routes enabled. The
primary reason for this is due to several testing mechanisms that rely
on basic authentication and custom roles (UI, Cypress). These tests will
be modified to use SAML authentication and serverless roles in the
immediate future. Once this occurs, we will disable these routes.**
### Testing
This PR also implements testing API access in serverless.
- The testing strategy for disabled routes in serverless is to verify a
`404 not found `response.
- The testing strategy for internal access routes in serverless is to
verify that without the internal request header
(`x-elastic-internal-origin`), a `400 bad request response` is received,
then verify that with the internal request header, a `200 ok response`
is received.
- The strategy for public routes in serverless is to verify a `200 ok`
or `203 redirect` is received.
~~blocked by #161930~~
~~blocked by #162149 for test implementation~~
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@elastic.co>
## Summary
It seems beneficial to have feature flag tests in a separate test config
file
- tests are still run on Kibana CI automatically
- tests are not run on MKI projects automatically, but you can deploy
custom project and run tests via feature flags config
All the feature flags within the same project should be places in the
same config to make sure there is no arguments conflict.
When the flag is moved to the yml configuration, we can rely on Kibana
CI and manually triggered deployment to make sure projects are
functioning correctly.
---------
Co-authored-by: Robert Oskamp <robert.oskamp@elastic.co>
## Summary
fixes https://github.com/elastic/kibana/issues/160126
This PR
- hides owner selection while adding lens to case from dashboard for
observability and securitySolution serverless
- hides add to new case and add to existing case lens action for
Elasticsearch serverless
**How to test**
- Run Observability/Securtiy solution serverless project and add lens
visualization to case from dashboard
- Run ES serverless project and check that lens do not have option to
add to case
- Classic kibana works as before (check dashboard from securitySolution
and generic dashboard as well)
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Since we already have some E2E tests running for serverless, this PR
turns on the internal API restriction flag to test whether our UI
functions _as such_ under these tests.
An alternative could be to have a specific smoke test for this, but it
seems this is thoroughly covered by piggy-backing off the existing set
of tests.
Blocks: https://github.com/elastic/kibana/pull/162149
Cypress tests were failing with esArchiver service unavailability.
This PR fixes it by adding basic services (es, kibanaServier, esArchive
& retry) from `@kbn/ftr-common-functional-services` in shared FTR
configuration file. This way all the child config files will have these
services pre-loaded.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR fixes few issues occurring while running FTR API tests against
actual serverless project.
How to run:
```
TEST_CLOUD=1 ES_SECURITY_ENABLED=1 NODE_TLS_REJECT_UNAUTHORIZED=0 TEST_ES_URL=<your_es_url_with_credentials> TEST_KIBANA_URL=<your_es_url_with_credentials> node --no-warnings scripts/functional_test_runner --es-version=8.9.0 --config x-pack/test_serverless/api_integration/test_suites/search/config.ts --bail
```
The first error is faced during Elasticsearch version validation
```
ERROR Error: attempted to use the "es" service to fetch Elasticsearch version info but the request failed: ResponseError: {"ok":false,"message":"Unknown resource."}
at SniffingTransport.request (/Users/dmle/github/kibana/node_modules/@elastic/transport/src/Transport.ts:535:17)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at Client.InfoApi [as info] (/Users/dmle/github/kibana/node_modules/@elastic/elasticsearch/src/api/api/info.ts:60:10)
at FunctionalTestRunner.validateEsVersion (functional_test_runner.ts:129:16)
at functional_test_runner.ts:64:11
at FunctionalTestRunner.runHarness (functional_test_runner.ts:251:14)
at FunctionalTestRunner.run (functional_test_runner.ts:48:12)
at log.defaultLevel (cli.ts:112:32)
at run.ts:70:7
at withProcRunner (with_proc_runner.ts:29:5)
at run (run.ts:69:5)
at FunctionalTestRunner.validateEsVersion (functional_test_runner.ts:131:13)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at functional_test_runner.ts:64:11
at FunctionalTestRunner.runHarness (functional_test_runner.ts:251:14)
at FunctionalTestRunner.run (functional_test_runner.ts:48:12)
at log.defaultLevel (cli.ts:112:32)
at run.ts:70:7
at withProcRunner (with_proc_runner.ts:29:5)
at run (run.ts:69:5)
```
Since there is no version term in case of serverless, we can skip
version check by using newly added to FTR schema `serverless` property
(`false` by default). It is set to `true` in root FTR config
`/shared/config.base`.
The next error is related to ESArchiver relying on `ES` FTR service to
provide ESClient.
```
ResponseError: security_exception
│ Root causes:
│ security_exception: unable to authenticate user [system_indices_superuser] for REST request [/kibana_sample_data_flights]
```
It is fixed by using the default user (from host url) instead of
`system_indices_superuser` we use in stateful run.
After a pull request has been merged, the `kibana-on-merge` pipeline
will
trigger a new pipeline, `kibana-serverless`. This pipeline contains our
serverless tests and will initially be non-blocking.
These tests will be merged into `kibana-on-merge` when proven stable.
https://buildkite.com/elastic/kibana-serverless
When running functional tests off a default distribution, serverless
configuration files do not exist. Instead of relying on the distribution
config folder, this uses the development config folder.
This is intended as a stopgap until tests can run against the serverless
image, which will have the full list of configuration files.
Testing
1) Create a kibana distribution, `node scripts/build`
2) Run a `x-pack/test_serverless` configuration using the
`--kibana-install-dir` flag pointing to the distribution
## Summary
This PR adds boilerplate code and a few initial end-to-end tests to
serverless plugins.
Note that the tests defined in this PR are not part of any CI run yet,
this will be done in a follow-up after this PR is merged.
### Details
The serverless test structure corresponds to what we have in
`x-pack/test` with API tests in `api_integration` and UI tests in
`functional`, each with their set of helper methods and sub-directories
for
- `common` functionality shared across serverless projects (core, shared
UX, ...)
- `observability` project specific functionality
- `search` project specific functionality
- `security` project specific functionality
The `shared` directory contains fixtures, services, ... that are shared
across `api_integration` abd `functional` tests.
```
x-pack/test_serverless/
├─ api_integration
│ ├─ services
│ ├─ test_suites
│ │ ├─ common
│ │ ├─ observability
│ │ ├─ search
│ │ ├─ security
├─ functional
│ ├─ page_objects
│ ├─ services
│ ├─ test_suites
│ │ ├─ common
│ │ ├─ observability
│ │ ├─ search
│ │ ├─ security
├─ shared
│ ├─ services
│ ├─ types
```
See also `x-pack/test_serverless/README.md`
### Run tests
Similar to how functional tests are run in `x-pack/test`, you can point
the functional tests server and test runner to config files in this
`x-pack/test_serverless` directory, e.g. from the `x-pack` directory
run:
```
node scripts/functional_tests_server.js --config test_serverless/api_integration/test_suites/common/config.ts
```
and
```
node scripts/functional_test_runner.js --config test_serverless/api_integration/test_suites/common/config.ts
```
### Additional changes
- The stateful `common_page` page object used the existence of the
global nav to determine `isChromeVisible` and `isChromeHidden`, which is
not working when the global nav is disabled. To solve this, a
`data-test-subj` that indicates the chrome visible state is added to the
Kibana app wrapper and is used for the checks.
- Add a few `data-test-subj` entries to the Observability overview page.
- Add optional `dataTestSubj` to the `Navigation` component and use that
for the serverless search nav.
- Add optional `titleDataTestSubj` to the `SolutionNav` component and
use it for the serverless security nav.
- Add a data-test-subj entry to the Search overview page.
