## Summary
There is no need to start servers (~1.5 min run time) if there are no
tests matching filters or maybe config itself has all tests skipped.
This PR uses Playwright cli with `--list` flag to quickly validate
playwright config and exit with status code `2` (`1` is reserved for
errors during servers start or test failures). it also useful to know in
advance how many tests were about to run:
case 1: tests found
```
$ node scripts/scout.js run-tests --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts --serverless=security
info scout: Test server configuration saved at /Users/dmle/github/kibana/.scout/servers/local.json
info scout: Validate Playwright config has tests
info scout: Total: 5 tests in 2 files
info Verifying Docker is installed.
│ info Docker version 20.10.14, build a224086349
...
```
case 2: no tests found
```
$ node scripts/scout.js run-tests --config x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts --stateful
info scout: Test server configuration saved at /Users/dmle/github/kibana/.scout/servers/local.json
info scout: Validate Playwright config has tests
ERROR scout: No tests found in [x-pack/solutions/observability/plugins/observability_onboarding/ui_tests/playwright.config.ts]
```
## Summary
There's a high likelihood that this causes some unwanted behavior where
the promise is not resolved and the `node` process just exists without
any error.
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 3 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/securitysolution-data-table` |
`x-pack/solutions/security/packages/data-table` |
| `@kbn/ecs-data-quality-dashboard` |
`x-pack/solutions/security/packages/ecs-data-quality-dashboard` |
| `@kbn/security-solution-side-nav` |
`x-pack/solutions/security/packages/side-nav` |
<details >
<summary>Updated references</summary>
```
./.i18nrc.json
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-repo-packages/package-map.json
./tsconfig.base.json
./tsconfig.base.type_check.json
./tsconfig.refs.json
./x-pack/solutions/security/packages/data-table/jest.config.js
./x-pack/solutions/security/packages/ecs-data-quality-dashboard/jest.config.js
./x-pack/solutions/security/packages/side-nav/jest.config.js
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
x-pack/solutions/security/packages/data-table/jest.config.js:11
x-pack/solutions/security/packages/data-table/tsconfig.json:2
x-pack/solutions/security/packages/ecs-data-quality-dashboard/jest.config.js:24
x-pack/solutions/security/packages/ecs-data-quality-dashboard/tsconfig.json:10
x-pack/solutions/security/packages/ecs-data-quality-dashboard/tsconfig.json:2
x-pack/solutions/security/packages/side-nav/jest.config.js:10
x-pack/solutions/security/packages/side-nav/tsconfig.json:2
```
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Most of the packages under `/packages` have been relocated in the
context of _Sustainable Kibana Architecture_.
The remaining packages are `devOnly: true`, and they can be grouped as
follows:
1. Packages that are ONLY used from /scripts/
2. Packages that are used from platform and solutions modules (they are
used from tests, cypress tests, storybook configs, ./scripts/ folders
inside some modules, or other non-prod-time logic).
This PR categorises the packages in (2) as `platform/(private|shared)`
to reflect that they are being used from platform and solutions modules
(even though they're used from non-production code).
Next, we're gonna have to decide whether we want to relocate some of
them (1, 2 or both) under a different path.
## Summary
As part of the SKA, some groups of modules have been broken-down and
relocated in different folders, negatively impacting DX. This is the
case for `serverless` modules, that are spread across:
```
- packages/serverless # storybook config
- src/platform/packages/private/serverless
- src/platform/packages/shared/serverless
```
The goal of this PR is to unify them back under
`src/platform/packages/shared/severless`.
In this scenario, I believe the simplification is worth the trade off.
CC @elastic/kibana-management @elastic/appex-sharedux WDYT?
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
As part of the SKA, some groups of modules have been broken-down and
relocated in different folders, negatively impacting DX. This is the
case for `kbn-management` modules, that are spread across:
```
- packages/kbn-management (storybook config)
- src/platform/packages/private/kbn-management
- src/platform/packages/shared/kbn-management
```
The goal of this PR is to unify them back under
`src/platform/packages/shared/kbn-management`.
In this scenario, I believe the simplification is worth the trade off.
CC @elastic/kibana-management WDYT?
Co-authored-by: Matthew Kime <matt@mattki.me>
## Summary
This PR add a method to wait for map to be loaded to replace generic
`renderable.waitForRender()`.
While investigating the recent test failure on CI I found out that for
maps case we can simplify the logic with few facts:
- before start waiting for render to complete, we need to wait for main
container `#maps-plugin` to be in DOM. It takes 2-3 seconds.
- there is always a single div block with `data-render-complete`
attribute, and there is a comment in source code stating `See if the
"data-render-complete" attribute is "true". If so we're done!` which
means we can simply wait for
`div[data-dom-id][data-render-complete="true"]`
6de2ef0e6d/x-pack/platform/plugins/shared/maps/public/connected_components/map_container/map_container.tsx (L103-L116)
`renderable.waitForRender()` is a good waiter, but probably for
dashboard with multiple panels.
## Summary
This PR aims at relocating some of the Kibana modules (plugins and
packages) into a new folder structure, according to the _Sustainable
Kibana Architecture_ initiative.
> [!IMPORTANT]
> * We kindly ask you to:
> * Manually fix the errors in the error section below (if there are
any).
> * Search for the `packages[\/\\]` and `plugins[\/\\]` patterns in the
source code (Babel and Eslint config files), and update them
appropriately.
> * Manually review
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to ensure that
any CI pipeline customizations continue to be correctly applied after
the changed path names
> * Review all of the updated files, specially the `.ts` and `.js` files
listed in the sections below, as some of them contain relative paths
that have been updated.
> * Think of potential impact of the move, including tooling and
configuration files that can be pointing to the relocated modules. E.g.:
> * customised eslint rules
> * docs pointing to source code
> [!NOTE]
> * This PR has been auto-generated.
> * Any manual contributions will be lost if the 'relocate' script is
re-run.
> * Try to obtain the missing reviews / approvals before applying manual
fixes, and/or keep your changes in a .patch / git stash.
> * Please use
[#sustainable_kibana_architecture](https://elastic.slack.com/archives/C07TCKTA22E)
Slack channel for feedback.
Are you trying to rebase this PR to solve merge conflicts? Please follow
the steps describe
[here](https://elastic.slack.com/archives/C07TCKTA22E/p1734019532879269?thread_ts=1734019339.935419&cid=C07TCKTA22E).
#### 3 packages(s) are going to be relocated:
| Id | Target folder |
| -- | ------------- |
| `@kbn/response-ops-alerts-apis` |
`src/platform/packages/shared/response-ops/alerts-apis` |
| `@kbn/response-ops-alerts-fields-browser` |
`src/platform/packages/shared/response-ops/alerts-fields-browser` |
| `@kbn/response-ops-alerts-table` |
`src/platform/packages/shared/response-ops/alerts-table` |
<details >
<summary>Updated references</summary>
```
./package.json
./packages/kbn-ts-projects/config-paths.json
./src/platform/packages/private/kbn-repo-packages/package-map.json
./src/platform/packages/shared/response-ops/alerts-apis/jest.config.js
./src/platform/packages/shared/response-ops/alerts-fields-browser/jest.config.js
./src/platform/packages/shared/response-ops/alerts-table/jest.config.js
./tsconfig.base.json
./yarn.lock
.github/CODEOWNERS
```
</details><details >
<summary>Updated relative paths</summary>
```
src/platform/packages/shared/response-ops/alerts-apis/jest.config.js:12
src/platform/packages/shared/response-ops/alerts-apis/tsconfig.json:2
src/platform/packages/shared/response-ops/alerts-fields-browser/jest.config.js:12
src/platform/packages/shared/response-ops/alerts-fields-browser/tsconfig.json:2
src/platform/packages/shared/response-ops/alerts-table/jest.config.js:12
src/platform/packages/shared/response-ops/alerts-table/tsconfig.json:2
```
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Umberto Pepato <umbopepato@users.noreply.github.com>
## Summary
Closes#210340
This PR adds synthtrace clients to scout as a test fixture, so you can
use it in your test to generate data.
The clients added were `apmSynthtraceEsClient`,
`infraSynthtraceEsClient` and `otelSynthtraceEsClient`.
## How to use them in parallel tests
As `synthtrace` ingests data into our indices, and sequential runs would
be the perfect way to introduce flakiness in our tests, there is a
better way to ingest data, using a hook, at the setup phase with
`globalSetup`.
We need to create a `global_setup.ts` file and link it into our
playwright config.
Then we can use something like
```
async function globalSetup(config: FullConfig) {
const data = {
apm: [
opbeans({
from: new Date(start).getTime(),
to: new Date(end).getTime(),
}),
],
infra: [
generateHosts({
from: new Date(start).toISOString(),
to: new Date(end).toISOString(),
}),
],
otel: [
sendotlp({
from: new Date(start).getTime(),
to: new Date(end).getTime(),
}),
],
};
return ingestSynthtraceDataHook(config, data);
}
```
Each key (apm, infra, otel) accepts an array of generators.
## How to use them in sequential tests
> [!WARNING]
> This should not be the standard behaviour, we should embrace
parallelism and use sequential testing when there is no other way.
### apmSynthtraceEsClient
```ts
test.before(
async ({ apmSynthtraceEsClient }) => {
await apmSynthtraceEsClient.index(
opbeans({
from: new Date(start).getTime(),
to: new Date(end).getTime(),
})
);
}
);
```
[opbeans
file](https://github.com/elastic/kibana/blob/main/x-pack/solutions/observability/plugins/apm/ftr_e2e/cypress/fixtures/synthtrace/opbeans.ts)
used in the example.
### otelSynthtraceEsClient
```ts
test.before(
async ({otelSynthtraceEsClient }) => {
await otelSynthtraceEsClient.index(
sendotlp({
from: new Date(start).getTime(),
to: new Date(end).getTime(),
})
);
}
);
```
[sendotlp
file](https://github.com/elastic/kibana/blob/main/x-pack/solutions/observability/plugins/apm/ftr_e2e/cypress/fixtures/synthtrace/sendotlp.ts)
which will create the data.
### infraSynthtraceEsClient
```ts
test.before(
async ({ infraSynthtraceEsClient }) => {
await infraSynthtraceEsClient.index(
generateHosts({
from: new Date(start).toISOString(),
to: new Date(end).toISOString(),
})
);
}
);
```
[generateHosts
file](https://github.com/elastic/kibana/blob/main/x-pack/solutions/observability/plugins/inventory/e2e/cypress/e2e/alert_count/generate_data.ts#L82)
used to generate data.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR solves an issue detected in the populate bundle cache plugin
after the webpack v5 migration. On the new version webpack v5 returns a
lot of incomplete paths when we walk over file dependencies or internal
modules. The heuristic logic used previously was faulty so the fixes
turns to use a cached filesystem api instead.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/21030
It was a quick work to do while we don't have many tests yet.
For reviewers: we most likely will review and update the rules to align
better with final test design for Scout before GA. I don't think we have
to deep dive into what rules are missing, but just to make sure I didn't
restrict something important from your perspective.
Rules are described in
https://github.com/playwright-community/eslint-plugin-playwright?tab=readme-ov-file#rules
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
### Summary
- Closes#89741
This PR contains the resulting work of a massive effort that ports our
on top bundler abstraction (called @kbn/optimizer) from Webpack v4 into
Webpack v5. It's essential in terms of long term maintenance since v4
was not receiving updates any longer but will also unblock some new
features that could be beneficial for our future DevEx endeavours.
Next you can find a small list of all the accomplished tasks on this
journey.
### Completed Tasks
- [x] Upgrade dependencies to match the ones on webpack v5
- [x] Fix null-loader usages
- [x] Fix raw-loader usages
- [x] Fix file-loader usages
- [x] Fix url-loader usages
- [x] Fix `@kbn/optimizer-webpack-helpers` to support webpack v5
- [x] Adopt previous webpack v4 polyfill-all strategy with
node-polyfill-webpack-plugin
- [x] Fix theme-loader on @kbn/optimizer
- [x] Migrate configurations and ad-hoc loader options on all webpack
configs from v4 to v5
- [x] Fix @kbn/test jest resolver for file-loader cases
- [x] Migrate public-path loader on UiSharedDeps
- [x] Fix all usages of webpack-merge
- [x] Migrate BundleRemoteModule
- [x] Migrate BundleRemotesPlugin
- [x] Correctly migrate PopulateBundleCachePlugin
- [x] Correctly migrate BundleMetricsPlugin
- [x] Check if the profiling plugins still work (--profile flag)
- [x] Recover if possible the previous webpack v4 cacheGroup chunks
rename to something like `data.plugin.chunk.0.js`
- [x] Run `/ci` and make sure we get our first green CI, otherwise work
on the errors until we do
- [x] Profile and solve bottlenecks until we get a cold build
performance similar to the one we had on webpack v4 (`node
scripts/build_kibana_platform_plugins --no-cache`).
- [x] OpenSSL Legacy Warnings: try to remove `--openssl-legacy-provider
` flags
- [x] Add Webpack to Renovate config
- [x] Explore removing `NodePolyfillPlugin`
([here](https://www.npmjs.com/package/node-polyfill-webpack-plugin)) and
add each polyfill needed individually per each webpack config to check
if we get smaller bundles. If we do it's better to go with the case by
case need approach instead of deploying a bunch of polyfills with
NodePolyfillPlugin. As another alternative, create a custom smaller
plugin with only the union of all needed polyfills.
- [x] Evaluate if we want to touch the resolutions on mainFields and
conditionNames
- [x] Understand why `@import 'src/core/public/mixins'` does not work
anymore (not a problem, we should use relative paths anyway but we want
to track why it changed from v4 to v5)
- [x] BUG: Child compilers are having errors hidden and/or changed from
error to warning
- [x] Fix license check for
[Artistic-2.0](https://spdx.org/licenses/Artistic-2.0.html) is the
license for
[domain-browser](https://github.com/bevry/domain-browser?tab=License-1-ov-file).
This package is a dependency of
[NodePolyfillPlugin](https://www.npmjs.com/package/node-polyfill-webpack-plugin).
Artistic 2.0 license is [classified as
yellow](https://github.com/elastic/open-source/blob/main/elastic-product-policy.md#yellow-list)
and should only be used for dev dependencies.
- [x] Make sure `resourceQuery: { not: /raw/ }` is not necessary on
other webpack configs like storybook one
- [x] Find what is being wrongly removed by usedExports optimization;
hint: I believe it is identifying a lot of exports inside the sync entry
of plugins as unused exports and removing them. Then `__kbnBootstrap__`
can't be found
- [x] Rebalance @kbn/optimizer pickMaxWorkerCount
- [x] Re-open the issue to fix sass-warnings
[#190345](https://github.com/elastic/kibana/issues/190345) or downgrade
sass-loader to v10
- [x] Remove previous esm no parse rules
- [x] Confirm esm support is working
- [x] Confirm console override is needed
- [x] Confirm react prod builds on ui shared deps for distributable
- [x] Remove customization for
[xyflow](https://github.com/xyflow/xyflow) from webpack configs
- [x] Clean all the code
- [x] Make sure collected metrics from stats are still aligned with what
we were collecting before; also verify if the modules used for optimizer
caches etc are well generated (@kbn/node-libs-browser)
- [x] Fix watch performance
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Brad White <brad.white@elastic.co>
Addresses #206436
- Replace styled-components with @emotion/react and @emotion/styled
across multiple components
- Remove `@kbn/ui-theme` instances
- Update test cases to use @emotion/jest matchers for toHaveStyleRule to
work properly
- Adjust styles to use `css` from `@emotion/react`
- Remove unused imports and updated snapshots
https://github.com/user-attachments/assets/48b9ab41-23fd-4c00-ace9-c0cbe58c91f7
## Summary
This PR adds a test config category to the scout reporting. This allows
us to distinguish between UI and API FTR tests.
A new property `testConfigCategory` has been added to all FTR configs
that don't already inherit it from a higher level config.
## Summary
This PR turns the AlertsTable into a standalone component, making it
independent from the `TriggersActionsUI` plugin.
#### Removes the alerts table registry
All configuration is now managed through the AlertsTable component
props. Shared configurations are handled by giving consumers the ability
to directly provide alerts table wrapper components (see for example the
`renderAlertsTable` prop of `getCases`).
#### Moves the alerts table to dedicated package(s)
Following the feature-driven structure we're introducing for ResponseOps
(alerting) client-side packages:
- `@kbn/response-ops-alerts-table`
- `@kbn/response-ops-alerts-apis`
- `@kbn/response-ops-alerts-fields-browser`
#### Initial work on improving composition and organization
- Reorganizes the table code into a by-entity-type folder structure
(`components/`, `hooks/`, ...)
- Simplifies some components and breaks into smaller units when possible
## To verify
For consumers of the alerts table:
- Check that all your tables have the same behavior as before (columns,
sort, row actions, bulk actions, etc.)
- Check that your "shared" tables (i.e. cases alerts view in O11y and
Security) have the expected configuration and behavior
> [!WARNING]
> This PR moves a lot of files. Git might not always recognize the
correct delete/add file pairs. If you see weird diffs feel free to reach
out for help!
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
### Identify risks
| Risk | Description | Severity | Mitigation |
|---|---|---|---|
| Table misconfigurations | Some table configurations might slightly
differ from the previous AlertsTableRegistry-backed version | Low |
Quick fix |
## References
Closes#195180
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christos Nasikas <xristosnasikas@gmail.com>
## Summary
- [x] Removed use of color calculating function `transparentize()`
- [x] Begin removal of JSON tokens when possible in favor of colors
tokens
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR is probably the final PR that makes the changes to support EUI
Borealis. It focuses on removing all the usage of EUI Json tokens.
You will notice different approaches while removing the tokens:
- for some cases, the changes were done using `css from
'@emotions/react'` as the components using the tokens were already using
`euiTheme` or adding it was straightforward and required the minimal
amount of changes
- for some cases, where the css changes were pretty involved, a hook was
created to be able to import the styles and apply them in the components
- finally for other cases, esepcially if the styled components were
extracted in a different file and were used within many others, I
decided to create reusable components. This allowed to not change all
the files impacted and limit the number of files modified in this PR.
Feel free to comment on any of the approaches and suggest better
options!
https://github.com/elastic/kibana/issues/201889
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
`@kbn/scout-oblt` is a test library that extends `@kbn/scout` with test
helpers specifically designed to test `Observability` applications in
Kibana. All Oblt plugins should only import from `@kbn/scout-oblt`
Its primary goal is to simplify the test development experience for
teams working on `Observability` plugins by providing custom Playwright
fixtures, page objects, and utilities tailored for Observability-related
testing scenarios.
Contributing:
- when Fixture/Page Object is sharable across all Solutions and Platform
(`fleetApi` fixture), it should be added in `@kbn/scout`
- when Fixture/Page Object is Oblt-specific but is shared across tests
under the multiple plugins (`OnboardingHome` page), it should be added
in `@kbn/scout-oblt`
- when Fixture/Page Object is only used in a single plugin (`onboarding`
internal APIs ?), it should be added in this plugin.
I also re-worked existing tests with few ideas in mind:
- Scout is **e2e testing tool** and should target primary e2e test
scenarios; We have _API integration tests_ to test multiple short
scenarios for APIs behavior (response, status code) and _jest/React
testing library_ to test components in isolation (elements rendering,
fields validation). Doing all the testing with e2e tool like Playwright
will dramatically affect cost efficiency and stability of tests, but
also slows overall CI execution and PRs delivery. The goal is to follow
testing pyramid and keep in mind its principles.
- We on purpose spin up new browser context for each `test` block to
make sure our **tests are independent**. Having too many short `test`
blocks in the file significantly slows down the execution: every block
triggers browser context, saml authentication, adding/removing Fleet
integrations (each call up to 2 seconds) and other beforeEach/afterEach
hooks. Real browser-based testing is expensive. It is not about putting
every step into 1 `test` block, but also not a Jest unit-test-style
design. When it is possible to group similar actions on the same page
and if it is a part of the same user flow - we should do it. It also
doesn't bring the testing value repeating the same UI steps multiple
times in different scenarios. _Our CI costs are critical to cut when it
is possible_
- Avoid **nesting describe** blocks: it complicates test readability and
also complicates for CI bot to properly skip the failing block (it will
skip the top level one). We encourage **Scout parallel test execution**
based on running test spec files in multiple workers, not the `test`
blocks within the same file. Having too many `test` blocks in the same
file will be slowly run in the single thread and in case of flakiness,
it means Team lose more test coverage than they probably expect.
Before (**59** test blocks - **8-8.5 min** per distro):
<img width="1709" alt="Screenshot 2025-02-08 at 18 01 40"
src="https://github.com/user-attachments/assets/5fd65a1c-85f9-4594-9dae-3f8e99a005ab"
/>
After (**15** test blocks - **3.5-4 min** per distro):
<img width="1578" alt="Screenshot 2025-02-10 at 18 14 42"
src="https://github.com/user-attachments/assets/6846898f-7dd2-4f6b-8bc5-d06741b0b120"
/>
For reviewers: updated tests are possible to run in 2 parallel workers
against the same Kibana/ES instance and run time is dropping to **2.5-3
min** 🚀 . It is up to UX-Logs team to decide if you want to keep
parallel run (new tests can be added either to parallel or sequential
run)
<img width="1578" alt="Screenshot 2025-02-11 at 12 14 30"
src="https://github.com/user-attachments/assets/e94113f2-d7f1-470e-a6d5-cb5154d99c41"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR migrates the last routes with `access:<privilege>` tags used in
route definitions to new security configuration.
Please refer to the documentation for more information: [Authorization
API](https://docs.elastic.dev/kibana-dev-docs/key-concepts/security-api-authorization)
### **Before Migration:**
Access control tags were defined in the `options` object of the route:
```ts
router.get({
path: '/api/path',
options: {
tags: ['access:<privilege_1>', 'access:<privilege_2>'],
},
...
}, handler);
```
### **After Migration:**
Tags have been replaced with the more robust
`security.authz.requiredPrivileges` field under `security`:
```ts
router.get({
path: '/api/path',
security: {
authz: {
requiredPrivileges: ['<privilege_1>', '<privilege_2>'],
},
},
...
}, handler);
```
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Part of https://github.com/elastic/kibana/issues/194171
PR removes all async imports run during uptime plugin setup and start
methods. This causes page load bundle size to increase and accurately
reflect its true size.
### Test instructions
* Start kibana locally
* Open network tab in browser
* Open home page. Verify `uptime.chunk` are not loaded. The screen shots
show the behavior in main where `uptime.chunk` are loaded on home page
<img width="600" alt="Screenshot 2025-02-05 at 9 06 56 AM"
src="https://github.com/user-attachments/assets/14218b85-3814-4e3c-9c04-bd73cf6c4dbd"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR removes the `isDraggable` prop throughout Security Solution.
Unless I'm mistaken, this property isn't necessary anymore, as we do not
use those draggable elements anymore. From what I could see, we had its
value set to `false` everywhere.
This lead to a lot of files impacted, but most of them have only a
couple of lines changed. In some files though, removing the
`isDraggable` prop allowed to remove more code than became obsolete.
**No UI changes should have been introduced in this PR!**
### What this PR does
- removes `isDraggable` everywhere
- performs the extra small cleanup when obvious
- updates all corresponding unit e2e and tests
### What this PR does
- rename files or component names to limit the already extensive impact
of the code change
Resolves#209261
## Summary
Removes the code used to render Logs Explorer. This does not result in
any functional changes.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
When we run Scout tests in parallel, we call SAML authentication in
parallel too and since by default `.security-profile-8` index does not
exist, we periodically getting 503 response:
```
proc [kibana] [2025-01-29T11:13:10.420+01:00][ERROR][plugins.security.user-profile]
Failed to activate user profile: {"error":{"root_cause":[{"type":"unavailable_shards_exception","reason":
"at least one search shard for the index [.security-profile-8] is unavailable"}],
"type":"unavailable_shards_exception","reason":"at least one search shard
for the index [.security-profile-8] is unavailable"},"status":503}. {"service":{"node":
{"roles":["background_tasks","ui"]}}}
```
The solution is to retry the SAML callback assuming that index will be
created and the issue will be solved.
We agreed with Kibana-Security to retry only **5xx** errors, because for
**4xx** we most likely have to start the authentication from the start.
For reviews: it is not 100% reproducible, so I added unit tests to
verify the retry logic is working only for 5xx requests. Please let me
know if I miss something
Retry was verified locally, you might be seeing this logs output:
```
proc [kibana] [2025-01-30T18:40:41.348+01:00][ERROR][plugins.security.user-profile] Failed to activate user profile:
{"error":{"root_cause":[{"type":"unavailable_shards_exception","reason":"at least one search shard for the index
[.security-profile-8] is unavailable"}],"type":"unavailable_shards_exception","reason":"at least one search shard
for the index [.security-profile-8] is unavailable"},"status":503}. {"service":{"node":{"roles":["background_tasks","ui"]}}}
proc [kibana] [2025-01-30T18:40:41.349+01:00][ERROR][plugins.security.authentication] Login attempt with "saml"
provider failed due to unexpected error: {"error":{"root_cause":[{"type":"unavailable_shards_exception","reason":
"at least one search shard for the index [.security-profile-8] is unavailable"}],"type":"unavailable_shards_exception",
"reason":"at least one search shard for the index [.security-profile-8] is unavailable"},"status":503}
{"service":{"node":{"roles":["background_tasks","ui"]}}}
proc [kibana] [2025-01-30T18:40:41.349+01:00][ERROR][http] 500 Server Error {"http":{"response":{"status_code":500},"request":{"method":"post","path":"/api/security/saml/callback"}},"error":
{"message":"unavailable_shards_exception\n\tRoot causes:\n\t\tunavailable_shards_exception: at least one
search shard for the index [.security-profile-8] is
ERROR [scout] SAML callback failed: expected 302, got 500
Waiting 939 ms before the next attempt
proc [playwright]
info [o.e.c.r.a.AllocationService] [scout] current.health="GREEN" message="Cluster health status changed
from [YELLOW] to [GREEN] (reason: [shards started [[.security-profile-8][0]]])."
previous.health="YELLOW" reason="shards started [[.security-profile-8][0]]"
```
To reproduce:
```
node scripts/scout.js run-tests --stateful --config x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
* Categorise and move `@kbn/timelines-plugin` as _platform/shared_,
target location: `x-pack/platform/plugins/shared/timelines`.
This helps reduce the scope of the illegal dependencies from `osquery`
plugin towards _security/private_ code.
cc @tomsonpl
* Simplify path and rename `@kbn/observability-alerting-rule-utils`
(platform/shared):
```
# Before
@kbn/observability-alerting-rule-utils
x-pack/platform/packages/shared/observability/alerting_rule_utils/
# After
@kbn/alerting-rule-utils
x-pack/platform/packages/shared/alerting_rule_utils/
```
* Simplify path and rename `@kbn/observability-logs-overview`
(platform/shared):
```
# Before
@kbn/observability-logs-overview
x-pack/platform/packages/shared/observability/logs_overview/
# After
@kbn/logs-overview
x-pack/platform/packages/shared/logs_overview/
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sets the traceparent for Playwright, so the trace from the test runner
includes the trace events from the browser and Kibana server.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
This PR makes few changes to scout reporter:
Recently I found out that Playwright load configuration file _multiple
times_:
- first time after you run `npx playwright test -c ...`
- on every worker start
log before:
```
[main][~/github/kibana]$ npx playwright test --config x-pack/platform/plugins/shared/maps/ui_tests/playwright.config.ts
[createPlaywrightConfig] called with runId: 18f35f735a10155c
Running 1 test using 1 worker
[createPlaywrightConfig] called with runId: 2633b4e4c20afa15
[chromium] › full_screen_mode.spec.ts:28:9 › Maps › Full screen mode @svlSecurity @svlOblt @svlSearch @ess
```
With our current logic unique `runId` will be generated on each
configuration load, meaning for parallel run we will report failures in
different directories instead of the same one.
Playwright doesn't expose any unique identifier for the run, so we have
do something similar described in
https://github.com/microsoft/playwright/issues/28941#issuecomment-1888008102
log after fix:
```
[main][~/github/kibana]$ npx playwright test --config x-pack/platform/plugins/shared/maps/ui_tests/playwright.config.ts
[createPlaywrightConfig] called with runId: 310a576f32d3b8a5
Running 1 test using 1 worker
[createPlaywrightConfig] called with runId: 310a576f32d3b8a5
[chromium] › full_screen_mode.spec.ts:28:9 › Maps › Full screen mode @svlSecurity @svlOblt @svlSearch @ess
```
We also had a chat with @dolaru and agreed that Scout reporters to be
disabled for local test run. Few reasons:
- Scout custom reporting targets CI execution: events-based reporter was
already disabled
- Failed test reporter purpose is to provide html boilerplate to be
annotated in pipeline build
- When you run tests with IDE playwright plugin it provides its own
reporter / history, should be enough.
### Background
In some scenarios we need to perform some setup at bootstrap time, this
setup could affect following scenarios.
Take for example
[failed_logs](https://github.com/elastic/kibana/blob/main/packages/kbn-apm-synthtrace/src/scenarios/failed_logs.ts)
scenario where we create a pipeline that will do some checks in
`log.level` property, if we try to run an scenario after that one we
will enter into some issues.
### Changes
This PR aims to introduce a `teardown` setup for scenarios where we
could undo the changes done at `bootstrap` time.
## Summary
This PR is a followup to #205990, which removed references to all of the
deprecated/renamed EUI vars in preparation for 9.0. Here, we address
some of the non-critical tech debt related to the EUI refresh, namely
the [removal of static EUI
tokens](https://github.com/elastic/kibana/issues/199715#json-tokens)
from our codebase.
I made every attempt not to change any styles in this PR, except to
simplify CSS to produce an equivalent design. A common example of this
was removing a static `margin` or `padding` declaration referencing
`euiThemeVars.size*`, and swapping it with an equivalent `gutterSize`
prop on the `EuiFlexGroup` container, or with an `align-self` or other
equivalent flexbox directive.
## Screenshots of Areas Affected
The majority of changes here involved the Exception List/Item pages.
I've attached screenshots of their current layout for comparison/review:
<details>
<summary>
<h3>Rule Exceptions Tab</h3>
</summary>
<kbd>
<h3>Before</h3>
<img width="1395" alt="Rule exceptions tab - before"
src="https://github.com/user-attachments/assets/db7a5487-7df3-4a5f-b88e-90ab34784970"
/>
</kbd>
<kbd>
<h3>After</h3>
<img width="1421" alt="Rule exceptions tab - after"
src="https://github.com/user-attachments/assets/77cbdefc-cbec-4b9e-8436-197f2f2f6677"
/>
</kbd>
</details>
<details>
<summary>
<h3>Shared Exception Lists</h3>
</summary>
<kbd>
<img width="1517" alt="Shared Exception Lists"
src="https://github.com/user-attachments/assets/5448dd65-60f7-470c-bf7a-7af75bb914fa"
/>
</kbd>
</details>
<details>
<summary>
<h3>Shared Exception List Details</h3>
</summary>
<kbd>
<img width="1517" alt="Shared Exception List Details"
src="https://github.com/user-attachments/assets/6e03fef6-af58-40bb-98c5-188651a584cc"
/>
</kbd>
</details>
<details>
<summary>
<h3>Threshold Input</h3>
</summary>
<kbd>
<img width="1046" alt="Threshold Input"
src="https://github.com/user-attachments/assets/28738857-6bdf-404f-a790-a9f4e66ff27a"
/>
</kbd>
</details>
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
While working on replacing all `styled-components`with `@emotion/react`
I realized that there is some dead code in the Timelines plugin. Both
the `Overflow` and `ColumnToggle` components used in the `HoverActions`
code exported to other plugins are not used anywhere.
Moreover that `HoverConfig` logic has been deprecated for years now...
I thought about removing all the other hover action components
(`AddToTimeline`, `Copy`, `FilterIn` and `FilterOut`) but these are
currently in used in other plugins (`threat_intelligence` and
`kubernetes_security`) so the effort to replace these are a bit more
involved and will be done in a separate PR
This PR also removes an unused LoadingPanel component.
**No changes to the application are introduced in this PR!**
## Summary
Resolves#180590
Since SLO instanceId is provided in all APIs, it will be made required
for sloWithDataResponseSchema and the SLOWithSummaryResponse type.
Checks for the existence of instanceId have either been removed or
changed to check for ALL_VALUE (`*`)
---------
Co-authored-by: Kevin Delemme <kevin.delemme@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
While working on 9.x Upgrade testing, i found that when running
synthtrace scenarios pointing to a 7.x cluster, it tries to fetch the
latest version of the APM package. In 7.x, the `latestVersion` item was
present under a different node in the API response. Hence adding this
fix.
## Summary
As part of our effort to harden API action definitions and enforce
standards this PR adds an utility `ApiPrivileges` class.
It is supposed to be used for both feature registration and API route
definition to construct the privilege name.
```ts
plugins.features.registerKibanaFeature({
privileges: {
all: {
app: [...],
catalogue: [...],
api: [ApiPrivileges.manage('subject_name')],
...
},
read: {
...
api: [ApiPrivileges.read('subject_name')],
...
},
},
})
....
// route definition
router.get(
{
path: 'api_path',
security: {
authz: {
requiredPrivileges: [ApiPrivileges.manage('subject_name')],
},
},
},
async (ctx, req, res) => {}
);
```
`require_kibana_feature_privileges_naming` eslint rule has been added to
show warning if the API privilege name doesn't satisfy the naming
convention.
### Naming convention
- API privilege should start with valid `ApiOperation`: `manage`,
`read`, `update`, `delete`, `create`
- API privilege should use `_` as separator
❌ `read-entity-a`
❌ `delete_entity-a`
❌ `entity_manage`
✅ `read_entity_a`
✅ `delete_entity_a`
✅ `manage_entity`
> [!IMPORTANT]
> Serverless ZDT update scenario:
>
> - version N has an endpoint protected with the `old_privilege_read`.
> - version N+1 has the same endpoint protected with a new
`read_privilege`.
>
> There might be a short period between the time the UI pod N+1 passes
SO migrations and updates privileges and the time it's marked as
ready-to-handle-requests by k8s, and when UI pod N is terminated.
>
> After discussion with @legrego and @azasypkin we decided to ignore it
due to the perceived risk-to-cost ratio:
> 1. The time window users might be affected is very narrow because we
register privileges late in the Kibana startup flow (e.g., after SO
migrations).
> 2. The transient 403 errors users might get won't result in session
termination and shouldn't lead to data loss.
> 3. The roll-out will be performed in batches over the course of
multiple weeks and implemented by different teams. This means the impact
per release shouldn't be significant.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
__Relates: https://github.com/elastic/kibana/issues/198716__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
We anticipate generating a single document per API version for the
foreseeable future. This PR updates our OAS document merging logic to
avoid adding the `Elastic-Api-Version` header to our spec.
This will also remove this header from current documentation.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Add MapsPage to the scout core to be re-used by others.
Add rudimentary docs to show how to run these tests.
Added a `waitForRender` method.
Add test id.
---------
Co-authored-by: Robert Oskamp <traeluki@gmail.com>
Co-authored-by: Nick Partridge <nicholas.partridge@elastic.co>
Co-authored-by: Nick Partridge <nick.ryan.partridge@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
## Summary
Adding script to discover Scout playwright tests in Kibana repo, will be
used to build CI pipeline step (running tests per plugin in a separate
worker for the start). We can also consider using it to decide if code
change should trigger only specific plugin tests to run.
Usage:
```
node scripts/scout.js discover-playwright-configs --searchPaths x-pack/platform/plugins/private/discover_enhanced
```
Output:
```
info Searching for playwright config files in the following paths:
info - x-pack/platform/plugins/private/discover_enhanced/**/ui_tests/{playwright.config.ts,parallel.playwright.config.ts}
info
info Discovered playwright config files in '1' plugins
info [discover_enhanced] plugin:
info - x-pack/platform/plugins/private/discover_enhanced/ui_tests/parallel.playwright.config.ts
info - x-pack/platform/plugins/private/discover_enhanced/ui_tests/playwright.config.ts
```
More usage examples:
```
node scripts/scout.js discover-playwright-configs // by default will search in ['src/platform/plugins', 'x-pack/**/plugins'] and return all existing ones
node scripts/scout.js discover-playwright-configs --searchPaths x-pack/platform // platform ones under x-pack
node scripts/scout.js discover-playwright-configs --searchPaths x-pack/** // all under x-pack
```
## Summary
This PR introduces support for configuring the `timestamp` field for
entity store enablement.
By default, the `timestamp` field is set to `@timestamp`, but users can
opt to use `event.ingested` or another preferred value based on their
requirements.
### Entity Store API changes
#### Entity Store enable
```
POST kbn:/api/entity_store/enable
{
"timestampField": "event.ingested"
}
```
#### Result
```
{
"engines": [
{
"status": "started",
"type": "user",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
},
{
"status": "started",
"type": "host",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
},
{
"status": "started",
"type": "universal",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
}
],
"status": "running"
}
```
```
POST kbn:/api/entity_store/enable
{
}
```
#### Result
```
{
"engines": [
{
"status": "started",
"type": "host",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
},
{
"status": "started",
"type": "user",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
},
{
"status": "started",
"type": "universal",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
}
],
"status": "running"
}
```
Different entity types can have distinct `timestampField` values, as
described below:
For instance, the `host` entity uses `event.ingested` as its
`timestampField`, while other entities default to the `@timestamp`
field.
```
POST kbn:/api/entity_store/engines/host/init
{
"timestampField": "event.ingested"
}
POST kbn:/api/entity_store/engines/user/init
{
}
```
#### Result
```
{
"engines": [
{
"status": "started",
"type": "universal",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
},
{
"status": "started",
"type": "host",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
},
{
"status": "started",
"type": "user",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
}
],
"status": "running"
}
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Testing steps
- Check out this PR branch.
- Start Kibana.
- Avoid enabling the Entity Store from the UI initially. Instead, enable
it via the API by following the steps outlined below.
```
DELETE kbn:/api/entity_store/engines/user
DELETE kbn:/api/entity_store/engines/host
GET kbn:/api/entity_store/status
All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
"timestampField": "event.ingested"
}
# All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
}
# Different entity with different timestamp Field
POST kbn:/api/entity_store/engines/host/init
{
"timestampField": "event.ingested"
}
POST kbn:/api/entity_store/engines/user/init
{
}
```
4. Afterward, you can test the functionality through the UI. First,
clear the entity data from the UI, then start the Entity Store. Note
that the UI currently does not offer a way to configure this parameter.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Mark Hopkin <mark.hopkin@elastic.co>
