## Summary
Enables the SIEM Migrations feature in serverless. The experimental flag
will be cleaned later.
> [!NOTE]
To release this feature as planned (Friday 28th March 2025), this PR
needs to merge between Monday **24** and Thursday **27**
### Screenshot
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Fixes https://github.com/elastic/kibana/issues/213324
For some reason this PR https://github.com/elastic/kibana/pull/213928
didn't catch that the changes make the test fail. This PR fixes it.
After this, we'll be able to backport the original PR.
(The pattern of using `data-attr-field` is already used in Discover so I
think it's a good opportunity to align)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Nick Partridge <nicholas.partridge@elastic.co>
Resolves https://github.com/elastic/kibana/issues/202564
## Summary
The test is testing for the number of alerts generated assuming the AAD
rule runs 3 times (2 alerts per run). It looks like sometimes, the rule
only runs twice before the telemetry is collected so this updates the
test to verify the number of alerts based on the number of times the
rule ran.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
We've done a lot of UA changes against 8.18 and 8.x but we did not port
them to `main`. The reason we did not do the active development against
`main` is because UA is disabled on main and we need those features in
8.last not on `9.0` initially. This port is to keep the codebase
consistent and to be able to use these new UA features in the future
post `9.0`.
Any issues that that are caused from this port we should wrap them in a
config and disabled them on main and enable this new flag on `8.x`.
What is being ported?
- [x] Upgrade assistant plugin
(`x-pack/platform/plugins/private/upgrade_assistant`)
- [x] Integration test changes
(`x-pack/test/upgrade_assistant_integration`)
- [x] new UA doc links
(`src/platform/packages/shared/kbn-doc-links/src/get_doc_links.ts`)
- [x] localization files sync
(`x-pack/platform/plugins/private/translations/translations/*.json`)
- [x] Unfreeze is no longer supported after `8.x`. So removed it from
data streams readonly migration step and throw an error for regular
indices migration.
Closes https://github.com/elastic/kibana/issues/210490
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Skips the basic license cases list view test since it is expecting the
cases list to not to be present for a basic/essentials tier license, but
the FIPS pipeline always runs with a platinum license override.
## Summary
Add a new telemetry task to the security solution plugin to collect
ingest pipeline stats. The new task runs once a day, calls the
`_nodes/stats/ingest` API, and sends an EBT event with the following
information:
```js
export interface NodeIngestPipelinesStats {
name: string;
totals: Totals;
pipelines: Pipeline[];
}
export interface Pipeline {
name: string;
totals: Totals;
processors: Processor[];
}
export interface Processor {
name: string;
totals: Totals;
}
export interface Totals {
count: number;
time_in_millis: number;
current: number;
failed: number;
}
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Ryland Herrick <ryalnd@gmail.com>
## Summary
Provides the Cases service to the detection engine alerts table. The
missing services caused the cases actions to disappear from the bulk
actions menu.
## Verification steps
1. Create Security rules that fire alerts
2. Visit the Security > Alerts page
3. Select one or more alert rows from the table
4. Open the `Selected X alerts` bulk action menu
5. Check that the cases bulk actions are available
## Release Notes
Fixes a regression that caused the cases actions to disappear from the
detections engine alerts table bulk actions menu.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
This PR establishes the foundation for executing API tests in the new
`search_ai_lake` tier, following the existing API integration test
structure and guidelines.
## Adding a New Test
To add a new test, follow these guidelines:
- Inside the `AI4DSOC` folder, create subfolders representing different
AI4DSOC functionalities.
- Each subfolder should be owned by an area team or the developers
actively working on it.
- The functionality folder must include a `search_ai_lake_tier`
subfolder.
- The `search_ai_lake_tier` subfolder should contain a `configs`
directory with a `serverless.config.ts` file that imports
`createTestConfig` from `config.base.ai4dsoc`.
- Add the test inside the `search_ai_lake_tier` subfolder.
- Ensure the test has the `@serverless` label and uses
`supertestWithoutAuth` instead of `supertest`, as `supertest` provides
basic authentication, whereas serverless environments require API key
authentication. See the `dummy_test.ts` for reference.
- The `search_ai_lake_tier` folder should have an `index.ts` file
referencing the tests to be executed, as demonstrated in this PR.
- Update the
`x-pack/test/security_solution_api_integration/package.json` file with
the necessary scripts to enable test execution locally.
- When adding a new `serverless.config.ts` file, ensure it is included
in `.buildkite/ftr_security_serverless_configs.yml`. Otherwise, the new
test(s) will not be executed as part of the PR process.
## Running Tests Locally
Execute the tests using the following Yarn scripts from
`x-pack/test/security_solution_api_integration`:
1. Start the server with the required configuration:
```sh
yarn ai4dsoc_cases:server:serverless
```
2. Run the tests using the started server:
```sh
yarn ai4dsoc_cases🏃serverless
```
## Key Considerations
- `Supertest` should not be used, as it provides basic authentication.
Instead, use supertestWithoutAuth for API key authentication.
- All tests must include the `@serverless` label.
- MKI is not yet supported for test execution.
- Temporary Ownership: The Security Engineering Productivity team will
initially own the AI4DSOC testing folder to ensure proper structure and
best practices. Once teams are familiar with the workflow, this
ownership will be removed.
## Security Engineering Productivity Code Ownership Responsibilities
The Security Engineering Productivity team should ensure:
- All tests are placed inside a functionality-specific subfolder.
- Each functionality subfolder has designated code owners.
- Tests include the `@serverless` label.
- `Supertest` is not used.
- The correct configuration is applied.
- Scripts are added to enable local execution.
- New configurations are added to
`.buildkite/ftr_security_serverless_configs.yml`.
## Follow-Up tasks
- Remove the existing dummy test.
- Integrate tests into the periodic pipeline.
- Add tests to the Kibana QA quality gate.
## Summary
The intent is to have a centralised place to store the list of Kibana
solutions and serverless project types.
To that end, this PR creates a `@kbn/projects-solutions-groups` package.
It also adds the new solution type `'chat'`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The code changes in this PR ensure that the transform ID is limited to
36 characters when creating or updating the transform for risk-score.
This adjustment aligns with ES constraint on transform ID length.
## Test Steps
1. Create a new namespace with a very long name. Ex :
`namespace_that_stretches_farther_than_the_universe_and_beyond_like_buzz`
🚀
2. Enable the Risk Score in the new namespace. It should successfully
get enabled.
3. Check the transform that was created (using dev tools)
```
GET _transform/risk_score_latest_transform_*?filter_path=transforms.id,transforms._meta.space_id
```
Output
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Mark Hopkin <mark.hopkin@elastic.co>
## Summary
Addresses https://github.com/elastic/kibana/issues/209048
1. Started ES with
```
yarn es snapshot --version 8.19.0 --data-archive=../kibana/x-pack/test/upgrade_assistant_integration/fixtures/data_archives/upgrade_assistant.zip
```
2. Then created updated archive
```
zip -r ../../../kibana/x-pack/test/upgrade_assistant_integration/fixtures/data_archives/upgrade_assistant.zip data -x "*/\.*"
```
## Summary
This PR reworks how APM handles getting its sources data, elevating the
necessary code to a private shared plugin so that Discover for Traces
can access the data and handle user provided configuration. It also
removes the need for Discover for Traces to rely on the APM static data
view, so the Trace data source and document profile will work on any
compatible/configured index, even in ESQL mode.
Closes#211414
<img alt="ESQL Discover Traces Screenshot 2025-03-04 173032"
src="https://github.com/user-attachments/assets/f5bbb736-8b8b-45dc-ac23-4bf7083aa47e"
/>
## How to test
Test with olbt-cli instance for now, will post for doing with synthtrace
data. Ensure the following is added to your kibana.dev.yml:
```yaml
discover.experimental.enabledProfiles:
- observability-traces-data-source-profile
```
- Make sure your space has the Observability solution view configured
- Go to Discover page
- Select Data Views mode if required and create a view with a `traces`
specific index. Or use the APM static data view.
- The default columns on the page should show the summary column with
four of the following badges: `service.name`, `event.outcome`,
`transaction.name`, `transaction.duration.us`, `span.name`,
`span.duration.us`
- Go to ESQL mode with the query targetting a `traces` index
- The default columns should show the same as in Data View mode
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Irene Blanco <irene.blanco@elastic.co>
## Summary
This PR establishes the baseline to execute Cypress tests in the new
`search_ai_lake` tier.
## Changes Introduced
- All tests under
`x-pack/test/security_solution_cypress/cypress/e2e/ai4dsoc` will be
executed using the new tier by default.
- These tests will run as part of the PR process within the `Serverless
AI4DSOC - Security Solution Cypress Tests` execution.
## Adding a New Test
To add a new test, follow these guidelines:
- Read the
[README](x-pack/test/security_solution_cypress/cypress/e2e/ai4dsoc/README.md).
- Inside the `AI4DSOC` folder, we should have different subfolders
representing the various AI4DSOC functionalities.
- Each subfolder should have ownership by either an area team or the
developers actively working on it.
- Make sure that any functionality you want to be tested in the new tier
is added inside the `AI4DSOC` folder; otherwise, that functionality will
be tested using the complete tier.
## Running Tests Locally
Run the tests with the following Yarn scripts from
`x-pack/test/security_solution_cypress`:
```sh
yarn cypress:open:ai4dsoc:serverless
```
Opens the Cypress UI with all tests in the `e2e/ai4dsoc` directory. This
also runs a mocked serverless environment using the `ai_soc` product
line and `search_ai_lake` tier by default.
```sh
yarn cypress:run:ai4dsoc:serverless
```
Runs all tests tagged as @serverless in the e2e/ai4dsoc directory in
headless mode using the ai_soc product line and search_ai_lake tier by
default.
## Key Considerations
- All tests must have the `@serverless` tag to be executed as part of
the PR process.
- MKI is not yet supported for test execution.
- The AI4DSOC Cypress tests will be executed each time there is a change
in one of its
[dependencies](https://github.com/elastic/kibana/blob/main/.buildkite/scripts/pipelines/pull_request/pipeline.ts).
- All tests are executed by default using the `platform_engineer` role.
- Temporary Ownership: The Security Engineering Productivity team will
own the entire AI4DSOC testing folder initially to ensure structure and
best practices. Once all teams understand the workflow, this ownership
will be removed.
- Execution Time: If test execution in a PR takes more than 45 minutes,
parallelism should be increased in the new
`.buildkite/pipelines/pull_request/security_solution/ai4dsoc.yml` file.
## Security Engineering Productivity Codeownership Responsibilities
The Security Engineering Productivity team should ensure:
- Best practices are followed.
- All tests are placed inside a functionality subfolder.
- Each functionality subfolder has designated code owners.
- Tests include the `@serverless` label.
- The execution of AI4DSOC tests does not exceed 45 minutes.
## Follow-Up Tasks
- Remove the dummy test (@tomsonpl feel free to delete it when you need
to add new tests to the navigation).
- Integrate tests into the periodic pipeline.
- Add tests to the Kibana QA quality gate.
- Update the README with MKI instructions once tests are added to the
periodic pipeline and Kibana QA quality gate.
- Clarify which roles will be used for the AI4DSOC effort and update the
tests accordingly.
Adding a link to the stream into the overview tab of the discover
document flyout:
<img width="228" alt="Screenshot 2025-03-12 at 08 57 48"
src="https://github.com/user-attachments/assets/dfd396e7-b0dc-4cca-a09c-637357cc88f9"
/>
Some reviewer notes:
* This is using the same strategy as the observability AI assistant via
the discover_shared registry - streams is not an observability-only
plugin, but for now we want to treat it like this. If we move closer to
this becoming a main feature, we can probably have discover depend on
streams directly
* For now, it's only showing the entry in the flyout if streams is
enabled so it's easy to test but doesn't show up accidentally. Before
the initial release, we can change this condition to always show for
observability spaces
* Resolving an index name to a data stream needs an Elasticsearch call
to get the index meta data. I created a new internal route for that. It
means that there is a loading state in theory, but in practice it should
resolve really quickly because it only hits the cluster state, not the
actual data.
* Even if no stream can be resolved it still shows the entry in the
flyout with a `-`. This is because it avoids shifting layout and it
doesn't seem to hurt if it's there.
* As I need to link to streams, I started introducing a locator - I'm
sure it will be needed more soon. I didn't add all the possible routes
yet, we can expand it as needed.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Related to #177932
Inline editing introduced a blurred boundary for the editing experience
in Lens, while the full editor relies on a Redux state manager to
dispatch state changes the inline editor has an hybrid approach.
Specifically the `fixAction` feature in the user messages system needs
access to the redux store which is not available within the embeddable
environment as for now.
So with this PR I've currently limited the scope provided for those
message with a `fixAction` and won't render the button any more.
For instance while investigating #177932 I've noticed that now the
embeddable doesn't crash any more, but rather shows the error message
with the `fixAction` button:
<img width="1496" alt="Screenshot 2025-03-06 at 16 07 35"
src="https://github.com/user-attachments/assets/1b2a5d73-56d9-4010-8a6f-82528efcb2ce"
/>
Note that clicking on the `Use filters` nothing will happen as
d2412a5f98/x-pack/platform/plugins/shared/lens/public/react_embeddable/user_messages/api.ts (L195)
is a mock. A simple `updateAttributes` call here won't suffice as it
requires the logic behind `updateDatasourceState` slice to harmonize the
datasource changes with the visualisation counter part.
With this PR the message will hide the button and render as follow:
<img width="1498" alt="Screenshot 2025-03-06 at 16 18 55"
src="https://github.com/user-attachments/assets/01d55f6c-7563-4e07-a18e-35d1062a8d79"
/>
It is a temporary fix but at least it won't feel broken.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Adds more coverage for FTRs to test Synonyms UI in serverless.
Adds test cases for synonyms set listing, synoyms set detail and adding
deleting rules.
Covers some happy paths.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Resolves#212784
Ensure that when an SLO is created, the id is verified across all
spaces.
## Release Notes
Ensure that when an SLO is created, the id is verified across all
spaces.
## Testing
1. Create an SLO and save the id returned in the response in a space "A"
2. Create a second SLO with the id saved from the first SLO in the
request in a different space "B"
3. User should receive a 409 error from the SLO API.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
**Resolves: #214633**
## Summary
I am fixing a flaky test in the `delete_rules_bulk_legacy.ts` file
introduced in the #213244.
Also, in two other files I am changing using expect from kbn to jest.
Partially resolves https://github.com/elastic/kibana/issues/180709
Adds `context.grouping` action variable in the following rules:
- Custom threshold rule
- APM Latency threshold rule
- APM Failed transaction rate rule
- APM Error count rule
I will open a follow up PR to add `context.grouping` action variable in
the following rules:
- Elasticsearch query rule
- SLO burn rate rule
Excluded from scope:
- Metric threshold rule (already has `context.groupByKeys`)
- Log threshold rule (already has `context.groupByKeys`)
- Inventory threshold rule (already has `context.group` and this rule
doesn't have explicit group by fields)
### Testing
1. Create each rule with group by fields, and with "active" and
"recovered" actions
3. In "active" and "recovered" action message, use `context.grouping`
variable
4. Ensure that both "active" and "recovered" alert notifications contain
correct information
5. Ensure that the action variables UI in rule form shows
`context.grouping` action variable
Example of action message for APM Latency threshold rule with group by
on `transaction.name`:
```
{
"grouping": "{{context.grouping}}",
"service.name": "{{context.grouping.service.name}}",
"service.environment": "{{context.grouping.service.environment}}",
"transaction.type": "{{context.grouping.transaction.type}}",
"transaction.name": "{{context.grouping.transaction.name}}"
}
```
Example of action message for Custom threshold rule with group by on
`host.name` and `container.id`:
```
{
"grouping": "{{context.grouping}}",
"host.name": "{{context.grouping.host.name}}",
"container.id": "{{context.grouping.container.id}}"
}
```
---------
Co-authored-by: Maryam Saeidi <maryam.saeidi@elastic.co>
## Summary
**Requirement:**
In stack and when its search solution space, we need to update search
index details breadcrumbs, when navigated via Content -> Index
Management :
- Index management list page - `Content / Index Management / Indices`
- Index list page -` Content / Index Management / indices /
<index_name>`
- drop `Stack management` from the breadcrumb
In Classic nav, index management index details page breadcrumbs will
have no change in UI. But index management app is rendered from
search_indices plugin
### Solutions
Currently, Index management app is rendered from
[management_app](https://github.com/elastic/kibana/blob/main/src/platform/plugins/shared/management/public/components/management_app/management_app.tsx).
The management app sets breadcrumbs for all the dependant apps. The
easiest way to implement is to set breadcrumbs based on active solution
type - `es` but this would alter breadcrumbs when index management app
is rendered from side nav footer ( management -> index management) and
other related management apps as well.
Other options is to modify setBreadcrumbs in
[ManagementAppMountParams](https://github.com/elastic/kibana/blob/main/src/platform/plugins/shared/management/public/types.ts#L79)
but the setBreadcrumbs is used by multiple other apps.
In this PR, index management app is mounted via search indices plugin.
In this way we can customize breadcrumbs for index management when
rendered from search_indices plugin. When its search solution type,
index management app will work independently from management app.
### Screenshots
#### Search solution Nav - Changed breadcrumb ( dropped stack management
& added index name)
<img width="1727" alt="Screenshot 2025-02-04 at 1 29 08 PM"
src="https://github.com/user-attachments/assets/bc6f733f-62f4-44bc-8373-24d92719f5df"
/>
#### Serverless
**Note:** No change in functionality from this PR. Added for additional
info
index details page breadcrumbs should be `Data/ Index Management /
Indices/<index_name>`
index list page breadcrumbs should be `Data/ Index Management /
Indices/`
**Serverless Details page**
<img width="1727" alt="Screenshot 2025-02-04 at 1 23 14 PM"
src="https://github.com/user-attachments/assets/72bac7a8-d7d1-40fc-9c73-bbd0545dba1f"
/>
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds a basic api for finding suggested dashboards by alert.
This internal API is currently not in use anywhere.
This api finds suggested dashboards in two ways:
1. Finding dashboards with lens visualizations that query against the
same data view
2. Finding dashboards with lens visualizations that utilize fields in
the rule configuration, or alert data.
These are two naive approaches to finding suggested dashboards. These
heuristics will be improved over time and incorporate more sophisticated
approaches that have been explored by numerous engineers across
Observability.
## Testing
A basic api integration test is included that covers matching dashboards
by index and field, across spaces. As we develop this feature further
over time more fine grain tests will be added
### Manual Testing
1. Run
```
node x-pack/scripts/data_forge.js --events-per-cycle 200 --lookback now-1h --ephemeral-project-ids 10 --dataset fake_stack --install-kibana-assets --kibana-url http://localhost:5601 --event-template bad
```
2. Save the file as a .ndjson file and import it via saved objects
import
https://p.elstc.co/paste/3BWKIHLU#f0WfGYx7G9DhWy88yDXhaEXTX16Fn+ovDcomNIx6E3a
3. Navigate to the alert details page, you should start to see a lot of
alerts
4. Click on one, and copy the alert id
5. Navigate to console and paste `GET
kbn:/internal/observability/alerts/suggested_dashboards?alertId=[YOUR_ALERT_ID]`
to see the recommended dashboards
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Nick Peihl <nickpeihl@gmail.com>
Addresses https://github.com/elastic/kibana/issues/182878
Fixes some alert assignment RBAC tests so that they can be unskipped in
MKI. Also adds some manual run tests to MKI that were skipped due to
feature flags not being supported, but feature is no longer behind
feature flag.
