## Summary
* Categorise and move `@kbn/timelines-plugin` as _platform/shared_,
target location: `x-pack/platform/plugins/shared/timelines`.
This helps reduce the scope of the illegal dependencies from `osquery`
plugin towards _security/private_ code.
cc @tomsonpl
* Simplify path and rename `@kbn/observability-alerting-rule-utils`
(platform/shared):
```
# Before
@kbn/observability-alerting-rule-utils
x-pack/platform/packages/shared/observability/alerting_rule_utils/
# After
@kbn/alerting-rule-utils
x-pack/platform/packages/shared/alerting_rule_utils/
```
* Simplify path and rename `@kbn/observability-logs-overview`
(platform/shared):
```
# Before
@kbn/observability-logs-overview
x-pack/platform/packages/shared/observability/logs_overview/
# After
@kbn/logs-overview
x-pack/platform/packages/shared/logs_overview/
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sets the traceparent for Playwright, so the trace from the test runner
includes the trace events from the browser and Kibana server.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/209566
### Problem
Any input change causes Canvas embeddable's to get re-created. This
means that setting a filter control or clicking the refresh button
causes embeddables to get re-created.
In the old embeddable system, the Canvas would only call
`embeddable.updateInput` and `embeddable.reload` on [input
changes](https://github.com/elastic/kibana/blob/8.13/x-pack/plugins/canvas/canvas_plugin_src/renderers/embeddable/embeddable.tsx#L163).
### Solution
PR updates embeddable renderer to store embeddable API. Then, on input
changes, Canvas calls `embeddable.setFilters`.
There is no `embeddable.updateInput` equivalent in the new embeddable
system. Instead, each state key needs to be updated by a setter. The
[Canvas
documentation](https://www.elastic.co/guide/en/kibana/current/canvas-function-reference.html#embeddable_fn)
states that the embeddable function only accepts `filters`. Therefore,
the only key that is expected to change from the input is `filters`.
Please correct me if this is an incorrect assumption.
### Test instructions
1) install sample web logs
2) install canvas saved object and reload kibana (otherwise canvas is
not available in the nav menu)
3) open new canvas
4) add map embeddable
5) add filter control. set source to sample web logs and field to
`geo.dest`.
<img width="200" alt="Screenshot 2025-02-04 at 2 58 01 PM"
src="https://github.com/user-attachments/assets/6862f0bc-4f61-4f16-aa7c-ea8008cfdbf9"
/>
6) prefix map element expression with `kibana | selectFilter` so it
looks like `kibana | selectFilter | embeddable config=...`
7) change filter. Verify map updates but map embeddable is not
re-created.
8) click refresh button, Verify map updates but is not re-created.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Based on the discussion regarding the availability of the Inference
Connector in 8.18, the decision was next:
- enable .inference connector type by default for 8.18
https://github.com/elastic/kibana/pull/209197
- do not show inference connector for Kibana GenAI if Inference Endpoint
does not exist. This is related to the preconfigured connector for
Elastic Rainbow Sprinkles availability and UX.
- preconfigured connector for Elastic Rainbow Sprinkles will always
exists in the Stack Management Connectors list, but on run will show the
banner message in case if that Inference Endpoint does not exist
<img width="2265" alt="Screenshot 2025-02-03 at 11 38 41 AM"
src="https://github.com/user-attachments/assets/029a814a-cc66-4a31-9e92-3512587e377f"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves#209172
### Old
<img width="1728" alt="Screenshot 2025-02-05 at 11 48 59 AM"
src="https://github.com/user-attachments/assets/c08e9544-5a84-462b-80fe-cf5e982bfccf"
/>
### New
<img width="1728" alt="Screenshot 2025-02-05 at 11 47 57 AM"
src="https://github.com/user-attachments/assets/d4f26f21-70a5-4c5c-9f43-05a9da6734c5"
/>
## Testing
Create an APM availability SLO with any service. Use default settings
(`*`) for transaction type and service name. Once SLI drops below
objective, you should see an alert. View alert details. On main branch,
several alerts appear indicating that Kibana is unable to load APM
visualizations, even if for just a second. Now, the errored out charts
(alerts) should not appear.
## Release Notes
Fixes an issue where APM charts were rendered without required
transaction type or service name, causing excessive alerts to appear
Related to https://github.com/elastic/kibana-team/issues/1236
Adds a couple missing explicity authorization opt-outs (since we rely on
Elasticsearch everywhere). For some endpoints in the dashboards we
didn't check Elasticsearch first, I added those checks.
### [Security Solution] [Attack discovery] Display additional Attack Chain tactics
This PR updates the Attack discovery _Attack Chain_ allow list to visualize additional tactics (e.g. `Defense Evasion`), as illustarted by the screenshot below:
_Above: The Attack Chain includes `Defense Evasion`, a newly allow listed tactic_
#### Details
This PR updates the Attack Chain allow list to include the following additional tactics:
- `Resource Development`
- `Defense Evasion`
- `Credential Access`
- `Collection`
- `Impact`
#### Desk testing
1. Navigate to Security > Attack discovery
2. Click `Generate` to generate Attack discoveries
**Expected result**
- The Attack Chain visualization includes the additional allow listed tactics noted in the details of this PR
3. Locate an Attack discovery where one of the newly allow listed tactics is red, (which indicates the tactic was part of the attack), and click the `View in AI Assistant` button
**Expected result**
- A `>` accordion button in the assistant appears next to the selected Attack discovery, indicating it will be included as context
4. Click the `>` button to expand the discovery in the assistant
**Expected result**
- The expanded `Attack Chain` markdown includes an entry for the newly allow listed tactic, and all other tactics that were colored red in the discovery
## 📓 Summary
Closes https://github.com/elastic/streams-program/issues/130
This work decouples the `SchemaEditor` component from the business logic
used for the stream management schema detail to make this part re-usable
with a consistent UX on the enrichment processing part.
The core changes of this work are:
- Move the new `SchemaEditor` component into its own folder and provide
it to the existing stream details section.
- Expose event handlers and custom hooks to facilitate interacting with
a definition streams.
- Refactor internal state to push down those states the consumer doesn't
need to know about (editing form, loadings)
It is now responsibility of a consumer to adapt into the supported
properties, which can of course be extended for upcoming changes.
```tsx
<SchemaEditor
fields={fields}
isLoading={isLoadingDefinition || isLoadingUnmappedFields}
stream={definition.stream}
onFieldUnmap={unmapField}
onFieldUpdate={updateField}
onRefreshData={refreshFields}
withControls
withFieldSimulation
withTableActions={!isRootStreamDefinition(definition.stream)}
/>
```
## Summary
This PR makes few changes to scout reporter:
Recently I found out that Playwright load configuration file _multiple
times_:
- first time after you run `npx playwright test -c ...`
- on every worker start
log before:
```
[main][~/github/kibana]$ npx playwright test --config x-pack/platform/plugins/shared/maps/ui_tests/playwright.config.ts
[createPlaywrightConfig] called with runId: 18f35f735a10155c
Running 1 test using 1 worker
[createPlaywrightConfig] called with runId: 2633b4e4c20afa15
[chromium] › full_screen_mode.spec.ts:28:9 › Maps › Full screen mode @svlSecurity @svlOblt @svlSearch @ess
```
With our current logic unique `runId` will be generated on each
configuration load, meaning for parallel run we will report failures in
different directories instead of the same one.
Playwright doesn't expose any unique identifier for the run, so we have
do something similar described in
https://github.com/microsoft/playwright/issues/28941#issuecomment-1888008102
log after fix:
```
[main][~/github/kibana]$ npx playwright test --config x-pack/platform/plugins/shared/maps/ui_tests/playwright.config.ts
[createPlaywrightConfig] called with runId: 310a576f32d3b8a5
Running 1 test using 1 worker
[createPlaywrightConfig] called with runId: 310a576f32d3b8a5
[chromium] › full_screen_mode.spec.ts:28:9 › Maps › Full screen mode @svlSecurity @svlOblt @svlSearch @ess
```
We also had a chat with @dolaru and agreed that Scout reporters to be
disabled for local test run. Few reasons:
- Scout custom reporting targets CI execution: events-based reporter was
already disabled
- Failed test reporter purpose is to provide html boilerplate to be
annotated in pipeline build
- When you run tests with IDE playwright plugin it provides its own
reporter / history, should be enough.
### Background
In some scenarios we need to perform some setup at bootstrap time, this
setup could affect following scenarios.
Take for example
[failed_logs](https://github.com/elastic/kibana/blob/main/packages/kbn-apm-synthtrace/src/scenarios/failed_logs.ts)
scenario where we create a pipeline that will do some checks in
`log.level` property, if we try to run an scenario after that one we
will enter into some issues.
### Changes
This PR aims to introduce a `teardown` setup for scenarios where we
could undo the changes done at `bootstrap` time.
## Summary
If you enable streams (which creates `logs`) and then try to create
`logs.child.grandchild` but `logs.child` already exists as either an
index or an unwired (Classic) stream, then we end up in a weird state
where `logs.child.grandchild` gets created as a wired child but then the
request fails as it tries to turn the unwired stream into a wired
stream.
This PR adds a step that asserts that there are no such conflicts in the
hierarchy before proceeding.
It also adds a check to ensure Streams are enabled before allowing the
creation of any streams, as well as blocking the creation of a root
stream that isn't `logs`.
Finally, there is some minor improvements to error handling for when a
data stream isn't found and error messages.
fix https://github.com/elastic/kibana/issues/167676 for 9.0 promotion
steps taken:
1. Start Node01:
`yarn es snapshot --version=8.19.0 --data-archive
src/core/server/integration_tests/saved_objects/migrations/archives/7.13.0_5k_so_node_01.zip
--base-path .es/node01`
2. Start Node02:
`yarn es snapshot --version=8.19.0 --data-archive
src/core/server/integration_tests/saved_objects/migrations/archives/7.13.0_5k_so_node_02.zip
--base-path .es/node02`
3. Wait for both nodes to start and be ready
4. Stop both ES nodes
5. change into the directory for node01:
`cd .es/node01/8.19.0`
6. Remove (or move) the outdated archive:
`rm
../../../src/core/server/integration_tests/saved_objects/migrations/archives/7.13.0_5k_so_node_01.zip`
7. zip the updated data and store as the original test filename in the
test directory:
`zip -r
../../../src/core/server/integration_tests/saved_objects/migrations/archives/7.13.0_5k_so_node_01.zip
data -x "*/\.*"`
Repeat these steps for node02:
8. `cd ../../../`
9. `cd .es/node02/8.19.0`
10. `rm
../../../src/core/server/integration_tests/saved_objects/migrations/archives/7.13.0_5k_so_node_02.zip`
11. `zip -r
../../../src/core/server/integration_tests/saved_objects/migrations/archives/7.13.0_5k_so_node_02.zip
data -x "*/\.*"`
12. `cd ../../../`
13. `git commit -m "Updated archives" && git push`
### Identify risks
Data archives have to be updated for every major.last before a new major
is released.
Should updating the archives not resolve test failures, there may be a
bug elsewhere in the stack. If so, carefully inspect the error logs to
determine the cause.
## Summary
Adds audit logging for successful calls to these API routes:
- Start/Stop migration
- Create migration
- Update Rule
- Install Rule
- Retrieve migration results.
- Uploaded Macro/Lookup
- Retrieved Macro/Lookup
Tested it manually by going through the workflow with audit logging
enabled:
Enable:
`xpack.security.audit.enabled: true`
Results:
```
{"event":{"action":"siem_migration_created","category":["database"],"type":["creation"],"outcome":"success"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default"},"trace":{"id":"00885dd4-7fd9-45fe-9a0b-2173adcac4ad"},"client":{"ip":"127.0.0.1"},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2025-01-22T15:05:12.875+01:00","message":"User created a new SIEM migration with [id=cd9552ce-05c8-4893-b659-b5a5ed9325d9","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":91324,"uptime":540.063456227},"transaction":{"id":"ee4606116856393c"}}
{"event":{"action":"siem_migration_started","category":["database"],"type":["start"],"outcome":"success"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default"},"trace":{"id":"e852b328-9e53-4c4d-b8ca-b8fa2b76383d"},"client":{"ip":"127.0.0.1"},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2025-01-22T15:11:36.569+01:00","message":"User started an existing SIEM migration with [id=3805f79e-123c-4962-b22b-8ddf365cdd89]","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":5438,"uptime":62.828177986},"transaction":{"id":"0ac652c8f722f1c4"}}
{"event":{"action":"siem_migration_stopped","category":["database"],"type":["end"],"outcome":"success"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default"},"trace":{"id":"136f3a38-d47d-455d-bca2-aaf38559b20a"},"client":{"ip":"127.0.0.1"},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2025-01-22T15:11:05.871+01:00","message":"User stopped an existing SIEM migration with [id=3805f79e-123c-4962-b22b-8ddf365cdd89]","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":5438,"uptime":32.12840553},"transaction":{"id":"be379686654f4bdf"}}
{"event":{"action":"siem_migration_updated_rule","category":["database"],"type":["change"],"outcome":"success"},"user":{"id":"u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0","name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default","session_id":"IPgQ3+R8DW9uxx4RQqUx9eZj+D5Es7SGQdcDoM/02l4="},"trace":{"id":"368b31e7-812d-464b-83d8-0e635c7fe5ed"},"client":{"ip":"127.0.0.1"},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2025-01-22T15:13:15.827+01:00","message":"User updated a translated detection rule with [id=29pWjpQB_LGnD_bEV66u]","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":5438,"uptime":162.09338085},"transaction":{"id":"404b3cb31be3c94f"}}
{"event":{"action":"siem_migration_installed_rule","category":["database"],"type":["creation"],"outcome":"success"},"user":{"id":"u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0","name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default","session_id":"IPgQ3+R8DW9uxx4RQqUx9eZj+D5Es7SGQdcDoM/02l4="},"trace":{"id":"d90396dc-a0d3-4308-b07e-54761b562803"},"client":{"ip":"127.0.0.1"},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2025-01-22T15:13:46.709+01:00","message":"User installed a new detection rule through SIEM migration with [id=3805f79e-123c-4962-b22b-8ddf365cdd89]","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":5438,"uptime":192.977728866},"transaction":{"id":"4b17bafb9fbf48a4"}}
{"event":{"action":"siem_migration_uploaded_macro","category":["database"],"type":["creation"],"outcome":"success"},"user":{"name":"elastic","roles":["superuser"]},"kibana":{"space_id":"default"},"trace":{"id":"e5fcb9cc-9d27-41cf-a171-13b9faf6078e"},"client":{"ip":"127.0.0.1"},"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2025-01-23T13:45:18.639+01:00","message":"User uploaded a new macro through SIEM migration with [id=2fc199ef-5bc7-4d87-a349-baeaea662273]","log":{"level":"INFO","logger":"plugins.security.audit.ecs"},"process":{"pid":89625,"uptime":106.911700803},"transaction":{"id":"fc95c131e86b0284"}}
```
## Release Note
Fix Structured log template to use single quotes
### Summary
Currently with a single backslash as escape character the template is
setting up a double quote surrounding it and it causes YAML Exception
```
1 | - kv:
2 | field: message
3 | field_split: "\|"
--------------------------^
YAMLException: unknown escape sequence (3:22)
```
This PR fixes to use single quotes to not break YAML parsing.
## Summary
Fixes#206555
This PR is an attempt to address the `null` bucket issue with `count` in
Lens formula via the `emptyAsNull` flag.
### Checklist
* [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### Risks
This PR introduces potentially some breaking changes, as count `null`
values, in particular coming from shifted computations, as now converted
to `0` if the flag has been enabled.
This change is not news in the code base as other aggs like
`distinct_count` or `value_count` already implements it, but not
`count`.
Apparently no test failed with this change, I've also added new unit
ones to freeze the current behaviour and detect future changes.
---------
Co-authored-by: Marco Vettorello <vettorello.marco@gmail.com>
Co-authored-by: Peter Pisljar <peter.pisljar@gmail.com>
Closes#207005
## Summary
This PR skips flaky tests - 2 of the flaky tests were already skipped so
I skipped the test on the top level (as all the tests will be skipped
anyway it makes more sense to have the whole test skipped)
I tried to fix it and managed to reproduce the flaky behavior only once
out of many runs (the input was not filled but the next check failed -
which was not an expected behavior as it should fail on the previous
step or retry it) - It's super hard to reproduce it. I followed the
steps locally and it worked as expected so it's not an actual issue:
https://github.com/user-attachments/assets/d0d33622-c186-4b31-bcf7-b2c27df330ac
As we plan to refactor the test anyway we should not spend more time on
it so I skipped it for now.
Intermediate releases to change rule or connector schemas in serverless
for #185032
## Summary
An Observability AI Assistant connector is available to be set as action
for Observability rules. When an alert is triggered, a conversation with
the AI assistant will be created sending the initial prompt set by the
user in the rule action. The conversation is then stored and can be
retrieved from the AI Assistant interface. the action is triggered on
any status change of the alert (active, recovered, untracked), creating
a new conversation for each of them using the same initial prompt which
may not be suitable for the 3 cases.
Improvement
The user is able to choose in when the action should be run (active,
recovered, untracked, all),. That would allow the user to specify more
than one AI Assistant action, with a different and more suitable prompt
in each case.
Closes#207734
This PR:
* Changes the behavior of the search form to always include quickstart
flows in search results even when a category is not selected
* Refactors and cleans up the code a bit
https://github.com/user-attachments/assets/e5de7092-2d9f-41be-8d69-25954e5e4bff
## How to test
Make sure that the search works as expected and when clicking on the
cards it leads to the right places.
# Summary
As part of the effort to add missing content for Security APIs, this PR
introduces a few missing request, response, and parameter examples for
Detection Engine Alert and migration APIs.
