This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[elastic-apm-node](https://redirect.github.com/elastic/apm-agent-nodejs)
| dependencies | patch | [`^4.11.0` ->
`^4.11.1`](https://renovatebot.com/diffs/npm/elastic-apm-node/4.11.0/4.11.1)
|
---
### Release Notes
<details>
<summary>elastic/apm-agent-nodejs (elastic-apm-node)</summary>
###
[`v4.11.1`](https://redirect.github.com/elastic/apm-agent-nodejs/releases/tag/v4.11.1)
[Compare
Source](https://redirect.github.com/elastic/apm-agent-nodejs/compare/v4.11.0...v4.11.1)
For more information, please see the
[changelog](https://www.elastic.co/guide/en/apm/agent/nodejs/current/release-notes-4.x.html#release-notes-4.11.1).
##### Elastic APM Node.js agent layer ARNs
|Region|ARN|
|------|---|
|af-south-1|arn:aws:lambda:af-south-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-east-1|arn:aws:lambda:ap-east-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-northeast-1|arn:aws:lambda:ap-northeast-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-northeast-2|arn:aws:lambda:ap-northeast-2:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-northeast-3|arn:aws:lambda:ap-northeast-3:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-south-1|arn:aws:lambda:ap-south-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-southeast-1|arn:aws:lambda:ap-southeast-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-southeast-2|arn:aws:lambda:ap-southeast-2:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ap-southeast-3|arn:aws:lambda:ap-southeast-3:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|ca-central-1|arn:aws:lambda:ca-central-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|eu-central-1|arn:aws:lambda:eu-central-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|eu-north-1|arn:aws:lambda:eu-north-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|eu-south-1|arn:aws:lambda:eu-south-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|eu-west-1|arn:aws:lambda:eu-west-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|eu-west-2|arn:aws:lambda:eu-west-2:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|eu-west-3|arn:aws:lambda:eu-west-3:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|me-south-1|arn:aws:lambda:me-south-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|sa-east-1|arn:aws:lambda:sa-east-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|us-east-1|arn:aws:lambda:us-east-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|us-east-2|arn:aws:lambda:us-east-2:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|us-west-1|arn:aws:lambda:us-west-1:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
|us-west-2|arn:aws:lambda:us-west-2:267093732750:layer:elastic-apm-node-ver-4-11-1:1|
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDpwcmV2LW1pbm9yIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
## Summary
Depends on #191106Closes#171591
This PR migrates Storybook from `6.x` to `8.x`. Please see the
[migration
guide](https://storybook.js.org/docs/migration-guide/from-older-version)
for an overview of the changes because there are many breaking changes
which effect Kibana. The TODO list below is not inclusive of all the
changes.
## Reviewers
### Each commit contains all files changed for a specific codeowner,
please find your respective commit to make review easier.
A **first step before code review** should be checking the [`Storybooks
Preview`](https://ci-artifacts.kibana.dev/storybooks/pr-195148/index.html)
from CI for any runtime or style issues which were missed. The preview
can be compared to a build from `main`
[here](https://ci-artifacts.kibana.dev/storybooks/pr-212585/index.html).
It is worth noting that some stories have runtime issues which existed
before this migration.
Most stories appear to have been migrated properly, but the Operations
team does not have prior knowledge into every story. Some of the
migration was able to be automated through Storybook provided scripts.
It is possible this wasn't entirely correct due to the structure of some
stories. Additionally, part of this migration is moving Storybook to
Webpack 5 which changed how styles are being loaded.
#### TODO
- [x] Migrate `stories.mdx`
- [x] storyshots
- [x] [Migrate
packages](https://storybook.js.org/docs/migration-guide/from-older-version#package-structure-changes)
which were removed in `8.0`
- [x] `react-doc-gen` resolution
- [x] [Migrate
blocks](https://github.com/storybookjs/storybook/blob/next/MIGRATION.md#addon-docs-removed-deprecated-blocksjs-entry)
- [x] Migrate deprecated `addon-knobs` to
[addon-controls](https://www.npmjs.com/package/@storybook/addon-controls)
- [x] React Fast Refresh support
- [x] Watch flag callback
- [x] `canvas` webpack
- [x] Rerun CSF migrations for new stories
- [x] Handle ESM import for `addon-docs`
- [x] `'@storybook/addon-actions' should be listed in the project's
dependencies. Run 'npm i -S @storybook/addon-actions' to add
iteslint[import/no-extraneous-dependencies](https://github.com/import-js/eslint-plugin-import/blob/v2.28.0/docs/rules/no-extraneous-dependencies.md)`
- [x] `addDecorator` migration
- [x] `addParameter` migration
- [x] static build
- [ ] determine if #176500 is solved or push to followup PR
- This will need to be fixed separately
- [x] revert `.buildkite/pipelines/pull_request/base.yml` &
`.buildkite/scripts/pipelines/pull_request/pipeline.ts` to `main`
---------
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Jacek Kolezynski <jacek.kolezynski@elastic.co>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
## Summary
Resolves https://github.com/elastic/kibana/issues/213958.
Related to https://github.com/elastic/kibana/pull/199567.
This PR makes it so that after adding an agentless integration, the
agentless status flyout is opened automatically on the policies page.
This is done by passing the query parameter
`?openEnrollmentFlyout=<agent policy id>` after successful creation:
<img width="1440" alt="image"
src="https://github.com/user-attachments/assets/720caab3-10dc-4c1e-bb9a-6a65eb5b5a48"
/>
## Testing
1. Add the following to kibana.dev.yml:
```
xpack.cloud.id: 'anything-to-pass-cloud-validation-checks'
xpack.fleet.agentless.enabled: true
xpack.fleet.agentless.api.url: 'https://localhost:8443'
xpack.fleet.agentless.api.tls.certificate: './config/certs/ess-client.crt'
xpack.fleet.agentless.api.tls.key: './config/certs/ess-client.key'
xpack.fleet.agentless.api.tls.ca: './config/certs/ca.crt'
```
2. Apply [this
patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4)
to prevent attempt to create agentless pod
3. Enroll a Fleet Server as usual
4. Go to CSPM integration, select agentless deployment mode
5. After saving, you should be redirected to CSPM integration > Policies
table with the agentless status flyout open for the one that you just
created
Do some regression testing around other redirect scenarios, such as:
* Agent-based CSPM -> manual (not cloudformation) should redirect to
agent policy details page with enrollment flyout open after clicking
"Add agent now"
* Editing various integration policies redirect as expected (from
integration > policies table should redirect back to integration page,
from agent policy details should redirect back to agent policy)
* Creating various integration policies with none/one/multiple agent
policies should redirect as expected (none should redirect to
integrations page, one/multiple should redirect back to first agent
policy)
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `3f6bcf4` ->
`66d427a` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Addresses https://github.com/elastic/kibana/issues/182878
Fixes some alert assignment RBAC tests so that they can be unskipped in
MKI. Also adds some manual run tests to MKI that were skipped due to
feature flags not being supported, but feature is no longer behind
feature flag.
We decided to allow non-additive changes. However, the simulation
endpoint was still counting parsing steps which would make non-additive
changes as errors.
This PR makes sure that additive changes are not counted towards the
error rate, but are still reported to the UI and shown as a warning.
Resolves https://github.com/elastic/kibana/issues/209427
## Summary
This PR fixes a bug when editing an es query rule with size set to 0. I
also refactored the tests to use react testing library.
### Checklist
Check the PR satisfies following conditions.
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### To verify
1. Create an es query rule
2. Set the size to be 0
3. Save your rule
4. Edit your rule and verify that the size is set to 0 when you open the
editor
## [Security Solution] [Attack discovery] Display error messages for invalid Anonymization configurations
This PR detects invalid Anonymization configurations when generating Attack discoveries.
It displays an error message when:
- The Security AI Anonymization settings are configured to not allow _any_ fields, as reported in <https://github.com/elastic/kibana/issues/214310>
- The Security AI Anonymization settings are configured to not allow the `_id` field, as reported by @aarju
### Out of scope: detecting configurations that don't include enough useful fields
The default fields in the Security AI Anonymization settings were chosen because they are most likely to provide relevant context for the AI Assistant and Attack discovery.
However, there isn't a well defined threshold for the minimum set of useful fields.
For example, Attack discovery may still produce useful results (depending on the data), if the `user.name`, `host.name`, and `source.ip` fields are not allowed, but in most cases omitting these important fields will reduce the quality of results.
Another example: A configuration that **only** allows just **two** fields, for example the `_id` field AND `user.name` fields is valid, but NOT _useful_.
- Detecting configurations that don't include enough _useful_ fields is beyond the scope of this PR
- Configurations that **only** allow the `_id` field are _valid_, but not _useful_
#### Desk testing
1. Navigate to Stack Management > AI Assistants > Security
2. Configure the Security AI Anonymization settings to deny all fields.
Note: At the time of this writing, using the bulk actions to update all `102` default fields may result in an `Unable to load page` error that appears below the table when it is saved. Refreshing the page reveals the settings are not updated after clicking save. As a workaround for this separate, unrelated issue, apply bulk actions to only one page at a time, and be sure to refresh the page after saving changes to verify the update(s) before continuing to the next step.
3. Navigate to Security > Attack discovery
4. Click `Generate`
**Expected result**
- The following error message is displayed:
```
Your Security AI Anonymization settings are configured to not allow any fields. Fields must be allowed to generate Attack discoveries.
```
as illustrated by the screenshot below:
5. Once again, navigate to Stack Management > AI Assistants > Security
6. Allow all the (`102`) default fields
7. Once again, navigate to Security > Attack discovery
8. Click `Generate`
**Expected result**
- Attack discoveries are generated
9. Navigate back to Stack Management > AI Assistants > Security
10. Configure the `_id` field, and another (arbitrary) field to NOT be allowed
11. Navigate back to Security > Attack discovery
12. Click `Generate`
**Expected result**
- The following error message is displayed:
```
Your Security AI Anonymization settings are configured to not allow the _id field. The _id field must be allowed to generate Attack discoveries.
```
as illustrated by the screenshot below:
We decided to not enforce additive parsing. This means it's OK for the
generated patterns to override @timestamp or message directly.
This PR removes the special handling.
**Partially addresses:** #211808,
https://github.com/elastic/security-docs/issues/5981 (internal)
**Resolves: #208329**
## Summary
This is the second part of the migration effort, containing changes for:
- BULK CRUD (removing, for v.9.0)
The PR also contains changes for ticket #208329 - as changes for
removing of dead code for handling Bulk CRUD endpoints had to be
combined together with removing the schema files for Bulk CRUD
endpoints.
This PR will be backported only to versions for Kibana v9
# Testing
1. cd x-pack/solutions/security/plugins/security_solution
2. yarn openapi:bundle:detections
3. Take the bundled file
(docs/openapi/ess/security_solution_detections_api_2023_10_31.bundled.schema.yaml)
and load it into bump.sh console to see the changes.
4. Compare the changes with the [Legacy
documentation](https://www.elastic.co/guide/en/security/current/rule-api-overview.html)
You can also use this [link](https://bump.sh/jkelas2/doc/kibana_wip2/)
where I deployed the generated bundled doc.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Follow up to https://github.com/elastic/kibana/pull/212694. This PR
switches more DE functions over to use `sharedParams`, an object that
contains the variety of parameters that we build at the top level of
rule execution in the common security rule wrapper. With `sharedParams`
available throughout rule execution, it's easier to access all the
parameters necessary for `wrapHits` when we call it so I also removed
the "factory" logic and just call `wrapHits` directly instead of passing
the function as a parameter on `sharedParams`.
There should be very few behavior changes in the code as a result of
this PR.
- `kibana.alert.rule.indices` is now populated for ES|QL alerts
- `ignoreFields` and `ignoreFieldsRegexes` are now respected by EQL, new
terms, and all suppressed rule types
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
`apmIndices` has the form:
```
"transaction": "traces-apm*,apm-*,traces-*.otel-*",
"span": "traces-apm*,apm-*,traces-*.otel-*",
"error": "logs-apm*,apm-*,logs-*.otel-*",
"metric": "metrics-apm*,apm-*,metrics-*.otel-*",
"onboarding": "apm-*",
"sourcemap": "apm-*"
```
`mapValues(apmIndices, () => ['read']),` was producing an object that
looked like :
```
{
"transaction": [
"read"
],
"span": [
"read"
],
"error": [
"read"
],
"metric": [
"read"
],
"onboarding": [
"read"
],
"sourcemap": [
"read"
]
}
```
It seems this ought to check privileges on the actual indices, so this
PR restructures the `index` parameter to this :
```
{
"traces-apm*": [
"read"
],
"apm-*": [
"read"
],
"traces-*.otel-*": [
"read"
],
"logs-apm*": [
"read"
],
"logs-*.otel-*": [
"read"
],
"metrics-apm*": [
"read"
],
"metrics-*.otel-*": [
"read"
]
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Adds `@elastic/platform-analytics` as codeowners of the telemetry
schemas to raise awareness of any changes implemented in the snapshot
telemetry.
The end goal is to remove Kibana Core from owning them.
**This is a follow-up to:https://github.com/elastic/kibana/pull/211045**
## Summary
This PR removes inefficiencies in prebuilt rule installation memory
consumption.
### Before
In the worst-case scenario:
1. All currently installed prebuilt rules were fully loaded into memory.
2. All latest rule versions from the rule packages were fully loaded
into memory.
3. All base rule versions were pulled into memory, even though they were
not used.
4. The algorithm then checked which rules could be installed and
installed them all at once.
### After
1. Pull only aggregated information about installed rules (only
`rule_id` and `versions`).
2. Pull the same lightweight aggregated info about the latest rules in
the package.
3. Using the collected `rule_id`s, fetch rule assets and install them in
small batches of up to 100 rules.
## Summary
Fixes#106553
This PR enables the Read Only editor feature for Lens panels, who will
let users in read mode (no matter broader permissions) to explore the
visualization configuration.
Short list of changes:
* Edit action tooltip now changed from `Edit {name}` into `Edit {name}
configuration`
* `isEditingEnabled` takes into account now also `Managed` state of both
visualization and `parentApi`
* A new `showConfigAction` has been created to show users without write
capabilities the current Lens chart configuration
* Edit inline flyout title changed to `Configuration` no matter the
context (this has impact also on creation, i.e. ES|QL new panel)
* Within the configuration panel the `Visualization configuration`
section title has changed to `Visualization layers`
* When the panel is in read-only mode a callout is shown and no
editing/saving action is shown
## UX guidance
Here's some guidance [inherited by @MichaelMarcialis
comment](https://github.com/elastic/kibana/pull/208554#issuecomment-2666551818)
about the different flows based on user permissions.
**Read/write UX**
* No change
**Read-only UX**
* The glasses icon's tooltip shows as "View visualization
configuration"?
* Flyout title should simply be "Configuration"
* On second read, "Read only panel changes will revert after closing"
sounds a bit odd. Can we change to "Read-only: Changes will be reverted
on close"? Also, can we change the callout icon to glasses?
* Change "Visualization configuration" accordion title to "Visualization
layers".
### Screenshots
**Read-only UX**
If user has no write permissions the `glasses` icon is shown for the
action:
And the panel is shown with the `read only` callout with no edit
buttons:
For a `Managed` dashboard the behaviour is the same as above (for the
user there's no difference between regular or managed dashboard, just
wanted to report here both cases):
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Marco Vettorello <vettorello.marco@gmail.com>
Co-authored-by: Michael Marcialis <michael.l.marcialis@gmail.com>
## Summary
This PR renames the path for the following advanced option from
`windows.advanced.events.registry.enforce_registry_filters` to
`windows.advanced.events.enforce_registry_filters` in order to harmonize
with Endpoint.
migration is not added, see this comment for rationale:
https://github.com/elastic/kibana/issues/212526#issuecomment-2724023199
### Release note
Elastic Defend package policy's ineffective advanced option
`enforce_registry_filters` is replaced with a new field which now
provides the expected effect.
Manual act is needed from users to fill the new field, while the old
field is still visible in policy response.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Originally we had intended to have the prebuilt rule diff algorithms
merge non-functional fields when the field's base version was missing
and a rule was marked as customized as described in
https://github.com/elastic/kibana/issues/210358
> - When the rule has a missing base version and is marked as
customized:
> - We should attempt to merge all non-functional mergeable fields (any
field that doesn't have consequences with how the rule runs e.g. tags)
and return them as SOLVABLE_CONFLICT
We ended up changing this logic to return the `Target` version for every
field that fit that description
(https://github.com/elastic/kibana/pull/214161 and
https://github.com/elastic/kibana/pull/213757) besides `tags` and in
order to support consistency rather than a very minor edge case, we now
just return the target version for every field with a missing base
version and let users sort it out on their end
This PR reverts the changes made to accommodate this edge case and
updates related tests to account for the new logic
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
