## [Security Solution] [Attack discovery] Display error messages for invalid Anonymization configurations
This PR detects invalid Anonymization configurations when generating Attack discoveries.
It displays an error message when:
- The Security AI Anonymization settings are configured to not allow _any_ fields, as reported in <https://github.com/elastic/kibana/issues/214310>
- The Security AI Anonymization settings are configured to not allow the `_id` field, as reported by @aarju
### Out of scope: detecting configurations that don't include enough useful fields
The default fields in the Security AI Anonymization settings were chosen because they are most likely to provide relevant context for the AI Assistant and Attack discovery.
However, there isn't a well defined threshold for the minimum set of useful fields.
For example, Attack discovery may still produce useful results (depending on the data), if the `user.name`, `host.name`, and `source.ip` fields are not allowed, but in most cases omitting these important fields will reduce the quality of results.
Another example: A configuration that **only** allows just **two** fields, for example the `_id` field AND `user.name` fields is valid, but NOT _useful_.
- Detecting configurations that don't include enough _useful_ fields is beyond the scope of this PR
- Configurations that **only** allow the `_id` field are _valid_, but not _useful_
#### Desk testing
1. Navigate to Stack Management > AI Assistants > Security
2. Configure the Security AI Anonymization settings to deny all fields.
Note: At the time of this writing, using the bulk actions to update all `102` default fields may result in an `Unable to load page` error that appears below the table when it is saved. Refreshing the page reveals the settings are not updated after clicking save. As a workaround for this separate, unrelated issue, apply bulk actions to only one page at a time, and be sure to refresh the page after saving changes to verify the update(s) before continuing to the next step.
3. Navigate to Security > Attack discovery
4. Click `Generate`
**Expected result**
- The following error message is displayed:
```
Your Security AI Anonymization settings are configured to not allow any fields. Fields must be allowed to generate Attack discoveries.
```
as illustrated by the screenshot below:
5. Once again, navigate to Stack Management > AI Assistants > Security
6. Allow all the (`102`) default fields
7. Once again, navigate to Security > Attack discovery
8. Click `Generate`
**Expected result**
- Attack discoveries are generated
9. Navigate back to Stack Management > AI Assistants > Security
10. Configure the `_id` field, and another (arbitrary) field to NOT be allowed
11. Navigate back to Security > Attack discovery
12. Click `Generate`
**Expected result**
- The following error message is displayed:
```
Your Security AI Anonymization settings are configured to not allow the _id field. The _id field must be allowed to generate Attack discoveries.
```
as illustrated by the screenshot below:
We decided to not enforce additive parsing. This means it's OK for the
generated patterns to override @timestamp or message directly.
This PR removes the special handling.
**Partially addresses:** #211808,
https://github.com/elastic/security-docs/issues/5981 (internal)
**Resolves: #208329**
## Summary
This is the second part of the migration effort, containing changes for:
- BULK CRUD (removing, for v.9.0)
The PR also contains changes for ticket #208329 - as changes for
removing of dead code for handling Bulk CRUD endpoints had to be
combined together with removing the schema files for Bulk CRUD
endpoints.
This PR will be backported only to versions for Kibana v9
# Testing
1. cd x-pack/solutions/security/plugins/security_solution
2. yarn openapi:bundle:detections
3. Take the bundled file
(docs/openapi/ess/security_solution_detections_api_2023_10_31.bundled.schema.yaml)
and load it into bump.sh console to see the changes.
4. Compare the changes with the [Legacy
documentation](https://www.elastic.co/guide/en/security/current/rule-api-overview.html)
You can also use this [link](https://bump.sh/jkelas2/doc/kibana_wip2/)
where I deployed the generated bundled doc.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Follow up to https://github.com/elastic/kibana/pull/212694. This PR
switches more DE functions over to use `sharedParams`, an object that
contains the variety of parameters that we build at the top level of
rule execution in the common security rule wrapper. With `sharedParams`
available throughout rule execution, it's easier to access all the
parameters necessary for `wrapHits` when we call it so I also removed
the "factory" logic and just call `wrapHits` directly instead of passing
the function as a parameter on `sharedParams`.
There should be very few behavior changes in the code as a result of
this PR.
- `kibana.alert.rule.indices` is now populated for ES|QL alerts
- `ignoreFields` and `ignoreFieldsRegexes` are now respected by EQL, new
terms, and all suppressed rule types
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
`apmIndices` has the form:
```
"transaction": "traces-apm*,apm-*,traces-*.otel-*",
"span": "traces-apm*,apm-*,traces-*.otel-*",
"error": "logs-apm*,apm-*,logs-*.otel-*",
"metric": "metrics-apm*,apm-*,metrics-*.otel-*",
"onboarding": "apm-*",
"sourcemap": "apm-*"
```
`mapValues(apmIndices, () => ['read']),` was producing an object that
looked like :
```
{
"transaction": [
"read"
],
"span": [
"read"
],
"error": [
"read"
],
"metric": [
"read"
],
"onboarding": [
"read"
],
"sourcemap": [
"read"
]
}
```
It seems this ought to check privileges on the actual indices, so this
PR restructures the `index` parameter to this :
```
{
"traces-apm*": [
"read"
],
"apm-*": [
"read"
],
"traces-*.otel-*": [
"read"
],
"logs-apm*": [
"read"
],
"logs-*.otel-*": [
"read"
],
"metrics-apm*": [
"read"
],
"metrics-*.otel-*": [
"read"
]
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Adds `@elastic/platform-analytics` as codeowners of the telemetry
schemas to raise awareness of any changes implemented in the snapshot
telemetry.
The end goal is to remove Kibana Core from owning them.
**This is a follow-up to:https://github.com/elastic/kibana/pull/211045**
## Summary
This PR removes inefficiencies in prebuilt rule installation memory
consumption.
### Before
In the worst-case scenario:
1. All currently installed prebuilt rules were fully loaded into memory.
2. All latest rule versions from the rule packages were fully loaded
into memory.
3. All base rule versions were pulled into memory, even though they were
not used.
4. The algorithm then checked which rules could be installed and
installed them all at once.
### After
1. Pull only aggregated information about installed rules (only
`rule_id` and `versions`).
2. Pull the same lightweight aggregated info about the latest rules in
the package.
3. Using the collected `rule_id`s, fetch rule assets and install them in
small batches of up to 100 rules.
## Summary
Fixes#106553
This PR enables the Read Only editor feature for Lens panels, who will
let users in read mode (no matter broader permissions) to explore the
visualization configuration.
Short list of changes:
* Edit action tooltip now changed from `Edit {name}` into `Edit {name}
configuration`
* `isEditingEnabled` takes into account now also `Managed` state of both
visualization and `parentApi`
* A new `showConfigAction` has been created to show users without write
capabilities the current Lens chart configuration
* Edit inline flyout title changed to `Configuration` no matter the
context (this has impact also on creation, i.e. ES|QL new panel)
* Within the configuration panel the `Visualization configuration`
section title has changed to `Visualization layers`
* When the panel is in read-only mode a callout is shown and no
editing/saving action is shown
## UX guidance
Here's some guidance [inherited by @MichaelMarcialis
comment](https://github.com/elastic/kibana/pull/208554#issuecomment-2666551818)
about the different flows based on user permissions.
**Read/write UX**
* No change
**Read-only UX**
* The glasses icon's tooltip shows as "View visualization
configuration"?
* Flyout title should simply be "Configuration"
* On second read, "Read only panel changes will revert after closing"
sounds a bit odd. Can we change to "Read-only: Changes will be reverted
on close"? Also, can we change the callout icon to glasses?
* Change "Visualization configuration" accordion title to "Visualization
layers".
### Screenshots
**Read-only UX**
If user has no write permissions the `glasses` icon is shown for the
action:
And the panel is shown with the `read only` callout with no edit
buttons:
For a `Managed` dashboard the behaviour is the same as above (for the
user there's no difference between regular or managed dashboard, just
wanted to report here both cases):
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Marco Vettorello <vettorello.marco@gmail.com>
Co-authored-by: Michael Marcialis <michael.l.marcialis@gmail.com>
## Summary
This PR renames the path for the following advanced option from
`windows.advanced.events.registry.enforce_registry_filters` to
`windows.advanced.events.enforce_registry_filters` in order to harmonize
with Endpoint.
migration is not added, see this comment for rationale:
https://github.com/elastic/kibana/issues/212526#issuecomment-2724023199
### Release note
Elastic Defend package policy's ineffective advanced option
`enforce_registry_filters` is replaced with a new field which now
provides the expected effect.
Manual act is needed from users to fill the new field, while the old
field is still visible in policy response.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Originally we had intended to have the prebuilt rule diff algorithms
merge non-functional fields when the field's base version was missing
and a rule was marked as customized as described in
https://github.com/elastic/kibana/issues/210358
> - When the rule has a missing base version and is marked as
customized:
> - We should attempt to merge all non-functional mergeable fields (any
field that doesn't have consequences with how the rule runs e.g. tags)
and return them as SOLVABLE_CONFLICT
We ended up changing this logic to return the `Target` version for every
field that fit that description
(https://github.com/elastic/kibana/pull/214161 and
https://github.com/elastic/kibana/pull/213757) besides `tags` and in
order to support consistency rather than a very minor edge case, we now
just return the target version for every field with a missing base
version and let users sort it out on their end
This PR reverts the changes made to accommodate this edge case and
updates related tests to account for the new logic
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| docker.elastic.co/wolfi/chainguard-base | digest | `69a0542` ->
`3f6bcf4` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOk9wZXJhdGlvbnMiLCJiYWNrcG9ydDpza2lwIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [@hapi/hapi](https://hapi.dev)
([source](https://redirect.github.com/hapijs/hapi)) | dependencies |
minor | [`^21.3.12` ->
`^21.4.0`](https://renovatebot.com/diffs/npm/@hapi%2fhapi/21.3.12/21.4.0)
|
---
### Release Notes
<details>
<summary>hapijs/hapi (@​hapi/hapi)</summary>
###
[`v21.4.0`](https://redirect.github.com/hapijs/hapi/compare/v21.3.12...v21.4.0)
[Compare
Source](https://redirect.github.com/hapijs/hapi/compare/v21.3.12...v21.4.0)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDpwcmV2LW1pbm9yIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
---------
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
fixes https://github.com/elastic/kibana/issues/210198
Fix rendering issue with annotations !!
This has been done because react suspense doesn't plays nicely with
elastic/chart rendering. And we have to render annotation with charts
dynamically.
## Summary
Fixes: https://github.com/elastic/kibana/issues/213754
The issue above describes a bug in timeline that makes it impossible to
change the width of a timeline column. This PR fixes that issue and
makes sure that timeline column width settings are saved to
localStorage. This mimics the behaviour of the alerts table elsewhere in
security solution.
https://github.com/user-attachments/assets/8b9803a0-406d-4f2d-ada5-4c0b76cd6ab8
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Resolves#213464
## Release Notes
SpaceId can now be referenced in document-based access filters for roles
to restrict a user's access to SLI data for spaces where they do not
have access.
Part of https://github.com/elastic/kibana/issues/200725
This PR adds telemetry to track model downloads and deployment updates.
It also includes tracking for failed deployments, as the previous
implementation only tracked successful deployments.
