## [Security Solution] [Attack discovery] Display error messages for invalid Anonymization configurations
This PR detects invalid Anonymization configurations when generating Attack discoveries.
It displays an error message when:
- The Security AI Anonymization settings are configured to not allow _any_ fields, as reported in <https://github.com/elastic/kibana/issues/214310>
- The Security AI Anonymization settings are configured to not allow the `_id` field, as reported by @aarju
### Out of scope: detecting configurations that don't include enough useful fields
The default fields in the Security AI Anonymization settings were chosen because they are most likely to provide relevant context for the AI Assistant and Attack discovery.
However, there isn't a well defined threshold for the minimum set of useful fields.
For example, Attack discovery may still produce useful results (depending on the data), if the `user.name`, `host.name`, and `source.ip` fields are not allowed, but in most cases omitting these important fields will reduce the quality of results.
Another example: A configuration that **only** allows just **two** fields, for example the `_id` field AND `user.name` fields is valid, but NOT _useful_.
- Detecting configurations that don't include enough _useful_ fields is beyond the scope of this PR
- Configurations that **only** allow the `_id` field are _valid_, but not _useful_
#### Desk testing
1. Navigate to Stack Management > AI Assistants > Security
2. Configure the Security AI Anonymization settings to deny all fields.
Note: At the time of this writing, using the bulk actions to update all `102` default fields may result in an `Unable to load page` error that appears below the table when it is saved. Refreshing the page reveals the settings are not updated after clicking save. As a workaround for this separate, unrelated issue, apply bulk actions to only one page at a time, and be sure to refresh the page after saving changes to verify the update(s) before continuing to the next step.
3. Navigate to Security > Attack discovery
4. Click `Generate`
**Expected result**
- The following error message is displayed:
```
Your Security AI Anonymization settings are configured to not allow any fields. Fields must be allowed to generate Attack discoveries.
```
as illustrated by the screenshot below:
5. Once again, navigate to Stack Management > AI Assistants > Security
6. Allow all the (`102`) default fields
7. Once again, navigate to Security > Attack discovery
8. Click `Generate`
**Expected result**
- Attack discoveries are generated
9. Navigate back to Stack Management > AI Assistants > Security
10. Configure the `_id` field, and another (arbitrary) field to NOT be allowed
11. Navigate back to Security > Attack discovery
12. Click `Generate`
**Expected result**
- The following error message is displayed:
```
Your Security AI Anonymization settings are configured to not allow the _id field. The _id field must be allowed to generate Attack discoveries.
```
as illustrated by the screenshot below:
We decided to not enforce additive parsing. This means it's OK for the
generated patterns to override @timestamp or message directly.
This PR removes the special handling.
**Partially addresses:** #211808,
https://github.com/elastic/security-docs/issues/5981 (internal)
**Resolves: #208329**
## Summary
This is the second part of the migration effort, containing changes for:
- BULK CRUD (removing, for v.9.0)
The PR also contains changes for ticket #208329 - as changes for
removing of dead code for handling Bulk CRUD endpoints had to be
combined together with removing the schema files for Bulk CRUD
endpoints.
This PR will be backported only to versions for Kibana v9
# Testing
1. cd x-pack/solutions/security/plugins/security_solution
2. yarn openapi:bundle:detections
3. Take the bundled file
(docs/openapi/ess/security_solution_detections_api_2023_10_31.bundled.schema.yaml)
and load it into bump.sh console to see the changes.
4. Compare the changes with the [Legacy
documentation](https://www.elastic.co/guide/en/security/current/rule-api-overview.html)
You can also use this [link](https://bump.sh/jkelas2/doc/kibana_wip2/)
where I deployed the generated bundled doc.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Follow up to https://github.com/elastic/kibana/pull/212694. This PR
switches more DE functions over to use `sharedParams`, an object that
contains the variety of parameters that we build at the top level of
rule execution in the common security rule wrapper. With `sharedParams`
available throughout rule execution, it's easier to access all the
parameters necessary for `wrapHits` when we call it so I also removed
the "factory" logic and just call `wrapHits` directly instead of passing
the function as a parameter on `sharedParams`.
There should be very few behavior changes in the code as a result of
this PR.
- `kibana.alert.rule.indices` is now populated for ES|QL alerts
- `ignoreFields` and `ignoreFieldsRegexes` are now respected by EQL, new
terms, and all suppressed rule types
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
`apmIndices` has the form:
```
"transaction": "traces-apm*,apm-*,traces-*.otel-*",
"span": "traces-apm*,apm-*,traces-*.otel-*",
"error": "logs-apm*,apm-*,logs-*.otel-*",
"metric": "metrics-apm*,apm-*,metrics-*.otel-*",
"onboarding": "apm-*",
"sourcemap": "apm-*"
```
`mapValues(apmIndices, () => ['read']),` was producing an object that
looked like :
```
{
"transaction": [
"read"
],
"span": [
"read"
],
"error": [
"read"
],
"metric": [
"read"
],
"onboarding": [
"read"
],
"sourcemap": [
"read"
]
}
```
It seems this ought to check privileges on the actual indices, so this
PR restructures the `index` parameter to this :
```
{
"traces-apm*": [
"read"
],
"apm-*": [
"read"
],
"traces-*.otel-*": [
"read"
],
"logs-apm*": [
"read"
],
"logs-*.otel-*": [
"read"
],
"metrics-apm*": [
"read"
],
"metrics-*.otel-*": [
"read"
]
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
**This is a follow-up to:https://github.com/elastic/kibana/pull/211045**
## Summary
This PR removes inefficiencies in prebuilt rule installation memory
consumption.
### Before
In the worst-case scenario:
1. All currently installed prebuilt rules were fully loaded into memory.
2. All latest rule versions from the rule packages were fully loaded
into memory.
3. All base rule versions were pulled into memory, even though they were
not used.
4. The algorithm then checked which rules could be installed and
installed them all at once.
### After
1. Pull only aggregated information about installed rules (only
`rule_id` and `versions`).
2. Pull the same lightweight aggregated info about the latest rules in
the package.
3. Using the collected `rule_id`s, fetch rule assets and install them in
small batches of up to 100 rules.
## Summary
Fixes#106553
This PR enables the Read Only editor feature for Lens panels, who will
let users in read mode (no matter broader permissions) to explore the
visualization configuration.
Short list of changes:
* Edit action tooltip now changed from `Edit {name}` into `Edit {name}
configuration`
* `isEditingEnabled` takes into account now also `Managed` state of both
visualization and `parentApi`
* A new `showConfigAction` has been created to show users without write
capabilities the current Lens chart configuration
* Edit inline flyout title changed to `Configuration` no matter the
context (this has impact also on creation, i.e. ES|QL new panel)
* Within the configuration panel the `Visualization configuration`
section title has changed to `Visualization layers`
* When the panel is in read-only mode a callout is shown and no
editing/saving action is shown
## UX guidance
Here's some guidance [inherited by @MichaelMarcialis
comment](https://github.com/elastic/kibana/pull/208554#issuecomment-2666551818)
about the different flows based on user permissions.
**Read/write UX**
* No change
**Read-only UX**
* The glasses icon's tooltip shows as "View visualization
configuration"?
* Flyout title should simply be "Configuration"
* On second read, "Read only panel changes will revert after closing"
sounds a bit odd. Can we change to "Read-only: Changes will be reverted
on close"? Also, can we change the callout icon to glasses?
* Change "Visualization configuration" accordion title to "Visualization
layers".
### Screenshots
**Read-only UX**
If user has no write permissions the `glasses` icon is shown for the
action:
And the panel is shown with the `read only` callout with no edit
buttons:
For a `Managed` dashboard the behaviour is the same as above (for the
user there's no difference between regular or managed dashboard, just
wanted to report here both cases):
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Marco Vettorello <vettorello.marco@gmail.com>
Co-authored-by: Michael Marcialis <michael.l.marcialis@gmail.com>
## Summary
This PR renames the path for the following advanced option from
`windows.advanced.events.registry.enforce_registry_filters` to
`windows.advanced.events.enforce_registry_filters` in order to harmonize
with Endpoint.
migration is not added, see this comment for rationale:
https://github.com/elastic/kibana/issues/212526#issuecomment-2724023199
### Release note
Elastic Defend package policy's ineffective advanced option
`enforce_registry_filters` is replaced with a new field which now
provides the expected effect.
Manual act is needed from users to fill the new field, while the old
field is still visible in policy response.
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
Originally we had intended to have the prebuilt rule diff algorithms
merge non-functional fields when the field's base version was missing
and a rule was marked as customized as described in
https://github.com/elastic/kibana/issues/210358
> - When the rule has a missing base version and is marked as
customized:
> - We should attempt to merge all non-functional mergeable fields (any
field that doesn't have consequences with how the rule runs e.g. tags)
and return them as SOLVABLE_CONFLICT
We ended up changing this logic to return the `Target` version for every
field that fit that description
(https://github.com/elastic/kibana/pull/214161 and
https://github.com/elastic/kibana/pull/213757) besides `tags` and in
order to support consistency rather than a very minor edge case, we now
just return the target version for every field with a missing base
version and let users sort it out on their end
This PR reverts the changes made to accommodate this edge case and
updates related tests to account for the new logic
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
## Summary
fixes https://github.com/elastic/kibana/issues/210198
Fix rendering issue with annotations !!
This has been done because react suspense doesn't plays nicely with
elastic/chart rendering. And we have to render annotation with charts
dynamically.
## Summary
Fixes: https://github.com/elastic/kibana/issues/213754
The issue above describes a bug in timeline that makes it impossible to
change the width of a timeline column. This PR fixes that issue and
makes sure that timeline column width settings are saved to
localStorage. This mimics the behaviour of the alerts table elsewhere in
security solution.
https://github.com/user-attachments/assets/8b9803a0-406d-4f2d-ada5-4c0b76cd6ab8
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Resolves#213464
## Release Notes
SpaceId can now be referenced in document-based access filters for roles
to restrict a user's access to SLI data for spaces where they do not
have access.
Part of https://github.com/elastic/kibana/issues/200725
This PR adds telemetry to track model downloads and deployment updates.
It also includes tracking for failed deployments, as the previous
implementation only tracked successful deployments.
## Summary
Resolves first part of https://github.com/elastic/kibana/issues/198783
(snooze API)
Creates a public API for adding snooze schedule to rules.
For this purpose we created new schedule schema which will be used as
standard schedule schema across rules and alerting framework.
**Note**
The code to be reviewed for public API is under `common/routes/schedule`
and inside `external` folders.
Rest of the code is just moving existing internal API route and its code
to `/internal` folders.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### How to test
- Create a rule in kibana
- Snooze that rule via new public API
```typescript
Path: https://localhost:5601/api/alerting/rule/<rule_id>/snooze_schedule
Method: POST
Body:
{
"schedule": {
"custom": {
"start": "2025-02-25T18:00:00.000Z",
"duration": "15m",
"recurring": {
"every": "1w",
"onWeekDay": ["TU", "FR"],
"occurrences": 10
}
}
}
}
```
- Verify various snooze schedule scenarios are generated correctly
#### Flaky test runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8009
### Release note
Allow users to create a snooze schedule for rule via API
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: lcawl <lcawley@elastic.co>
## Summary
Both Storybook and the theme switcher addon use Amsterdam by default.
This PR adds Borealis to the theme switcher and defaults it to Borealis.
## NOTE
This PR may conflict with #195148 ... it should likely be merged into
that PR, or into `main` if the conflict is minor. I leave it to @Ikuni17
to determine the best path forward.
## Summary
Fix broken tests !!
These got broken due to changes on alerts overview page, i am also
expanding the scope to run on all observability plugin changes !!
**Resolves:** https://github.com/elastic/kibana/issues/214302
## Summary
This PR prevents showing rule upgrade confirmation modal on lower licenses where prebuilt rules customization is not allowed.
## Details
Users may see a rules upgrade confirmation modal when trying to upgrade prebuilt rules even if prebuilt rules customization is disabled due to insufficient license. It happens due to improper response from `upgrade/_perform` which doesn't respect `pick_version`. It's expected rule upgrade goes smoothly when `pick_version` is one of `BASE`, `CURRENT` or `TARGET`.
The fix makes sure dry run request isn't fired and a prebuilt rules upgrade confirmation modal isn't shown when running with insufficient for prebuilt rules customization license.
There is a [ticket](https://github.com/elastic/kibana/issues/214338) to address this issue in the API endpoint.
Add streams API to documentation as an experimental feature
<img width="2555" alt="Screenshot 2025-03-07 at 11 44 54"
src="https://github.com/user-attachments/assets/f54e5e6e-0c20-4bad-9cff-27747d0f76e2"
/>
There are a couple of changes in here:
* Split streams API in internal and public and mark the public parts as
experimental
* Add the public parts to the Kibana documentation
* Add description and summary
* Adjust the server repository wrapper to pass through summary and
description
# To test
* Generate OAS bundle: `node scripts/capture_oas_snapshot --include-path
/api/streams --update`
* Apply overlays `cd oas_docs && make api-docs`
* Make sure bump.sh is installed (`npm install -g bump-cli`)
* Run for preview: `cd oas_docs && bump preview output/kibana.yaml`
# Open questions
* Does the split into public and internal make sense?
* Is it a problem if this is visible in the user-facing documentation
page before we actually release streams? Or would it be OK if the API is
marked as experimental? (mostly a question for @LucaWintergerst )
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The following changes are being done to Artifact Card's Menu (which
displays the option to Delete or Update the artifact) in support of
space awareness feature (currently behind Feature Flag:
`endpointManagementSpaceAwarenessEnabled`):
- Global Artifacts: If displaying a global artifact and user does not
have the new Global Artifact Management privilege - disable the Edit
menu icon and display a tooltip on hover
- Per-Policy Artifacts: if displaying a per-policy artifact in a space
other than one of the `ownerSpaceId` spaces that the artifact is
associated with and the user does not have the new Global Artifact
Management privilege - disable the Edit menu icon and display a tooltip
when the user hover over that button
> [!NOTE]
> Changes were **NOT** done to Endpoint Exceptions with this PR.
