## Summary
This PR introduces the first building blocks for the [Entity Analytics
Privileged
Monitoring](https://github.com/elastic/security-team/issues/9971).
We follow the approach used in the Entity Store and add a new "Engine",
which consists of the following components:
* Public API
* INIT and HEALTH routes
* Kibana task
* Privilege Monitoring Data Client
* Engine Saved Object
* API key manager
* Related storage indices
* Feature Flag: `privilegeMonitoringEnabled` set to `false` by default.
* API integration test configuration
* only tests that the health endpoint is available
* Auditing and Telemetry
## Testing steps
1. Make sure to add `privilegeMonitoringEnabled` to your
`kibana.dev.yaml`
2. In devtools, ensure the API is working with `GET
kbn:/api/entity_analytics/monitoring/privileges/health`
3. Start the engine with: `POST
kbn:/api/entity_analytics/monitoring/engine/init`
4. Look for `DEBUG` logs mentioning the
`entity_analytics:monitoring:privileges:engine` task
---------
Co-authored-by: CAWilson94 <charlotte.wilson@elastic.co>
Co-authored-by: Charlotte Alexandra Wilson <CAWilson94@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/206556
This PR adds a setting to remote ES outputs for also uninstalling
integrations on remote clusters when integrations sync is enabled.
This new setting can be toggled in the UI with a new switch:
<img width="1728" alt="Screenshot 2025-04-09 at 11 53 43"
src="https://github.com/user-attachments/assets/34544aa9-28fd-4360-a32f-5031e3d4293f"
/>
### Testing
* Follow the steps in
https://github.com/elastic/kibana/blob/main/x-pack/platform/plugins/shared/fleet/dev_docs/local_setup/remote_clusters_ccr.md
to set up two clusters with integrations syncing.
* Add some integrations in your main cluster and check that they are
also installed in the remote cluster.
* Disable uninstalling integrations on remote.
* Remove an integration in your main cluster and check that it is NOT
removed from the remote cluster.
* Enable uninstalling integrations on remote.
* Remove an integration in your main cluster and check that it is also
removed from the remote cluster.
* In your remote cluster, enroll an agent onto a policy that points to
at least 1 package policy of the installed integrations (cf. Docker
commands below if using dockerized fleet-server/agent).
* In your main cluster, uninstall the integration that is used by the
agent policy in the remote. This should cause the uninstall to fail into
the remote cluster.
* In your remote cluster, inspect the package SO of that integration
with `GET .kibana_ingest/_search?q=type:epm-packages`: the
`latest_uninstall_failed_attempts` field should be populated.
Docker command for running a fleet-server in your remote cluster:
```
docker run \
-e ELASTICSEARCH_HOST=http://host.docker.internal:9500 \
-e KIBANA_HOST=http://host.docker.internal:5701/<path> \
-e KIBANA_USERNAME=elastic \
-e KIBANA_PASSWORD=changeme \
-e KIBANA_FLEET_SETUP=1 \
-e FLEET_INSECURE=1 \
-e FLEET_SERVER_ENABLE=1 \
-e FLEET_SERVER_POLICY_ID=fleet-server-policy \
-p 8220:8220 \
--rm docker.elastic.co/beats/elastic-agent:9.0.0-SNAPSHOT
```
Docker command for enrolling an agent in your remote cluster:
```
docker run \
-e ELASTICSEARCH_HOST=http://host.docker.internal:9500 \
-e KIBANA_HOST=http://host.docker.internal:5701/<path> \
-e FLEET_URL=https://host.docker.internal:8220 \
-e FLEET_ENROLL=1 \
-e FLEET_ENROLLMENT_TOKEN=<token> \
-e FLEET_INSECURE=1 \
--rm docker.elastic.co/beats/elastic-agent:9.0.0-SNAPSHOT
```
### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
This feature is currently in development and behind the
`enableSyncIntegrationsOnRemote` feature flag.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**Resolves: https://github.com/elastic/security-docs/issues/3019**
## Summary
In this PR I am improving documentation for the bulk_actions endpoint.
The focus is on improving descriptions of parameters and providing
examples for all important situations, especially setting / adding
actions.
# Testing
1. cd x-pack/solutions/security/plugins/security_solution
2. yarn openapi:bundle:detections
3. Take the bundled file
(docs/openapi/ess/security_solution_detections_api_2023_10_31.bundled.schema.yaml)
and load it into bump.sh console to see the changes.
4. Compare the changes with the [Legacy
documentation](https://www.elastic.co/guide/en/security/current/rule-api-overview.html)
You can also use this [link](https://bump.sh/jkelas/doc/kibana_wip/)
where I deployed the generated bundled doc.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/192363
## Summary
Add endpoint that compares integrations installed on remote cluster with
integrations in ccr index `fleet-synced-integrations-ccr-<outputId>`.
Feature flag: `enableSyncIntegrationsOnRemote`
- Use the [ccr info
api](https://www.elastic.co/guide/en/elasticsearch/reference/current/ccr-get-follow-info.html)
to check that the ccr index is active
- Compare the content of the two indices and report the sync status for
each integration:
```
GET kbn:/api/fleet/remote_synced_integrations/status
{
"integrations": [
{
"package_name": "akamai",
"package_version": "2.28.0",
"updated_at": "2025-03-27T10:29:52.485Z",
"sync_status": true
},
{
"package_name": "auth0",
"package_version": "1.21.0",
"updated_at": "2025-03-26T12:06:26.268Z",
"sync_status": false,
"error": "Installation status: not_installed"
},
]
```
- Check that the integrations have custom assets and report their sync
status (follow up of https://github.com/elastic/kibana/pull/215504)
### Testing
Setup local env with the guide added in dev_docs
([preview](b438a719ee/x-pack/platform/plugins/shared/fleet/dev_docs/local_setup/remote_clusters_ccr.md))
- Install some integrations on local cluster, wait that they are synced
on remote
- From remote cluster dev tools, run
```
GET kbn:/api/fleet/remote_synced_integrations/status
```
- To verify that custom assets are synced choose an integration, for
instance `system`
- From the package policy select a var, advanced options and add a
custom mapping and a custom pipeline. In my example I used `system`
<img width="1195" alt="Screenshot 2025-04-01 at 11 18 40"
src="https://github.com/user-attachments/assets/c339d6f0-4404-4e9b-b489-7b37977f201b"
/>
- Run the endpoint again and you should see the status of custom assets
too:
```
{
"integrations": [
{
"package_name": "akamai",
"package_version": "2.28.0",
"updated_at": "2025-03-27T10:29:52.485Z",
"sync_status": "completed"
},
{
"package_name": "elastic_agent",
"package_version": "2.2.0",
"updated_at": "2025-03-26T14:06:29.216Z",
"sync_status": "completed"
},
{
"package_name": "synthetics",
"package_version": "1.4.1",
"updated_at": "2025-03-26T14:06:31.909Z",
"sync_status": "completed"
},
{
"package_name": "system",
"package_version": "1.67.3",
"updated_at": "2025-03-28T10:08:00.602Z",
"sync_status": "completed"
}
],
"custom_assets": {
"component_template:logs-system.auth@custom": {
"name": "logs-system.auth@custom",
"type": "component_template",
"package_name": "system",
"package_version": "1.67.3",
"sync_status": "completed"
},
"ingest_pipeline:logs-system.auth@custom": {
"name": "logs-system.auth@custom",
"type": "ingest_pipeline",
"package_name": "system",
"package_version": "1.67.3",
"sync_status": "completed"
}
}
}
```
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes
[4718](https://github.com/elastic/ingest-dev/issues/4718?reload=1?reload=1?reload=1%3Freload%3D1)
## Summary
- After much discussion, updated to not show percentages but be very
similar to existing upgrade text, but with an icon/tooltip letting the
user know it was created from an automatic upgrade. Also applied to
completed actions.
- Made `policyId` persistent in newly created actions in order to
streamline accessing the upgrade modal from agent activity
- Added `is_automatic` field to `ActionStatus` type by retrieving from
the source doc in `getActions`
- Updated audit log to show if the action was created by the user or
from the auto-upgrade functionality
- Updated badging on table to show retry attempts
- Added `manage auto-upgrade` button to agent activity actions created
by automatic upgrades
- Updated check in `automatic_upgrade_task` to only consider active
agents in order to resolve an issue where uninstalled agents could
affect the upgrade of new ones.
- Reworked rounding functionality when percentages or counts of agents
to upgrade were over or under where they should be. Rounding is now done
in a way such that no agents get left behind, and we dont try to upgrade
more than exist.
- Added new test coverage for the rounding functionality, the new active
vs inactive agents check, as well as ensuring the manage auto-upgrades
button always renders.
Simplified UI with tooltip and button to quickly access auto-upgrade
settings for the policy the action belongs to:
Updated tooltip to let the user know that rounding is in place:
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
N/A
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
* Removes manually maintained OAS for Core's `_export` and `_import`
Saved Object APIs
* seeks to remove some code duplication and move our OAS descriptions
closer to the code definitions.
## Summary
Some users using the security AI assistant through the API may want to
disable content references (aka citations) programmatically. This PR
allows users to disable content references using the
`content_references_disabled` query arg in the following routes:
- POST
/internal/elastic_assistant/actions/connector/gpt4oAzure/_execute?content_references_disabled=true
- POST
/api/security_ai_assistant/chat/complete?content_references_disabled=true
## How to test:
Bellow are 2 example curl requests (with
content_references_disabled=true). When executed, you will notice that
the response stream does not contain the string `{reference(...)}`
anywhere. If you remove the query arg, the reference will be visible.
####
/internal/elastic_assistant/actions/connector/gpt4oAzure/_execute?content_references_disabled=true
```curl
curl --location 'http://localhost:5601/internal/elastic_assistant/actions/connector/gpt4oAzure/_execute?content_references_disabled=true' \
--header 'Accept: */*' \
--header 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
--header 'Connection: keep-alive' \
--header 'Content-Type: application/json' \
--header 'Origin: http://localhost:5601' \
--header 'Referer: http://localhost:5601/app/security/dashboards?sourcerer=(default:(id:security-solution-default,selectedPatterns:!()))&timeline=(activeTab:query,graphEventId:%27%27,isOpen:!f)' \
--header 'Sec-Fetch-Dest: empty' \
--header 'Sec-Fetch-Mode: cors' \
--header 'Sec-Fetch-Site: same-origin' \
--header 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36' \
--header 'elastic-api-version: 1' \
--header 'kbn-build-number: 9007199254740991' \
--header 'kbn-version: 9.1.0' \
--header 'sec-ch-ua: "Chromium";v="134", "Not:A-Brand";v="24", "Google Chrome";v="134"' \
--header 'sec-ch-ua-mobile: ?0' \
--header 'sec-ch-ua-platform: "macOS"' \
--header 'x-elastic-internal-origin: Kibana' \
--header 'x-kbn-context: %7B%22type%22%3A%22application%22%2C%22name%22%3A%22securitySolutionUI%22%2C%22url%22%3A%22%2Fapp%2Fsecurity%2Fdashboards%22%2C%22page%22%3A%22list%22%7D' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--data '{
"subAction": "invokeStream",
"actionTypeId": ".gen-ai",
"replacements": {},
"message": "What is elastic security? Include citations",
"screenContext": {
"timeZone": "Asia/Calcutta"
},
"alertsIndexPattern": ".alerts-security.alerts-default",
"size": 100
}'
```
####
/api/security_ai_assistant/chat/complete?content_references_disabled=true
```curl
curl --location 'http://localhost:5601/api/security_ai_assistant/chat/complete?content_references_disabled=true' \
--header 'Content-Type: application/json' \
--header 'Accept: application/octet-stream' \
--header 'kbn-xsrf: 123' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--data '{
"messages": [
{
"role": "user",
"content": "what is semantic text in esql. Include references to the product documentation"
}
],
"persist": false,
"connectorId": "gpt4oAzure",
"isStream": false
}'
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Currently, our `requiredPrivileges` structure supports `allRequired` and
`anyRequired` for defining authorization logic. However, there is [a
need to
support](https://github.com/elastic/kibana/pull/205335#issuecomment-2569275302)
more complex scenarios as `(privilege1 AND privilege2) OR (privilege3
AND privilege4)`
To achieve `anyRequired` has been extended to allow defining multiple
AND conditions evaluated with OR logic:
```ts
security: {
authz: {
requiredPrivileges: [{
anyRequired: [
{ allOf: ['privilege1', 'privilege2'] },
{ allOf: ['privilege3', 'privilege4'] }
]
}
]
}
}
```
`allRequired` now also supports scenarios `(privilege1 OR privilege2)
AND (privilege3 OR privilege4)`
```ts
security: {
authz: {
requiredPrivileges: [{
allRequired: [
{ anyOf: ['privilege1', 'privilege2'] },
{ anyOf: ['privilege3', 'privilege4'] }
]
}
]
}
}
```
> [!IMPORTANT]
> We expect to have unique privileges in `anyOf` or `allOf` conditions,
assuming that most complex conditions can be simplified by boolean
algebra laws (OR/AND distributive etc).
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
__Closes: https://github.com/elastic/kibana/issues/210977__
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/streams-program/issues/38.
- Adds configuration driven support for processors
- Adds support for the following processors:
- Geoip
- KV
- Rename
- Set
- URL Decode
- User agent
## Notes
- `if` is now conditional as not all processors support a condition
- With `set` you can either have a `value` OR a `copy_from`. To avoid
types, form, and validation complexity for now I've just allowed the use
of `value`.
- I didn't add `extract_device` to the user agent configuration as it's
Beta.
## Future improvements
- We could add configuration for the grok, dissect, and date processors
too. This would align things so that each processor has things "in one
place". We could extend the field types available to provide a `custom`
type, this would allow those processors to return a React component for
their specialised fields / UI (still driven by `react-hook-form` under
the hood). Overall the code footprint would decrease.
- A couple of types can possibly be refined but I wanted to try and get
this open before the Milestone 2 deadline.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The error callouts were getting squished to the side, I have moved them
to be in their respective engine section.
**Before**
<img width="1728" alt="Screenshot 2025-03-27 at 11 05 38"
src="https://github.com/user-attachments/assets/c244acd1-a75e-4b43-a509-8bff9966b31e"
/>
**After**
<img width="1728" alt="Screenshot 2025-03-27 at 19 39 53"
src="https://github.com/user-attachments/assets/2b0dd9ad-568a-4e9c-aad7-6922b2085d69"
/>
Bonus refactor:
I have added type definitions for the error object keys on the engine
status object and moved the error callout to a common component as it's
now used in 3 places.
## Test Steps
the easies way to test this is to try and init the entity store in a
space that is lobger than 45 chars long.
- create a space with a very long name e.g
'this_is_a_very_long_space_name_indeed_hello_world_123456789'
- enable the entity store from the entity analytics dashboard (it should
error)
- navigate to the entity store management page to see the error
callouts.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This change reproduces the asciidoc change made in
https://github.com/elastic/kibana/pull/209403 into the new API docs.
It'll be picked up the next time the API docs output is generated and
refreshed.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**Partially resolves: #211808**
## Summary
This is the first part of the migration effort, containing changes for:
- CRUD endpoints
- BULK Actions
- Export / Import Rule
- Find Rule
- List Tags
- Get Status
- Install Rule
I migrated the examples and the description of the fields. Some of the
fields contained description that was very similar to the legacy, then I
didn't change it. I only modified the descriptions where it was
valuable.
I also discovered some problems. For example the value for the 'query'
field, always shows 'EQL query to execute'. I reported this to the docs
team, in the 'next-api-reference' channel.
Another issue was with 'related_integrations field', which also didn't
show the long description. I also wrote about it to the docs team
[here](https://elastic.slack.com/archives/C05UL5YC06B/p1740137094701209).
In this PR I decided to try moving the description one level up, where
it renders properly.
# Testing
1. cd x-pack/solutions/security/plugins/security_solution
2. yarn openapi:bundle:detections
3. Take the bundled file
(docs/openapi/ess/security_solution_detections_api_2023_10_31.bundled.schema.yaml)
and load it into bump.sh console to see the changes.
4. Compare the changes with the [Legacy
documentation](https://www.elastic.co/guide/en/security/current/rule-api-overview.html)
You can also use this [link](https://bump.sh/jkelas/doc/kibana_wip/)
where I deployed the generated bundled doc.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**Resolves: https://github.com/elastic/security-docs/issues/3545**
## Summary
I am fixing documentation for the version and revision fields.
I used this page as the base documentation:
https://docs.elastic.dev/security-solution/dev-docs/detections/rule-versions
The changes:
- by marking the RuleVersion as read-only I am making sure we are
properly documenting that this field is not to be used in the request.
It is only returned in the response.
- by introducing RuleRevision type and marking it as read-only, I am
doing the same for this field, saying that this field is not supposed to
be used in the request.
- I am not changing any code in the app, as the ticket says we shouldn't
do any breaking changes and the update of the version should not cause
400 error. Basically current behavior is kept: users can still update
the version to whatever value they want, including going backwards, and
the changes to revision field is completely ignored.
- I am adding a condensed description of these fields.
I wanted to introduce an internal link between these two fields, but I
couldn't make it work in Bump.sh (even though this should work, normal
Markdown links) so I abandoned this idea.
You can also use this [link](https://bump.sh/jkelas2/doc/kibana_wip2)
where I deployed the generated bundled doc.
Screenshots:
<img width="664" alt="image"
src="https://github.com/user-attachments/assets/34d82eb2-f7f0-4369-ad8e-2fd3c1f35447"
/>
<img width="660" alt="image"
src="https://github.com/user-attachments/assets/dc7772af-0185-4850-816e-60be003775d6"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes#210619
## Summary
Provides a tags array on the request and response bodies of dashboards.
This allows consumers of the Dashboards HTTP API and internal RPC API to
add an array of tag names to the attributes in the body of create and
update endpoints. The dashboard server will be responsible for
converting the tag names into references in the saved object.
If, during creation or update, a tag name does not have a matching tag
saved object, a new tag saved object will be created. If the user lacks
permissions to manage tags, then an error will be logged in the server
and the tag will not be added to the dashboard.
The server also injects the tag references as an array of tag names in
the attributes of the response body of get and search endpoints of the
HTTP and RPC APIs.
For backwards compatibility in create and update endpoints, tags can
alternatively be specified in the `references` array in the options
instead of (or in addition to) the `attributes.tags` in the request
body. Similarly, for backwards compatibility, tag references are
returned in the `references` of the response body of get and search
endpoints.
Client-side tag handling is out of scope for this PR. Dashboards listing
page and dashboard settings continue to use the tag references and do
not use the `tags` attribute in the response.
For example:
Here's how we currently create a dashboard with tag references.
```
## Creating a dashboard with tag references
POST kbn:/api/dashboards/dashboard
{
"attributes": {
"title": "my tagged dashboard"
},
"references": [
{
"type": "tag",
"id": "37aab5de-a34d-47cb-9aa5-9375d5db595f",
"name": "tag-ref-37aab5de-a34d-47cb-9aa5-9375d5db595f"
},
{
"type": "tag",
"id": "5ed29bba-c14d-4302-9a8c-9602e40dbc2a",
"name": "tag-ref-5ed29bba-c14d-4302-9a8c-9602e40dbc2a"
},
{
"type": "tag",
"id": "fc7890e8-c00f-44a1-88a2-250e4d27e61d",
"name": "tag-ref-fc7890e8-c00f-44a1-88a2-250e4d27e61d"
}
]
}
```
With this PR, creating a dashboard with tags is much simpler.
```
## Creating a dashboard with tag names
POST kbn:/api/dashboards/dashboard
{
"attributes": {
"title": "my tagged dashboard",
"tags": [
"boo",
"foo",
"bingo",
"bongo"
]
}
}
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] If there are more than one tag saved objects with the same name,
only one of the tag references will be added to the saved object when
creating a dashboard. Creating tags with duplicate names are not
permitted via the UI. But there is no such restrictions when creating
tags from imported saved objects. Having multiple tags with the same
name is an edge case that Kibana guards against with reasonable
restrictions, so I think we should not be too concerned about it.
- [ ] ...
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [@redocly/cli](https://redirect.github.com/Redocly/redocly-cli) |
devDependencies | patch | [`^1.33.1` ->
`^1.34.0`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.34.0/1.34.0)
|
| [@redocly/cli](https://redirect.github.com/Redocly/redocly-cli) |
dependencies | patch | [`^1.33.1` ->
`^1.34.0`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.34.0/1.34.0)
|
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDpwcmV2LW1pbm9yIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [@redocly/cli](https://redirect.github.com/Redocly/redocly-cli) |
devDependencies | patch | [`^1.33.0` ->
`^1.33.1`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.33.0/1.33.1)
| `1.34.0` |
| [@redocly/cli](https://redirect.github.com/Redocly/redocly-cli) |
dependencies | patch | [`^1.33.0` ->
`^1.33.1`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.33.1/1.33.1)
| `1.34.0` |
---
### Release Notes
<details>
<summary>Redocly/redocly-cli (@​redocly/cli)</summary>
###
[`v1.33.1`](https://redirect.github.com/Redocly/redocly-cli/releases/tag/%40redocly/cli%401.33.1)
[Compare
Source](https://redirect.github.com/Redocly/redocly-cli/compare/@redocly/cli@1.33.0...@redocly/cli@1.33.1)
##### Patch Changes
- Fixed `generate-arazzo` command to properly handle output file paths.
The `output-file` parameter must have a value when provided.
- Updated
[@​redocly/respect-core](https://redirect.github.com/redocly/respect-core)
to v1.33.1.
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDpwcmV2LW1pbm9yIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
---------
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Part of https://github.com/elastic/streams-program/issues/38
This work adds the `date` processor along with the dissect and grok ones
in the enrichment section.
It scales well following the current folder structure, but we should
definitely polish it a bit more once more processors get added, such as
getting the right form component, improve form state derivation, etc.
https://github.com/user-attachments/assets/824d15c8-ce9d-455a-ae0b-97aeec8cf025
---------
Co-authored-by: Kerry Gallagher <kerry.gallagher@elastic.co>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Kerry Gallagher <471693+Kerry350@users.noreply.github.com>
# Purpose
This change introduces new validations that ensure no loss of data is
possible if a user accidentally sets the Security Entity Store enrich
policy execution interval to a value that “doesn’t play nicely” with the
lookback period value.
The specific logic (greater than or equal to half the value) was chosen
to not only ensure no loss of data, but also provide extra resiliency in
case of a failed enrich policy execution.
(Note that this is not considered a breaking change, as the parameters
are not yet available on any version of Elastic, including Serverless.)
# How to test
1. Load appropriate entity log data to your Kibana instance (for
example, using the
[security-documents-generator](https://github.com/elastic/security-documents-generator))
2. Navigate to the Developer console
3. Attempt to enable the Entity Store via the /enable or /init routes
(examples below), and pass in values that are expected to error. For
example, “lookbackPeriod”: “24h” and “enrichPolicyExecutionInterval”:
“24h” should fail, because of the validation logic
4. Expect results similar to those shown below, specifically a 400
error, or else a success message
<img width="1902" alt="Screenshot 2025-02-27 at 12 57 45 AM"
src="https://github.com/user-attachments/assets/a7f4b0fb-9899-4e00-a0ae-d172245bd506"
/>
<img width="1909" alt="Screenshot 2025-02-27 at 12 58 06 AM"
src="https://github.com/user-attachments/assets/372acde2-9d7b-4c75-8596-af8374088f79"
/>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Creates basic routes to download and upload content packs associated to
a stream.
Only dashboard assets will be exported and linked to a stream.
The endpoints are currently a proxy to the savedObjects
importer/exporter interfaces:
- download exports the dashboard linked to a stream
- upload imports a content pack file and link the dashboards to the
targeted stream. Dashboards are imported as-is with no index pattern
replacement performed, this will be implemented separately
### Testing
- download `curl -XPOST -H "x-elastic-internal-origin: 'kibana'" -H
"kbn-xsrf: true"
http://elastic:changeme@localhost:5601/pat/api/streams/logs/content/export
--output content.json`
- upload `curl -XPOST -H "kbn-xsrf: true"
http://elastic:changeme@localhost:5601/pat/api/streams/logs.foo/content/import
-F 'content=@content.json'`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**Partially addresses:** #211808,
https://github.com/elastic/security-docs/issues/5981 (internal)
**Resolves: #208329**
## Summary
This is the second part of the migration effort, containing changes for:
- BULK CRUD (removing, for v.9.0)
The PR also contains changes for ticket #208329 - as changes for
removing of dead code for handling Bulk CRUD endpoints had to be
combined together with removing the schema files for Bulk CRUD
endpoints.
This PR will be backported only to versions for Kibana v9
# Testing
1. cd x-pack/solutions/security/plugins/security_solution
2. yarn openapi:bundle:detections
3. Take the bundled file
(docs/openapi/ess/security_solution_detections_api_2023_10_31.bundled.schema.yaml)
and load it into bump.sh console to see the changes.
4. Compare the changes with the [Legacy
documentation](https://www.elastic.co/guide/en/security/current/rule-api-overview.html)
You can also use this [link](https://bump.sh/jkelas2/doc/kibana_wip2/)
where I deployed the generated bundled doc.
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Resolves first part of https://github.com/elastic/kibana/issues/198783
(snooze API)
Creates a public API for adding snooze schedule to rules.
For this purpose we created new schedule schema which will be used as
standard schedule schema across rules and alerting framework.
**Note**
The code to be reviewed for public API is under `common/routes/schedule`
and inside `external` folders.
Rest of the code is just moving existing internal API route and its code
to `/internal` folders.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### How to test
- Create a rule in kibana
- Snooze that rule via new public API
```typescript
Path: https://localhost:5601/api/alerting/rule/<rule_id>/snooze_schedule
Method: POST
Body:
{
"schedule": {
"custom": {
"start": "2025-02-25T18:00:00.000Z",
"duration": "15m",
"recurring": {
"every": "1w",
"onWeekDay": ["TU", "FR"],
"occurrences": 10
}
}
}
}
```
- Verify various snooze schedule scenarios are generated correctly
#### Flaky test runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8009
### Release note
Allow users to create a snooze schedule for rule via API
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: lcawl <lcawley@elastic.co>
Add streams API to documentation as an experimental feature
<img width="2555" alt="Screenshot 2025-03-07 at 11 44 54"
src="https://github.com/user-attachments/assets/f54e5e6e-0c20-4bad-9cff-27747d0f76e2"
/>
There are a couple of changes in here:
* Split streams API in internal and public and mark the public parts as
experimental
* Add the public parts to the Kibana documentation
* Add description and summary
* Adjust the server repository wrapper to pass through summary and
description
# To test
* Generate OAS bundle: `node scripts/capture_oas_snapshot --include-path
/api/streams --update`
* Apply overlays `cd oas_docs && make api-docs`
* Make sure bump.sh is installed (`npm install -g bump-cli`)
* Run for preview: `cd oas_docs && bump preview output/kibana.yaml`
# Open questions
* Does the split into public and internal make sense?
* Is it a problem if this is visible in the user-facing documentation
page before we actually release streams? Or would it be OK if the API is
marked as experimental? (mostly a question for @LucaWintergerst )
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
[META
issue](https://github.com/elastic/security-docs-internal/issues/57)
This PR improves the API documentation for timeline/notes/pinned events
as per the definition in the meta issue.
### Notes
#### Timeline API
- `eventType`:
- is always set to `all`
- looks like it's been superseded by dataView?
- I marked it as `deperecated`
- `eventIdToNoteIds`:
- there's a type mismatch between what the frontend expects and what is
returned
- also it does not seem to be used anymore?
- it needs to be investigated further
#### Notes API
- `GlobalNote`:
- looks like it's not used anymore
- I removed it, nothing broke
- `eventIngested`, `eventTimestamp`, `eventDataView` and `overrideOwner`
are all not used on the patch note endpoint, I removed them. The
`event*` ones I have never seen before to be honest.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| [@redocly/cli](https://redirect.github.com/Redocly/redocly-cli) |
devDependencies | patch | [`^1.32.2` ->
`^1.33.0`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.33.0/1.33.0)
| `1.33.1` |
| [@redocly/cli](https://redirect.github.com/Redocly/redocly-cli) |
dependencies | patch | [`^1.32.2` ->
`^1.33.0`](https://renovatebot.com/diffs/npm/@redocly%2fcli/1.33.0/1.33.0)
| `1.33.1` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJUZWFtOkNvcmUiLCJiYWNrcG9ydDpwcmV2LW1pbm9yIiwicmVsZWFzZV9ub3RlOnNraXAiXX0=-->
Co-authored-by: elastic-renovate-prod[bot] <174716857+elastic-renovate-prod[bot]@users.noreply.github.com>
