## Summary
BUG: https://github.com/elastic/security-team/issues/11783
This PR fixes the behaviour of the
`/api/security_ai_assistant/chat/complete` route where the
`conversationId` can be passed as an empty string. This may lead to
unexpected results described in
https://github.com/elastic/security-team/issues/11783#issuecomment-2696529040.
### Expected behaviour
We should throw a bad request (400) http error when empty
`conversationId` has been passed.
### Testing
* Use this `curl` command to test the endpoint.
```
curl --location 'http://localhost:5601/api/security_ai_assistant/chat/complete' \
--header 'kbn-xsrf: true' \
--header 'Content-Type: application/json' \
--data '{
"connectorId": "{{my-gpt4o-ai}}",
"conversationId": "",
"isStream": false,
"messages": [
{
"content": "Follow up",
"role": "user"
}
],
"persist": true
}'
```
You should see next error as a response:
```
{
"statusCode": 400,
"error": "Bad Request",
"message": "[request body]: conversationId: String must contain at least 1 character(s), conversationId: No empty strings allowed"
}
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes logic on fresh cluster where the ELSER was not started yet, in
this case API reports `status` as `uninstalled`, but it doesn't mean
that the Product documentation was actually uninstall, but rather it's a
default state.
Added internal `product_documentation_status` to KB status API to make
sure we keep track of the status internally and present the banner only
if the docs were intentionally uninstalled
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## For reviewers:
Only `*.schema.yml` files were edited (excluding
`*.bundled.schema.yml`). Rest of the changes comes from auto generation
and can be ignored.
## Description
Part of DW team effort - elastic/security-team#11804
This PR aligns the property/schema descriptions and examples in
AsciiDocs with OpenAPI schemas. The primary goal of this PR was not to
extend or enhance the documentation but to migrate from one system to
another.
Ascii docs -
https://www.elastic.co/guide/en/security/8.17/management-api-overview.html
OpenApi generated docs -
https://www.elastic.co/docs/api/doc/kibana/operation/operation-endpointgetactionslist
Changes:
Copied missing property descriptions from AsciiDoc to OpenApi properties
Copied existing AsciiDoc examples for both requests and responses
Fixed falsy query object in some GET requests - in OpenApi it was
defined as an object, not as path query params.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com>
Part of DW team effort -
https://github.com/elastic/security-team/issues/11804
This PR aligns the property/schema descriptions and examples in
AsciiDocs with OpenAPI schemas. The primary goal of this PR was not to
extend or enhance the documentation but to migrate from one system to
another.
Ascii docs -
https://www.elastic.co/guide/en/kibana/8.17/osquery-manager-api.html
OpenApi generated docs -
https://www.elastic.co/docs/api/doc/kibana/operation/operation-osqueryfindlivequeries
Changes:
1. Copied missing property descriptions from AsciiDoc to OpenApi
properties
2. Copied existing AsciiDoc examples for both requests and responses
3. Fixed falsy query object in some GET requests - in OpenApi it was
defined as an object, not as path query params.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com>
Part of DW team effort -
https://github.com/elastic/security-team/issues/11804
This PR aligns the property/schema descriptions and examples in
AsciiDocs with OpenAPI schemas. The primary goal of this PR was not to
extend or enhance the documentation but to migrate from one system to
another.
Ascii docs -
https://www.elastic.co/guide/en/kibana/8.17/osquery-manager-api.html
OpenApi generated docs -
https://www.elastic.co/docs/api/doc/kibana/operation/operation-endpointgetactionslist
Changes:
Copied missing property descriptions from AsciiDoc to OpenApi properties
Copied existing AsciiDoc examples for both requests and responses
Fixed falsy query object in some GET requests - in OpenApi it was
defined as an object, not as path query params.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com>
## Summary
It closes#210712
This PR adds the following features:
- Introduces a new `status` API for the asset inventory, which returns
the status of the entity store, including statuses like
`insufficient_privileges`, `disabled`, `initializing`, `ready`, and
`empty`.
- Implements a mechanism to check the current status of the host entity
store and processes the transform component metadata.
It also introduces the following changes:
- Implemented the use of the host entity store to support the new status
API while awaiting the readiness of the 'generic' entity store.
- Added a `metadata` field to the entity store stats object and used it
to add additional metadata from the transform components.
- Replaced the retrieval of the entity store ID with the inclusion of
`stats.id` from the transform stats.
- Unit tests were added to ensure that the new API and status behavior
are functioning correctly, covering all possible states and edge cases.
### Key Changes:
- **API Changes**: A new `status` endpoint was added to query the asset
inventory's status, leveraging the host entity store for now.
- **Metadata Changes**: The `metadata` field was added to the entity
store stats to include transform-related metadata, such as
`documents_processed` and `trigger_count`.
- **Testing**: New unit tests were added to validate the behaviour of
the status API, including scenarios like insufficient privileges, entity
store installation, and document processing status.
### How to test it locally
To test it locally, execute the following in the Kibana dev tools:
```
GET kbn:/api/asset_inventory/status
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/211666
Allow any type in `PackageInfoSchema` and `KibanaAssetReferenceSchema`
to allow new type of epm packages without change in kibana.
Covered with unit test.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This pull request introduces the new `unassigned` criticality level for
the asset criticality's `bulk_upload` .
### Key Changes:
#### Schema and Configuration Updates:
* Added `unassigned` to the list of criticality levels in multiple
schema files (`kibana.serverless.yaml`, `kibana.yaml`,
`common.schema.yaml`,
`ess/security_solution_entity_analytics_api_2023_10_31.bundled.schema.yaml`,
`serverless/security_solution_entity_analytics_api_2023_10_31.bundled.schema.yaml`).
#### TypeScript and Constants:
* Updated `AssetCriticalityLevel` enum in `common.gen.ts` to include
`unassigned`.
* Added `UNASSIGNED` to `CriticalityLevels` enum and
`CriticalityModifiers` in `constants.ts`.
#### Tests:
* Updated test cases to include `unassigned` as a valid criticality
level in `parse_asset_criticality_csv_row.test.ts`,
`validations.test.ts`, `asset_criticality_data_client.test.ts`, and
`asset_criticality_csv_upload.ts`.
#### Backend Logic:
* Modified `AssetCriticalityDataClient` to handle `unassigned`
criticality level appropriately.
<img width="1488" alt="Screenshot 2025-01-30 at 2 03 11 PM"
src="https://github.com/user-attachments/assets/938411c4-725a-451c-ab38-aca36a704e91"
/>
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Testing Steps
The main idea for this change is that the `unassigned` criticality level
is actually marked as `deleted` in the ES documents.
ES index name for default space :
`.asset-criticality.asset-criticality-default`
#### API
1. `POST /api/asset_criticality/upload_csv`
```
curl --location 'http://localhost:5601/api/asset_criticality/upload_csv?output=stream' \
--header 'kbn-xsrf: hello' \
--header 'Accept: multipart/form-data' \
--header 'Authorization: *******' \
--form 'file=@"<Full path of the CSV file for asset criticality>"
```
Error response :
```
{"errors":[{"message":"Invalid criticality level \"unassigned_impact\", expected one of extreme_impact, high_impact, medium_impact, low_impact, unassigned","index":4}],"stats":{"successful":3,"failed":1,"total":4}}%
```
Success response :
```
{"errors":[],"stats":{"successful":4,"failed":0,"total":4}}%
```
Query the ES using below query to see if the criticality level is
`deleted`
```
GET .asset-criticality.asset-criticality-default/_search
{
"query": {
"match": {
"asset.criticality": "deleted"
}
}
}
```
### UI
1. Navigate to Entity Store page
2. Upload a csv file with incorrect asset criticality level
3. Rectify and upload the same file with correct criticality levels.
4. Navigate to EA Dashboard and scroll down to the Entities section.
5. Select an entity and open the flyout.
6. Try changing the asset criticality of the entity. No blank/empty
value should be present or assigned to asset criticality.
7. Should be able to successfully modify the asset criticality for the
entity.
Confirm this by querying the ES with the query :
```
GET .asset-criticality.asset-criticality-default/_search
{
"query": {
"match": {
"asset.criticality": "deleted"
}
}
}
```
Ensure that the Elastic search document's `_source` contains three keys,
with `deleted` present as the ` `criticality_level` for the below three
keys.
`criticality_level`
`host.asset.criticality`
`asset.criticality`
Example :
### Bulk upload
```
curl --location 'http://localhost:5601/api/asset_criticality/bulk' \
--header 'kbn-xsrf: hello' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic ****' \
--data '{
"records": [
{
"id_value": "host-1",
"id_field": "host.name",
"criticality_level": "low_impact"
},
{
"id_value": "host-2",
"id_field": "host.name",
"criticality_level": "medium_impact"
},
{
"id_value": "host-6",
"id_field": "host.name",
"criticality_level": "medium_impact"
},
{
"id_value": "host-3",
"id_field": "host.name",
"criticality_level": "high_impact"
},
{
"id_value": "host-4",
"id_field": "host.name",
"criticality_level": "high_impact"
},
{
"id_value": "host-bulkupload",
"id_field": "host.name",
"criticality_level": "unassigned"
}
]
}
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes a small UI bug in the citations feature. Previously, after a
message with citations finished streaming, the citations would disappear
for a fraction of a second and then reappear again. This PR makes
improves the UI by making the citations not flash off and on after the
stream finishes.
### Changes:
- Fix flashing citations
- Refactor code related to parsing content references (to make it more
maintainable).
- Update the citations prompt slightly.
### Before:
https://github.com/user-attachments/assets/1021dd53-018a-43ba-b1f4-24aab44faca9
<img width="1782" alt="image"
src="https://github.com/user-attachments/assets/723cd29a-48a2-48e7-b031-0893484746b9"
/>
### After:
https://github.com/user-attachments/assets/21f340bc-9015-42b6-a574-0439d2f8f192
### How to test
- Enable the feature flag
```yaml
# kibana.dev.yml
xpack.securitySolution.enableExperimental: ['contentReferencesEnabled']
```
- Open the security assistant
- Ask it a question about your alerts of a document in your KB. The
response should contain citations.
- Observe the response stream carefully. Ensure the citations e.g. `[1]`
do not flash off and on when the response stream finishes. The expected
behavior is that while the message is streaming, the citations are
disabled and once the stream finishes the citations get enabled (while
always being visible).
#### Edge case to test
It is possible that citations completely disappear after streaming
finishes. This happens when the LLM produces an invalid citation.
Invalid citations are hidden client side when a message finishes
streaming. You can verify this behavior by asking GPT4o this question:
```
Prepend each line with this placeholder citation "{reference(1234)}" and append the actual citation at the end of the line. How many alerts do I have? Use the open and acknowledged alerts count tool to answer and repeat the answer 50 times on new lines.
```
While the response is getting streamed it should look like this:
<img width="200" alt="image"
src="https://github.com/user-attachments/assets/03d160bf-2404-4a4e-8701-e3183c604cc4"
/>
And when the stream finishes it should look like this:
<img width="200" alt="image"
src="https://github.com/user-attachments/assets/06367379-17da-438f-a93a-9d539067ab90"
/>
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [X] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [X]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [X] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [X] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [X] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [X] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
# Summary
As part of the effort to add missing content for Security APIs, this PR
introduces a few missing request, response, and parameter examples for
Detection Engine Alert and migration APIs.
## Summary
We anticipate generating a single document per API version for the
foreseeable future. This PR updates our OAS document merging logic to
avoid adding the `Elastic-Api-Version` header to our spec.
This will also remove this header from current documentation.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR introduces support for configuring the `timestamp` field for
entity store enablement.
By default, the `timestamp` field is set to `@timestamp`, but users can
opt to use `event.ingested` or another preferred value based on their
requirements.
### Entity Store API changes
#### Entity Store enable
```
POST kbn:/api/entity_store/enable
{
"timestampField": "event.ingested"
}
```
#### Result
```
{
"engines": [
{
"status": "started",
"type": "user",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
},
{
"status": "started",
"type": "host",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
},
{
"status": "started",
"type": "universal",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
}
],
"status": "running"
}
```
```
POST kbn:/api/entity_store/enable
{
}
```
#### Result
```
{
"engines": [
{
"status": "started",
"type": "host",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
},
{
"status": "started",
"type": "user",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
},
{
"status": "started",
"type": "universal",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
}
],
"status": "running"
}
```
Different entity types can have distinct `timestampField` values, as
described below:
For instance, the `host` entity uses `event.ingested` as its
`timestampField`, while other entities default to the `@timestamp`
field.
```
POST kbn:/api/entity_store/engines/host/init
{
"timestampField": "event.ingested"
}
POST kbn:/api/entity_store/engines/user/init
{
}
```
#### Result
```
{
"engines": [
{
"status": "started",
"type": "universal",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
},
{
"status": "started",
"type": "host",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "event.ingested"
},
{
"status": "started",
"type": "user",
"indexPattern": "",
"filter": "",
"fieldHistoryLength": 10,
"timestampField": "@timestamp"
}
],
"status": "running"
}
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Testing steps
- Check out this PR branch.
- Start Kibana.
- Avoid enabling the Entity Store from the UI initially. Instead, enable
it via the API by following the steps outlined below.
```
DELETE kbn:/api/entity_store/engines/user
DELETE kbn:/api/entity_store/engines/host
GET kbn:/api/entity_store/status
All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
"timestampField": "event.ingested"
}
# All engines with the same timestamp Field
POST kbn:/api/entity_store/enable
{
}
# Different entity with different timestamp Field
POST kbn:/api/entity_store/engines/host/init
{
"timestampField": "event.ingested"
}
POST kbn:/api/entity_store/engines/user/init
{
}
```
4. Afterward, you can test the functionality through the UI. First,
clear the entity data from the UI, then start the Entity Store. Note
that the UI currently does not offer a way to configure this parameter.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Mark Hopkin <mark.hopkin@elastic.co>
## Summary
This PR adds the following parameters to the `INIT` engine API:
* `frequency`: the transform run frequency
* `timeout`: the timeout for the initial creation of the transform
* `docsPerSecond`: transform throttling option. See
[here](https://arc.net/l/quote/vxcmfnhh)
* `delay`: The transform delay duration. See
[here](https://arc.net/l/quote/mzvaexhv)
Coming soon
In addition, the PR adds these fields to the Saved Object with the
engine descriptor, as well as providing a migration with the appropriate
backfilling.
Finally, there are some utility function that were/are helpful in
working with objects.
## How to test
*NOTE*: Always make sure the security default data view exists. Easiest
way it to just navigate to some Security UI.
### Checking the new defaults
1. Initialize an engine via dev tools by calling: `POST
kbn:/api/entity_store/engines/<entity_type>/init {}`
2. Call `GET kbn:/api/entity_store/status`. This response should now
contain all the default optional values.
### Observing the parameters are being applied
1. Initialize an engine via the API. This time pass any of the `timeout,
frequency, delay and docsPerSecond` options in the request body.
2. Once the `status` changes to `started`, query the respective
transform: `GET
_transform/entities-v1-latest-security_<entity_type>_default`
3. Check that the parameters have been applied to the transform
### Checking Saved Object Migration
1. Check out `main`.
2. Initialize the store.
3. Query `GET kbn:/api/entity_store/status`. Note down the fields in the
engine object.
4. Check out this branch.
5. Restart kibana.
6. Query `GET kbn:/api/entity_store/status` again. Observe the new
fields have been added and backfilled
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**Resolves:** https://github.com/elastic/security-team/issues/10410
## Summary
We want to make Rule Customization available at higher license tiers.
### **Intended Workflows/UX**
#### **Basic/Platinum/Security Essentials License Tiers**
- **Editing Prebuilt Rules:**
- Allow the 8.16 behavior: only actions, exceptions, snoozing, and
enable/disable options can be modified.
- On the rule editing page, all tabs except *Actions* are disabled.
Disabled tabs will display a hover explanation:
- "Upgrade to Enterprise to enable prebuilt rule customization" for ECH.
- "Upgrade to Security Complete to enable prebuilt rule customization"
for Serverless.
<img width="356" alt="image"
src="https://github.com/user-attachments/assets/72e60933-aaaf-45a0-9660-4cd066d3afec"
/>
- Rule editing via API is not restricted (tracked separately:
https://github.com/elastic/security-team/issues/11504.
- **Bulk Actions:**
- Modifications to rule content via bulk actions are not allowed.
Prebuilt rules are excluded from bulk actions if the license level is
insufficient. Users will see an explanation for the exclusion.
- Serverless
<img width="737" alt="image"
src="https://github.com/user-attachments/assets/99fef72f-dd38-4c73-a9e3-7b4c8018b4ed"
/>
- ECH
- On the API level (`_bulk_action`), an error is returned if a user
tries to modify a prebuilt rule without the required license. Response
in this case looks like this:
```json
{
"statusCode": 500,
"error": "Internal Server Error",
"message": "Bulk edit failed",
"attributes": {
"errors": [
{
"message": "Elastic rule can't be edited",
"status_code": 500,
"rules": []
}
]
}
}
```
- **Rule Updates:**
- Updates are restricted to Elastic’s incoming updates only.
- The rule upgrade flyout is in read-only mode.
<img width="949" alt="image"
src="https://github.com/user-attachments/assets/16a56430-63e6-4096-8ffd-b97f828abdd4"
/>
- For previously customized rules where customization is now disabled
due to insufficient licensing, a notification will appear on the upgrade
flyout, clarifying that only an upgrade to Elastic's version is
available.
- On the API level (`_perform`), only requests with `pick_version =
target` are permitted. Requests with `rule.fields` values are not
allowed.
API response when `pick_version` is not `target`:
```json
{
"message": "Only the 'TARGET' version can be selected for a rule update;
received: 'CURRENT'",
"status_code": 400
}
```
API response when the `fields` value is provided:
```json
{
"message": "Rule field customization is not allowed. Received fields:
name, description",
"status_code": 400
}
```
- **Customized Rules:**
- Existing customizations remain intact, and the “Modified” badge is
retained
- On the rule management, monitoring, and update tables:
- On the rule update flyout:
- On the rule details page:
- When we edit a rule with customizations (e.g., change rule's actions),
the rule should stay marked as customized
- **Import/Export Scenarios:**
- These are handled separately
(https://github.com/elastic/security-team/issues/11502)
#### **Enterprise/Security Complete License Tiers**
- All rules can be fully edited
- Upgraded prebuilt or customized rules will have an editable view,
enabling full customization
Closes https://github.com/elastic/kibana/issues/186266
## Release notes
Enhanced Role management to manage larger number of roles by adding
server side filtering, pagination and querying.
## Summary
- Replaced the usage of Get Roles API with Query Role API
- Added server side pagination and filtering with a maximum limit of
10000 keys (default for max results on index). Added new label to
indicate that we show only 10k results.
- Search box replicates client side implementation by only filtering on
Role names.
### Run locally
Start ES ~with the JVM option to enable this feature~:
```
yarn es snapshot --license=trial
```
Start Kibana normally
```
yarn start --no-base-path
```
Navigate to Stack Management > Roles and verify the same behavior as the
screen recording below
### Screen recording
https://github.com/user-attachments/assets/a447e7df-8aa1-4044-a6b2-0aafe56844a9
## Technical notes
- Client side EuiInMemory table has been replaced by EuiSearchBar,
EuiBasicTable and Filters
- One new Kibana endpoint added
- `roles/_query`
- Replicates existing get_role endpoint by being public and added to
Open API spec
- Extra logic to handle previously UI only filter to show/hide reserved
roles
- Parse the query to construct the correct DSL if the filter is present
- Update Get All Roles by Space internal API to use the Query Role and
filter by space id using query DSL.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
> [!note]
> Planning to merge before the 9.0 feature freeze.
Documentation update issue:
https://github.com/elastic/security-docs/issues/6473
> [!tip]
> ### Tip for the reviewer
> As a starting point to review this PR I would suggest reading the
section "How does it work (on a high level)" and viewing the hyperlinked
code. The linked code covers the main concepts of this feature and the
majority of the remaining changes in the PR are related to API schema
updates and tests.
## Summary
This PR adds citations to the security AI assistant. Citations are
produced when tools are used and they are displayed in the LLM response
as numbered superscript elements. A label appears when the user hovers
over the numbered elements and clicking on the label opens a new tab
that displays the cited data.
## How to test:
1. Enables the feature flags:
- Set value to `true`
[here](https://github.com/elastic/kibana/pull/206683/files#diff-f55be7c50853801c3933b48064ab9cbf0356e2941cd97c4c365be7a6ded9bffdR125)
2. Populate the security knowledge base with some information (e.g. a
document, an index, product documentation, the global threat report,
etc...).
3. Open the security assistant
4. Ask it the following questions about data in the knowledge base:
- What is an elastic search cold tier? - Make sure product docs are
installed
- What topics are covered in the security lab content?
- Ask it a question about one of your knowledge base documents.
- Which platform did my most recent alert happen on? - Make sure you
have a recent alert
## How does it work (on a high level)?
Citations are stored inside the
[ContentReferencesStore](https://github.com/elastic/kibana/pull/206683/files#diff-baf03ce192db4f13999748a38b4d920428358a4ffc62527a1d6ac0d9b234f306R17)
object. When tools are called, the tools [add citations to the
ContentReferencesStore](https://github.com/elastic/kibana/pull/206683/files#diff-5a333fdd9bf864dced06500263577e495c95c9b32c7dae9074090775df542d22R97-R99)
and pass the Id of the [ContentReferences back to the
LLM](https://github.com/elastic/kibana/pull/206683/files#diff-5a333fdd9bf864dced06500263577e495c95c9b32c7dae9074090775df542d22R102)
along side the result of the tool. The LLM can then use those
contentReference IDs in its response by forming a response like:
```
The sky is blue {reference(12345)}
```
The web client [parses out the
contentReference](https://github.com/elastic/kibana/pull/206683/files#diff-3a5c8305ac899a9e78903b0b60141dd997ba61e87342de2b9ec377165d99cfe6R23)
(`{reference(12345)}`) from the assistant message and[ replaces it with
the citation react
component](https://github.com/elastic/kibana/pull/206683/files#diff-db928fb87a862e3ebf7247baefc418de539f9c0f3fc5134a2ef56f921a52bdcbR125-R129).
### Tools that are cited:
Include citations for the following tools:
alert_counts_tool -> cites to alerts page
knowledge_base_retrieval_tool -> cites knowledge base management page
with specific entry pre-filtered
open_and_acknowledged_alerts_tool -> cites to specific alert
security_labs_tool -> cites knowledge base management page with specific
entry pre-filtered
knowledge_base indices -> opens ESQL view selecting the particular
document used
product_documentation -> cites documentation
### Endpoints impacted
- POST
/internal/elastic_assistant/actions/connector/{connectorId}/_execute
- POST /api/security_ai_assistant/chat/complete
- GET /api/security_ai_assistant/current_user/conversations/_find
- GET /api/security_ai_assistant/current_user/conversations/:id
- PUT /api/security_ai_assistant/current_user/conversations/{id}
### Considerations:
- One of the main objectives of this feature was to produce in-text
citations to create a great user experience. Multiple approaches were
tested to do this reliably. Attempts were made to make the LLM return
structured JSON containing the citations however this was unreliable
with smaller models. Generation post-processing (issuing an additional
LLM call to annotate the response with citations) was also explored
however this also had limitations as the second LLM call would not
contain enough contextual information to create the citations reliably.
Eventually, the approach described in the section above was used
alongside few shot promoting.
- Instead of using the ContentReferencesStore to store citations, the
langGraph state could be used to save the citations. I looked at doing
this but currently, there are a few blockers in the langgraph API the
prevent this.
- Lang graph must be updated to @langchain/langgraph>=0.2.31 to get
access to the Command type so that tools can update the graph state.
- It seems that DynamicStructuredTools do not support the Command type
yet. This is something that we can clarify with the langchain team.
Once these blockers have been addressed, ContentReferencesStore could
easily be refactored to the graph state.
- The feature has been put behind a feature flag so we can test during
the feature freeze and sync the release of the documentation update. The
only thing that is not behind a feature flag is the new anonymization
button in the settings menu (don't think it is necessary and it means a
lot more code changes are required).
On few occasions, you can nudge the LLM a bit more to include citations
by appending "Include citations" to your message.
Furthermore, the settings menu has been updated to include anonymized
values and citation toggles:
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [X] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [X] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [X] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.
- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...
## Release note
Adds in-text citations to security solution AI assistant responses.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
## Rule gaps
This PR contains schema changes, which have already been reviewed and
should be [merged](https://github.com/elastic/kibana/pull/192347) and
released as an intermediate release.
**Feature flag needed:**
`xpack.securitySolution.enableExperimental: [
'storeGapsInEventLogEnabled']`
Video demo of user use cases - is
[here](https://elastic.slack.com/archives/C071E7FCR1A/p1737044341833789)
<img width="1433" alt="Screenshot 2025-01-16 at 15 38 46"
src="https://github.com/user-attachments/assets/2696a1ee-7191-4ca1-9a79-9dff1764cf9d"
/>
<img width="1698" alt="Screenshot 2025-01-16 at 15 38 30"
src="https://github.com/user-attachments/assets/1b0ef6db-6c28-4986-a47e-f544bdd9c27c"
/>
TLDR:
- new API to find/fill gaps, get information for rules about gaps
- update gaps after the backfill run is completed (all tasks should
finished)
- event log now can update documents
- show a list of gaps in the UI
- show the amount of rules with gaps
## Event log changes
It support updating document right now.
There schema changes, as for events we return _id, _index and fields
needed for updates (seq_no, primary_term)
## How gaps update works (steps)
#### Scheduling Backfill
- Identify all gaps overlapping with the specified backfill date range.
- Update the gap state:
- Add the intervals to `in_progress` intervals.
- Log the updated gaps in the event log.
#### Completing Backfill
- At the end of a backfill run (when all tasks are completed):
- Find all gaps overlapping with the backfill date range.
- Recalculate the gap state:
- Move intervals into `filled_intervals` based on the backfill schedule.
- Update the gap information in the event log.
#### Handling Errors and Recalculating State
- If the backfill encounters errors, timeouts, or is removed, we can't
just remove intervals from in progress. We need to take into account
other overlapping backfills
- Fetch all backfills overlapping with the gaps.
- Recalculate the entire gap state to ensure consistency.
## How gaps update works (requests)
We process gaps in butch by 500 documents, and then update them in bulk.
For only conflicting documents we retry 3 times up to 3 times.
### Status changes
```mermaid
stateDiagram
[*] --> Unfilled
Unfilled --> Unfilled_with_in_progress_inervals: Backfill scheduled
Unfilled_with_in_progress_inervals --> Filled: Backfill completed
Unfilled_with_in_progress_inervals --> PartiallyFilled: Backfill completed
Filled --> [*]
```
### How to have gaps
1. Create a rule with 1 minute interval 1s lookback time, wait for at
least 1 execution
2. Disable rule
3. Wait for at least 5 minutes (5 x interval time)
4. Enable rule and observe that it's failing with gap message
# API Endpoints
## 1. Find Gaps
Retrieves a paginated list of gaps for rules with filtering options.
### Request
**POST** `/internal/alerting/rules/gaps/_find`
#### Request Body
| Field | Type | Required | Description |
|-------------|--------------------------------------------------------|----------|----------------------------------------|
| page | number | Yes | Page number for pagination |
| per_page | number | Yes | Number of items per page |
| rule_id | string | Yes | ID of the rule to find gaps for |
| start | string (ISO date) | Yes | Start of time range |
| end | string (ISO date) | Yes | End of time range |
| sort_field | string | No | Field to sort by |
| sort_order | asc \| desc | No | Sort direction |
| statuses | Array<'unfilled' \| 'filled' \| 'partially_filled'> | No |
Filter by gap statuses |
### Response
```json
{
"body": {
"page": number,
"per_page": number,
"total": number,
"data": [
{
"_id": string,
"@timestamp": string,
"status": "unfilled" | "filled" | "partially_filled",
"range": { "gte": string, "lte": string },
"unfilled_intervals": [{ "gte": string, "lte": string }],
"filled_intervals": [{ "gte": string, "lte": string }],
"in_progress_intervals": [{ "gte": string, "lte": string }],
"total_gap_duration_ms": number,
"filled_duration_ms": number,
"unfilled_duration_ms": number,
"in_progress_duration_ms": number
}
]
}
}
```
---
## 2. Fill Gap by ID
Triggers a backfill operation for a specific gap.
### Request
**POST** `/internal/alerting/rules/gaps/_fill_by_id`
#### Query Parameters
| Parameter | Type | Required | Description |
|-----------|--------|----------|---------------------------------|
| rule_id | string | Yes | ID of the rule containing the gap |
| gap_id | string | Yes | ID of the gap to fill |
### Response
Returns a standard schedule backfill API response.
---
## 3. Get Rules with Gaps
Retrieves a list of rule IDs that have gaps in the specified time range.
### Request
**POST** `/internal/alerting/rules/gaps/_get_rules`
#### Request Body
| Field | Type | Required | Description |
|-------|---------------|----------|-------------------------------|
| start | string (ISO date) | Yes | Start of time range |
| end | string (ISO date) | Yes | End of time range |
### Response
```json
{
"body": {
"total": number, // Total number of rules with gaps
"rule_ids": string[] // Array of rule IDs
}
}
```
---
## 4. Get Gaps Info by Rule IDs
Retrieves gap statistics for specified rules.
### Request
**POST** `/internal/alerting/rules/gaps/_get_gaps_info_by_rule_ids`
#### Request Body
| Field | Type | Required | Description |
|----------|---------------|----------|------------------------------------|
| rule_ids | string[] | Yes | Array of rule IDs to get gap info for |
| start | string (ISO date) | Yes | Start of time range |
| end | string (ISO date) | Yes | End of time range |
### Response
```json
{
"body": {
"data": [
{
"rule_id": string,
"total_unfilled_duration_ms": number,
"total_in_progress_duration_ms": number,
"total_filled_duration_ms": number
}
]
}
}
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
- partially addresses https://github.com/elastic/kibana/issues/202545
(except of IM rule type)
- extends logged requests preview for:
- [x] New terms
- [x] Query
- [x] ML
- [x] Threshold
- For Threshold, Query, New terms rule type introduced Page view, where
each loop of rule execution is presented as a separate page
- Only first 2 search queries requests of each type are logged for
performance reasons(rule can have very a large and multiple requests).
That's why property **request** was made not mandatory in
`rule_preview.schema.yaml`
### DEMO
https://github.com/user-attachments/assets/abfbd3ff-d06c-4892-b805-0f05084042ed
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Store gaps in event log
Setting new fields hidden behind feature flag
(`storeGapsInEventLogEnabled`). It should be possible to do intermediate
release, just to update mappings.
In this PR:
- store gaps in event log
- update event log mappings
- add gap range and unfilled_gap_ms mappings into rule `last_run`
metrics for rule schema
### How it works
In this stage we use our current mechanism for gap detection in security
rules. Instead of just report duration in ms, we will also report date
range of the gap. To set this value we use `RuleMonitoringService` from
the security rule execution.
Later in task runner we check if last run from `RuleMonitoringService`
has gap range information. If it there, we use `AlertingEventLogger` to
create Gap document in event log.
New field in event log in `kibana.alert.rule.gap` fields:
`status`: keyword. Gap status
`range`: date_range. Full gap range
`filled_intervals`: date_range. Filled intervals
`unfilled_intervals`: date_range. Unfilled intervals
`in_progress_intervals`: date_range Intervals which currently have
potential rule executions planned (from manual rule run)
`total_gap_duration_ms`: long
`filled_gap_duration_ms`: long
`unfilled_gap_duration_ms`: long
`in_progress_gap_duration_ms`: long
**kibana.alert.rule.execution**: extend execution info with gap_range
`kibana.alert.rule.execution.gap_range`
## Rule schema
Added `unfilled_gap_ms` into rule response schema. We will calculate
total gap duration after each rule execution, and write in to rule
object for 1d,3d,7d. It's allow us to use it in Rule monitoring table,
and be able sort by that field
I plan add integrations tests, in next PR's when we will have API to get
gaps info
### How this schema will be used:
More details
[here](https://docs.google.com/document/d/1pDJZon-OzUwmk1ENNZJo0QUoSD-MCwy_tn9GCCBX_1Y/edit?tab=t.0)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR enables configurability of the lookbackPeriod for Security’s
Entity Store through the enable API.
### Testing
1. Open the latest instance of Kibana.
2. Ensure both the Entity Store and Risk Score are enabled.
3. Navigate to Dev Tools.
4. Use the enable endpoint with lookbackPeriod, where the lookbackPeriod
must be a number followed by one of the supported units (m, s, or h).
```
POST kbn:api/entity_store/enable
{
"lookbackPeriod": "72h" // example value
}
```
5. After enabling, use the status endpoint to confirm the configuration:
```
GET kbn:api/entity_store/status
{}
```
6. Post some documents, to check the entities available within the lookbackPeriod - confirm you can only see those within this period, and not outside of this period e.g. "5h" period, post with a timestamp of today (within 5 hours) and from yesterday and view results.
```
POST lookback-period-test/_doc
{
"entity_id": "LookbackPeriod Test Today",
"entity_name": "LookbackPeriod Test Today",
"timestamp": "2025-01-17T14:31:00Z"
}
```
```
POST lookback-period-test/_doc
{
"entity_id": "LookbackPeriod Test Yesterday",
"entity_name": "LookbackPeriod Test Yesterday",
"timestamp": "2025-01-16T14:31:00Z"
}
```
7. Can also go to entity store, check one of the transforms and check
the JSON gte ranges from here too, they should show the new
lookbackPeriod.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Add `enrichPolicyExecutionInterval`param to entity enablement and init
APIs
### How to test?
* Start kibana
* Call the entity store enablement API with a short value for
`enrichPolicyExecutionInterval` param
* Check in the logs if the enrichment process is running frequently
* Clear the entity store
* Call the entity store enablement API without
`enrichPolicyExecutionInterval` param
* Check in the logs if the enrichment process is running less frequently
**Enable Entity store API call:**
```
POST kbn:/api/entity_store/enable {
"enrichPolicyExecutionInterval": "10s"
}
```
**Init Entity store API call:**
```
POST kbn:/api/entity_store/engines/user/init {
"enrichPolicyExecutionInterval": "10s"
}
```
**Enrich policy log message:**
```
│ info [o.e.x.e.EnrichPolicyRunner] [...] Policy [entity_store_field_retention_user_default_v1.0.0]: Running enrich policy
```
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves#202090.
Incidentally also fixes and unskips the failing test reported in
#203346.
This PR makes it so that:
- Agentless agent policies and package policies (those with
`supports_agentless: true` set) cannot be directly set to use a
non-local ES output for integration data
- This restriction applies to UI and API level
- When a non-local ES output would be updated to be the global Fleet
default integration data output, existing agentless policies without an
explicit output set will have their output directly set to the current
default output ID
- This is the same mechanism used today to ensure that Fleet Server and
Synthetics integrations do not accidentally have their output set to
non-ES as well
## Testing
1. Apply
[patch](https://gist.github.com/jen-huang/dfc3e02ceb63976ad54bd1f50c524cb4)
to skip actual agentless creation
2. Create different types of outputs in addition to default local ES
3. Enable beta integrations
4. Add an agentless integration, for example Box Connector, observe that
outputs list only shows ES outputs:
<img width="1422" alt="image"
src="https://github.com/user-attachments/assets/72e43220-702f-4bb7-8e37-8be69aa4e6ea"
/>
5. Switch to agent-based setup technology, observe that outputs list now
shows all outputs
6. Create the agentless integration, go to its agent policy
7. Observe that outputs list only enables ES outputs:
<img width="1425" alt="image"
src="https://github.com/user-attachments/assets/3bc5985f-07bf-407a-8b62-4248b28904a5"
/>
8. Play around with setting global default outputs, it should be not
possible to get into a state where an agentless policy is using a
non-local ES output
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
# Summary
As part of the effort to add missing content for Security APIs, this PR
introduces a few missing request, response, and parameter examples for
Detection Engine Value Lists APIs.
## Summary
* Display a callout to install uninstalled entity types
* It will be displayed when the entity store is running, but some
available entity types are not installed
* Add `entityTypes` param to the init entity store API
* Enable `serviceEntityStoreEnabled` flag by default
### Checklist
### How to test it?
* Disable the experimental flag on
`x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts`
* Open `Manage/Entity Store/Engine Status`
* Verify that the service entity store is NOT installed
* Reenable the flag on
`x-pack/solutions/security/plugins/security_solution/common/experimental_features.ts`
* Open the Entity Dashboard page
* It should display a callout to install more entity types
* Click "enable"
* It should install the service entity store
* Open `Manage/Entity Store/Engine Status`
* Verify that the service entity store IS installed
Reviewers should verify this PR satisfies this list as well.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
# Summary
As part of the effort to add missing content for Security APIs, this PR
introduces a few missing request, response, and parameter examples for
Detection Engine Exception APIs.
# Summary
As part of the effort to add missing content for Security APIs, this PR
introduces a few missing request, response, and parameter examples for
Security Entity Analytics APIs.
All of the affected APIs are the Asset Criticality APIs.
# How to Test
Until the docs are deployed, this can be tested by navigating to
https://editor.swagger.io/ and pasting in the fully bundled file, which
can be found
[here](f905638686/x-pack/solutions/security/plugins/security_solution/docs/openapi/ess/security_solution_entity_analytics_api_2023_10_31.bundled.schema.yaml)
in the PR
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
Closes#197731
- Using the new `audit_unenroll_reason`, added `Orphaned` and
`Uninstalled` Badges to the table UI to make those entries distinct from
the normal `offline` entries
- Added conditional rendering on the labels at the top of the table to
only show up when the matches are greater than zero
- Updated `offline` counts to be distinct from the `orphaned` and
`uninstalled` items in those labels
- Added Orphaned and Uninstalled filters to status filter dropdown
Screenshot of new:
### Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
### Identify risks
n/a
## Release Note
Improves filtering & visibility of `Uninstalled` and `Orphaned` Agents
in Fleet, by differentiating them from `Offline` agents. Status filters
have been added for both `Uninstalled` and `Orphaned` agents. Agent
`status` runtime field has been updated to return accordingly when an
agent is `uninstalled` or `orphaned`. Additionally, improved UI by
hiding agent labels with `0` agent matches.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## Summary
* Refactor types to prevent usages of `host` and `user`.
* Use `EntityType` instead.
* Use a generic function that receives `EntityType` as a parameter
instead of custom user and host functions
* Consolidate duplicated entity types
* Add service to entity types and update all references on the
EntityAnalyticsDashboards page, Risk score page and Entity Store page.
* Refactor Risk score APIs to be more generic and accept EntityType and
a param
* Refactor if statement like `isUserRiskScore` to be more generic and
accept `service`
* Delete `RiskScoreEntity` in favour of `EntityType`.
* Update the branch to support the universal entity
### Not included
* Service Flyout
### Images
### Generic Entity Support
We need to support risk score and asset criticality for
Generic/Universal entities according to
https://github.com/elastic/security-team/issues/10740
> We expect that the below will be supported:
>
> Entity flyout for service/generic entity
> Entity risk scoring for service/generic entity
> Asset criticality assignments for service/generic entity
This PR already implements that support. However, I have introduced a
function per feature that returns the enabled entity types. At the
moment, I defined universal/generic entities as unsupported for this PR
to preserve the current behaviour. But to allow universal/generic
entities, we only need to delete a couple of lines.
Risk Score will need extra work because the entity types are hard-coded
on some parts of the code.
### How to test it
1
* Start kabana with security solution data
* You can use the document generator with `yarn start entity-store`
* Enable Entity Store and Risk engine
* Test the EA features, and they should work normally
2
* Start kabana with security solution data
* You can use the document generator with `yarn start entity-store`
* Enable the `serviceEntityStoreEnabled` flag
* Enable Entity Store and Risk engine
* Test the EA features, and you should see a new type of Entity called
'service'
* Service Entity should work with all Entity analytics features
3
* Start kabana with security solution data
* You can use the document generator with `yarn start entity-store`
* Enable the `assetInventoryStoreEnabled` flag
* Enable Entity Store and Risk engine
* Test the EA features, and you should not see universal/generic entity
except for the entity store status pages
### Checklist
Reviewers should verify this PR satisfies this list as well.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Release note
Add the "service" type to Security Entity Analytics - Entity Store. It
will find services by the `service.name` field, calculate risk score,
and allow asset criticality assignment.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
- Adds response actions client/APIs for isolate and release actions for
Microsoft Defender for Endpoint
- The feature is behind a feature flag
`responseActionsMSDefenderEndpointEnabled`
