mirror of
https://github.com/elastic/kibana.git
synced 2025-04-18 23:21:39 -04:00
b1ffcf3060
## Summary Introduce a new Entity Engine Definition called Generic. The larger context on why we are introducing a new entity definition is described on this [private github issue](https://github.com/elastic/security-team/issues/11857). The tldr; is that we would like to have an entity store with all the entities described by the [entity ecs field](https://github.com/elastic/security-team/issues/11857). The decision to call `generic` entity definition comes from the fact that any entity can be described with the `entity` field - user, host, service, database, queue, subscription and so on. Therefore it makes sense to have the concept called `generic` entity, meanwhile the existent entity definitions will be called concrete entities, because they describe a very concrete type of entity (currently user, host, service). Other changes included on this PR: - Don't override `entity.name` with `entity.id`, only set if no value is found - Migrate the usage of `entity.type` as the entity definition type to `entity.EngineMetadata.Type` - Changes touching Entity Analytics code around `getRiskEngineEntityTypes` and `getAssetCriticalityEntityTypes`. There was a somewhat unnecessary and duplicated logic in these functions which essentially described the concrete entity definitions to be used by entity analytics flows. A new function called `getEntityAnalyticsEntityTypes` was introduced which unifies this logic and returns the entity types that Entity Analytics care about. Video of a scroll through the entities processed by the generic entity store, source of the data is cloudbeat asset management integration. https://github.com/user-attachments/assets/450afd05-dee0-4449-aaec-2cd69645d6ec #### How to test: - In Advanced Settings (`/app/management/kibana/settings`), enable `securitySolution:enableAssetInventory` <img width="883" alt="image" src="https://github.com/user-attachments/assets/c342abb2-efb3-40a8-b945-d9558f085f34" /> - In Entity Store management (`/security/entity_analytics_entity_store`) enable entity store <img width="1251" alt="image" src="https://github.com/user-attachments/assets/41f709e1-0aea-47dc-9c98-ffaebf18fdb1" /> - Verify Generic Engine Status <img width="1203" alt="image" src="https://github.com/user-attachments/assets/d26b764a-4695-436e-85f7-e3ed7df5a3be" /> - Ingest documents with `entity.id` and `entity.*` fields. Personally I run `cloudbeat` asset discovery locally - Verify ingested documents in `.entities.v1.latest.security_generic_default` <img width="1496" alt="image" src="https://github.com/user-attachments/assets/88286cb9-38c1-4f9d-83a7-57ba33811c60" /> -- **OBS: Also test enabling the store without the uiSetting enabled, so you can make sure that it doesn't enable** ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [x] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [x] I see risk on performance, given the amount of aggregations the generated transform does - tested, although we see a higher spike in CPU than before, it's behind a feature flag and it's going to be used in controlled data sets (entity centric logs that contain `entity.id` field) we decided it's good enough to go. - [ ] Enablement/disablement of entity store in a different uiSetting configuration. - [ ] Enable entity store with `securitySolution:enableAssetInventory` disabled. Then enable `securitySolution:enableAssetInventory` ==> No generic entity definition installed. You can manually install it in the EntityStore status page - [ ] Enable entity store with `securitySolution:enableAssetInventory` enabled. Then disable `securitySolution:enableAssetInventory` definition ==> hanging assets of generic entity store that can be deleted manually --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| kibana.serverless.yaml | ||
| kibana.yaml | ||