kibana/packages/kbn-securitysolution-list-constants
Vitalii Dmyterko 505d8265c8
[Security Solution][Detection Engine] move lists to data stream (#162508)
## Summary

- addresses https://github.com/elastic/security-team/issues/7198
- moves list/items indices to data stream
  - adds `@timestamp` mapping to indices mappings
- migrate to data stream if indices already exist(for customers < 8.11)
or create data stream(for customers 8.11+ or serverless)
- adds
[DLM](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/data-streams-put-lifecycle.html)
to index templates
- replaces update/delete queries with update_by_query/delete_by_query
which supported in data streams
  - fixes existing issues with update/patch APIs for lists/items
    - update/patch for lists didn't save `version` parameter in ES
- update and patch APIs for lists/items were identical, i.e. for both
routes was called the same `update` method w/o any changes

<details>

<summary>Technical detail on moving API to
(update/delete)_by_query</summary>


`update_by_query`, `delete_by_query` do not support refresh=wait_for,
[only false/true
values](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/docs-update-by-query.html#_refreshing_shards_2).
Which might break some of the use cases on UI(when list is removed, we
refetch all lists. Deleted list will be returned for some time. [Default
refresh time is
1s](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/docs-refresh.html)).
So, we retry refetching deleted/updated document before finishing
request, to return reindexed document

`update_by_query` does not support OCC [as update
API](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/optimistic-concurrency-control.html).
Which is supported in both
[list](https://www.elastic.co/guide/en/security/current/lists-api-update-container.html)/[list
item
](https://www.elastic.co/guide/en/security/current/lists-api-update-item.html)updates
through _version parameter.
_version is base64 encoded "_seq_no", "_primary_term" props used for OCC

So, to keep it without breaking changes: implemented check for version
conflict within update method
</details>

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-23 19:42:57 +01:00
..
index.ts [Security Solution][Detection Engine] move lists to data stream (#162508) 2023-08-23 19:42:57 +01:00
kibana.jsonc [Security Solution] Clean up CODEOWNERS and other files for the Detection Engine team (#159729) 2023-06-14 10:39:50 -07:00
package.json Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
README.md [Security Solution] Utilizes constants package and deletes duplicate code (#100513) 2021-05-24 18:38:14 -06:00
tsconfig.json Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00

kbn-securitysolution-list-constants

This is where shared constants for security solution lists should go that are going to be shared among plugins. This was originally created to remove the dependencies between security_solution and other projects.