Commit graph

5 commits

Author SHA1 Message Date
Mary Gouseti
91f4023e27
Expose global retention settings via data stream lifecycle API (#112210)
In this PR we expose the global retention via the `GET
_data_stream/{target}/_lifecycle` API.

Since the global retention is a main feature of the data stream
lifecycle we chose to expose it by default.

```
GET /_data_stream/my-data-stream/_lifecycle
{
 "global_retention": {
      "default_retention": "7d",
      "max_retention": "365d"
  }, 
  "data_streams": [...]
}
```
2024-09-02 18:40:08 +10:00
Mary Gouseti
bed6e18fa3
Exclude internal data streams from global retention (#112100)
With #111972 we enable users to set up global retention for data streams that are managed by the data stream lifecycle. This will allow users of elasticsearch to have a more control over their data retention, and consequently better resource management of their clusters.

However, there is a small number of data streams that are necessary for the good operation of elasticsearch and should not follow user defined retention to avoid surprises.

For this reason, we put forth the following definition of internal data streams.

A data stream is internal if it's either a system index (system flag is true) or if its name starts with a dot.

This PR adds the `isInternalDataStream` param in the effective retention calculation making explicit that this is also used to determine the effective retention.
2024-08-28 11:28:35 +03:00
Mary Gouseti
ed60470518
Display effective retention in the relevant data stream APIs (#112019) 2024-08-22 17:42:49 +03:00
Mary Gouseti
732c7c4c30
[DSL] Remove REST APIs for global retention (#107565) 2024-04-17 21:36:26 +03:00
Mary Gouseti
2122da31cd
[DSL] Introduce data stream global retention - Part 3 (#105682)
In this PR we introduce the API that will expose the global retention configuration and will allow users to take advantage of it.

These APIs are protected by the dedicated introduced privileges:

`manage_data_stream_global_retention` or higher, which allows all operations on the global retention configuration
`monitor_data_stream_retention` or higher, which allows the retrieval of the global retention configuration.

This PR is the final PR that makes the global retention available for our users.
2024-03-28 10:40:33 +02:00