elasticsearch/docs/reference/data-streams
Mary Gouseti bed6e18fa3
Exclude internal data streams from global retention (#112100)
With #111972 we enable users to set up global retention for data streams that are managed by the data stream lifecycle. This will allow users of elasticsearch to have a more control over their data retention, and consequently better resource management of their clusters.

However, there is a small number of data streams that are necessary for the good operation of elasticsearch and should not follow user defined retention to avoid surprises.

For this reason, we put forth the following definition of internal data streams.

A data stream is internal if it's either a system index (system flag is true) or if its name starts with a dot.

This PR adds the `isInternalDataStream` param in the effective retention calculation making explicit that this is also used to determine the effective retention.
2024-08-28 11:28:35 +03:00
..
lifecycle Exclude internal data streams from global retention (#112100) 2024-08-28 11:28:35 +03:00
change-mappings-and-settings.asciidoc (Doc+) Link API doc to parent object - part1 (#111951) 2024-08-20 14:58:18 -06:00
data-stream-apis.asciidoc [DSL] Remove REST APIs for global retention (#107565) 2024-04-17 21:36:26 +03:00
data-streams.asciidoc Adding minimal docs around using index mode logs. (#110932) 2024-07-18 03:52:58 +10:00
downsampling-dsl.asciidoc Add documentation for configuring downsampling using DSL (#108043) 2024-04-30 09:23:39 -07:00
downsampling-ilm.asciidoc Introduce lazy rollover for mapping updates in data streams (#103309) 2024-01-08 15:07:16 +02:00
downsampling-manual.asciidoc (Doc+) Link API doc to parent object - part1 (#111951) 2024-08-20 14:58:18 -06:00
downsampling.asciidoc Remove average from downsampling statistics in documentation (#110189) 2024-06-26 17:23:06 +03:00
logs.asciidoc Rename logs index mode to logsdb (#111054) 2024-07-19 13:38:58 +02:00
modify-data-streams-api.asciidoc (Doc+) Link API doc to parent object - part1 (#111951) 2024-08-20 14:58:18 -06:00
promote-data-stream-api.asciidoc (Doc+) Link API doc to parent object - part1 (#111951) 2024-08-20 14:58:18 -06:00
set-up-a-data-stream.asciidoc [DOCS] Add note that Logstash sets up data streams (#109502) 2024-06-10 12:24:23 -04:00
set-up-tsds.asciidoc Add a note about enabling time series index mode via a component template (#110050) 2024-06-25 17:22:31 +10:00
tsds-index-settings.asciidoc Adjust index.mapping.dimension_fields.limit index setting default in docs. (#109091) 2024-05-31 10:52:31 +02:00
tsds-reindex.asciidoc (Doc+) Link API doc to parent object - part1 (#111951) 2024-08-20 14:58:18 -06:00
tsds.asciidoc Add support for boolean dimensions (#111457) 2024-07-31 23:00:32 +10:00
use-a-data-stream.asciidoc Introduce lazy rollover for mapping updates in data streams (#103309) 2024-01-08 15:07:16 +02:00