Commit graph

13 commits

Author SHA1 Message Date
Mary Gouseti
ed60470518
Display effective retention in the relevant data stream APIs (#112019) 2024-08-22 17:42:49 +03:00
Stef Nestor
c1019d4c5d
(Doc+) Link API doc to parent object - part1 (#111951)
* (Doc+) Link API to parent Doc part1

---------

Co-authored-by: shainaraskas <shaina.raskas@elastic.co>
Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
2024-08-20 14:58:18 -06:00
Niels Bauman
ba91bfdc94
Lazily create the failure store (#109289)
Rather than initializing the failure store right away when a new
data stream is created, we leave it empty and mark it for lazy
rollover. This results in the failure store only being initialized
(i.e. an index created) when a failure has actually occurred.

The exception to the rule is when a failure occurs while the data
stream is being auto-created. In that case, we do want to initialize
the failure store right away.
2024-06-19 13:18:47 +02:00
Niels Bauman
920290a37b
Extract DataStreamIndices from DataStream class (#107562)
We were seeing more and more common fields between "regular" backing indices and failure store indices (i.e. `indices`, `rolloverOnWrite`, `autoShardingEvent`). To avoid having to duplicate these fields (and possibly any future fields), we extract a class that contains these fields.
2024-05-14 11:14:13 +02:00
Lee Hinman
87928b2303
GA data stream lifecycles in documentation (#107582)
This commit removes the warning about techincal preview for the data stream lifecycle documentation pages.
2024-04-18 09:09:44 -06:00
Mary Gouseti
732c7c4c30
[DSL] Remove REST APIs for global retention (#107565) 2024-04-17 21:36:26 +03:00
Mary Gouseti
2988799079
[DSL Global Retention] Use data stream global retention metadata (#106221) 2024-03-20 20:27:08 +02:00
Mary Gouseti
046cdeae23
Introduce lazy rollover for mapping updates in data streams (#103309)
In this PR we implement the idea to introduce a flag, that a data stream needs to be rolloved over before the next document is indexed.
2024-01-08 15:07:16 +02:00
Andrei Dan
2212df73e8
[DOCS] migrate ILM to DSL headings and TLDR (#102068)
This adds some headings and a TL;DR section to the migration to DSL
tutorial.
2023-11-23 06:37:16 -05:00
James Baiera
6fa7f60073
Add ability to create a data stream failure store (#99134)
Adds the ability to configure a data stream to create a new kind of backing index called a failure store which will eventually be used to store error information when ingest pipelines fail to ingest a document or when a document fails to be parsed correctly by the configured mapping on the data stream.
2023-11-15 15:32:51 -05:00
Andrei Dan
6054a5eb18
[DOCS] Fix typo (#101791) 2023-11-13 06:07:03 -05:00
Andrei Dan
7b436bae2c
[DOCS] DSL: More visible tech preview tags (#101313) 2023-10-26 12:06:15 +01:00
Andrei Dan
74ea04fb2d
[DOCS] document tail merging and create tutorial for migrating to DSL (#101117)
This documents tail merging, the enabled flag, and
adds a tutorial to migrate a data stream from ILM to DSL.
2023-10-25 11:12:36 +01:00