mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-25 15:47:23 -04:00
839afdc331
This releases the Data stream lifecycle feature as a Technical Preview feature. Data stream lifecycle, albeit in technical preview, will allow data streams to take advantage of a native simplified and resilient lifecycle implementation.
138 lines
5.1 KiB
Text
138 lines
5.1 KiB
Text
[role="xpack"]
|
|
[[tutorial-manage-existing-data-stream]]
|
|
=== Tutorial: Update existing data stream
|
|
|
|
preview:[]
|
|
|
|
To update the lifecycle of an existing data stream you do the following actions:
|
|
|
|
. <<set-lifecycle>>
|
|
. <<delete-lifecycle>>
|
|
|
|
[discrete]
|
|
[[set-lifecycle]]
|
|
==== Set a data stream's lifecycle
|
|
|
|
To add or to change the retention period of your data stream you can use the <<data-streams-put-lifecycle, PUT lifecycle API>>.
|
|
|
|
* You can set infinite retention period, meaning that your data should never be deleted. For example:
|
|
+
|
|
[source,console]
|
|
----
|
|
PUT _data_stream/my-data-stream/_lifecycle
|
|
{ } <1>
|
|
----
|
|
// TEST[setup:my_data_stream]
|
|
<1> An empty payload means that your data stream is still managed but the data will never be deleted. Managing a time
|
|
series data stream such as logs or metrics enables {es} to better store your data even if you do not use a retention period.
|
|
|
|
* Or you can set the retention period of your choice. For example:
|
|
+
|
|
[source,console]
|
|
----
|
|
PUT _data_stream/my-data-stream/_lifecycle
|
|
{
|
|
"data_retention": "30d" <1>
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
<1> The retention period of this data stream is set to 30 days. This means that {es} is allowed to delete data that is
|
|
older than 30 days at its own discretion.
|
|
|
|
The changes in the lifecycle are applied on all backing indices of the data stream. You can see the effect of the change
|
|
via the <<data-streams-explain-lifecycle, explain API>>:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
GET .ds-my-data-stream-*/_lifecycle/explain
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
The response will look like:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"indices": {
|
|
".ds-my-data-stream-2023.04.19-000002": {
|
|
"index": ".ds-my-data-stream-2023.04.19-000002", <1>
|
|
"managed_by_lifecycle": true, <2>
|
|
"index_creation_date_millis": 1681919221417,
|
|
"time_since_index_creation": "6.85s", <3>
|
|
"lifecycle": {
|
|
"enabled": true,
|
|
"data_retention": "30d" <4>
|
|
}
|
|
},
|
|
".ds-my-data-stream-2023.04.17-000001": {
|
|
"index": ".ds-my-data-stream-2023.04.17-000001", <5>
|
|
"managed_by_lifecycle": true, <6>
|
|
"index_creation_date_millis": 1681745209501,
|
|
"time_since_index_creation": "48d", <7>
|
|
"rollover_date_millis": 1681919221419,
|
|
"time_since_rollover": "6.84s", <8>
|
|
"generation_time": "6.84s", <9>
|
|
"lifecycle": {
|
|
"enabled": true,
|
|
"data_retention": "30d" <10>
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
// TESTRESPONSE[skip:the result is for illustrating purposes only]
|
|
<1> The name of the backing index.
|
|
<2> This index is managed by a data stream lifecycle.
|
|
<3> The time that has passed since this index has been created.
|
|
<4> The data retention for this index is at least 30 days, as it was recently updated.
|
|
<5> The name of the backing index.
|
|
<6> This index is managed by the built-in data stream lifecycle.
|
|
<7> The time that has passed since this index has been created.
|
|
<8> The time that has passed since this index was <<index-rollover,rolled over>>.
|
|
<9> The time that will be used to determine when it's safe to delete this index and all its data.
|
|
<10> The data retention for this index as well is at least 30 days, as it was recently updated.
|
|
|
|
[discrete]
|
|
[[delete-lifecycle]]
|
|
==== Remove lifecycle for a data stream
|
|
|
|
To remove the lifecycle of a data stream you can use the <<data-streams-delete-lifecycle-request,delete lifecycle API>>. As consequence,
|
|
the maintenance operations that were applied by the lifecycle will no longer be applied to the data stream and all its
|
|
backing indices. For example:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
DELETE _data_stream/my-data-stream/_lifecycle
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
You can then use the <<data-streams-explain-lifecycle, explain API>> again to see that the indices are no longer managed.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
GET .ds-my-data-stream-*/_lifecycle/explain
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
// TEST[teardown:data_stream_cleanup]
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"indices": {
|
|
".ds-my-data-stream-2023.04.19-000002": {
|
|
"index": ".ds-my-data-stream-2023.04.19-000002", <1>
|
|
"managed_by_lifecycle": false <2>
|
|
},
|
|
".ds-my-data-stream-2023.04.17-000001": {
|
|
"index": ".ds-my-data-stream-2023.04.19-000001", <3>
|
|
"managed_by_lifecycle": false <4>
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[skip:the result is for illustrating purposes only]
|
|
<1> The name of the backing index.
|
|
<2> Indication that the index is not managed by the data stream lifecycle.
|
|
<3> The name of another backing index.
|
|
<4> Indication that the index is not managed by the data stream lifecycle.
|