mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-24 23:27:25 -04:00
c4c3c8b422
This commit adds support data stream support to CCR's auto following by making the following changes: * When the auto follow coordinator iterates over the candidate indices to follow, the auto follow coordinator also checks whether the index is part of a data stream and if the name of data stream also matches with the auto follow pattern then the index will be auto followed. * When following an index, the put follow api also checks whether that index is part of a data stream and if so then also replicates the data stream definition to the local cluster. * In order for the follow index api to determine whether an index is part of a data stream, the cluster state api was modified to also fetch the data stream definition of the cluster state if only the state is queried for specific indices. When a data stream is auto followed, only new backing indices are auto followed. This is in line with how time based indices patterns are replicated today. This means that the data stream isn't copied 1 to 1 into the local cluster. The local cluster's data stream definition contains the same name, timestamp field and generation, but the list of backing indices may be different (depending on when a data stream was auto followed). Closes #56259
89 lines
3.3 KiB
Text
89 lines
3.3 KiB
Text
[role="xpack"]
|
|
[testenv="platinum"]
|
|
[[ccr-auto-follow]]
|
|
=== Manage auto-follow patterns
|
|
To replicate time series indices, you configure an auto-follow pattern so that
|
|
each new index in the series is replicated automatically. Whenever the name of
|
|
a new index on the remote cluster matches the auto-follow pattern, a
|
|
corresponding follower index is added to the local cluster.
|
|
|
|
You can also create auto-follow patterns for data streams. When a new backing
|
|
index is generated on a remote cluster, that index and its data stream are
|
|
automatically followed if the data stream name matches an auto-follow
|
|
pattern. If you create a data stream after creating the auto-follow pattern,
|
|
all backing indices are followed automatically.
|
|
|
|
|
|
Auto-follow patterns are especially useful with
|
|
<<index-lifecycle-management,{ilm-cap}>>, which might continually create
|
|
new indices on the cluster containing the leader index.
|
|
|
|
[[ccr-access-ccr-auto-follow]]
|
|
To start using {ccr} auto-follow patterns, access {kib} and go to
|
|
*Management > Stack Management*. In the side navigation, select
|
|
*Cross-Cluster Replication* and choose the *Auto-follow patterns* tab
|
|
|
|
[[ccr-auto-follow-create]]
|
|
==== Create auto-follow patterns
|
|
When you <<ccr-getting-started-auto-follow,create an auto-follow pattern>>,
|
|
you are configuring a collection of patterns against a single remote cluster.
|
|
When an index is created in the remote cluster with a name that matches one of
|
|
the patterns in the collection, a follower index is configured in the local
|
|
cluster. The follower index uses the new index as its leader index.
|
|
|
|
[%collapsible]
|
|
.Use the API
|
|
====
|
|
Use the <<ccr-put-auto-follow-pattern,create auto-follow pattern API>> to add a
|
|
new auto-follow pattern configuration.
|
|
====
|
|
|
|
[[ccr-auto-follow-retrieve]]
|
|
==== Retrieve auto-follow patterns
|
|
To view existing auto-follow patterns and make changes to the backing
|
|
patterns, <<ccr-access-ccr-auto-follow,access {kib}>> on your _remote_ cluster.
|
|
|
|
Select the auto-follow pattern that you want to view details about. From there,
|
|
you can make changes to the auto-follow pattern. You can also view your
|
|
follower indices included in the auto-follow pattern.
|
|
|
|
[%collapsible]
|
|
.Use the API
|
|
====
|
|
Use the <<ccr-get-auto-follow-pattern,get auto-follow pattern API>> to inspect
|
|
all configured auto-follow pattern collections.
|
|
====
|
|
|
|
[[ccr-auto-follow-pause]]
|
|
==== Pause and resume auto-follow patterns
|
|
To pause and resume replication of auto-follow pattern collections,
|
|
<<ccr-access-ccr-auto-follow,access {kib}>>, select the auto-follow pattern,
|
|
and pause replication.
|
|
|
|
To resume replication, select the pattern and choose
|
|
*Manage pattern > Resume replication*.
|
|
|
|
[%collapsible]
|
|
.Use the API
|
|
====
|
|
Use the <<ccr-pause-auto-follow-pattern,pause auto-follow pattern API>> to
|
|
pause auto-follow patterns.
|
|
Use the <<ccr-resume-auto-follow-pattern,resume auto-follow pattern API>> to
|
|
resume auto-follow patterns.
|
|
====
|
|
|
|
[[ccr-auto-follow-delete]]
|
|
==== Delete auto-follow patterns
|
|
To delete an auto-follow pattern collection,
|
|
<<ccr-access-ccr-auto-follow,access {kib}>>, select the auto-follow pattern,
|
|
and pause replication.
|
|
|
|
When the pattern status changes to Paused, choose
|
|
*Manage pattern > Delete pattern*.
|
|
|
|
[%collapsible]
|
|
.Use the API
|
|
====
|
|
Use the <<ccr-delete-auto-follow-pattern,delete auto-follow pattern API>> to
|
|
delete a configured auto-follow pattern collection.
|
|
====
|