mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-24 23:27:25 -04:00
3420be0ca5
This commit fixes the situation where a user wants to use CCR to replicate indices that are part of
a data stream while renaming the data stream. For example, assume a user has an auto-follow request
that looks like this:
```
PUT /_ccr/auto_follow/my-auto-follow-pattern
{
"remote_cluster" : "other-cluster",
"leader_index_patterns" : ["logs-*"],
"follow_index_pattern" : "{{leader_index}}_copy"
}
```
And then the data stream `logs-mysql-error` was created, creating the backing index
`.ds-logs-mysql-error-2022-07-29-000001`.
Prior to this commit, replicating this data stream means that the backing index would be renamed to
`.ds-logs-mysql-error-2022-07-29-000001_copy` and the data stream would *not* be renamed. This
caused a check to trip in `TransportPutLifecycleAction` asserting that a backing index was not
renamed for a data stream during following.
After this commit, there are a couple of changes:
First, the data stream will also be renamed. This means that the `logs-mysql-error` becomes
`logs-mysql-error_copy` when created on the follower cluster. Because of the way that CCR works,
this means we need to support renaming a data stream for a regular "create follower" request, so a
new parameter has been added: `data_stream_name`. It works like this:
```
PUT /mynewindex/_ccr/follow
{
"remote_cluster": "other-cluster",
"leader_index": "myotherindex",
"data_stream_name": "new_ds"
}
```
Second, the backing index for a data stream must be renamed in a way that does not break the parsing
of a data stream backing pattern, whereas previously the index
`.ds-logs-mysql-error-2022-07-29-000001` would be renamed to
`.ds-logs-mysql-error-2022-07-29-000001_copy` (an illegal name since it doesn't end with the
rollover digit), after this commit it will be renamed to
`.ds-logs-mysql-error_copy-2022-07-29-000001` to match the renamed data stream. This means that for
the given `follow_index_pattern` of `{{leader_index}}_copy` the index changes look like:
| Leader Cluster | Follower Cluster |
|--------------|-----------|
| `logs-mysql-error` (data stream) | `logs-mysql-error_copy` (data stream) |
| `.ds-logs-mysql-error-2022-07-29-000001` | `.ds-logs-mysql-error_copy-2022-07-29-000001` |
Which internally means the auto-follow request turned into the create follower request of:
```
PUT /.ds-logs-mysql-error_copy-2022-07-29-000001/_ccr/follow
{
"remote_cluster": "other-cluster",
"leader_index": ".ds-logs-mysql-error-2022-07-29-000001",
"data_stream_name": "logs-mysql-error_copy"
}
```
Relates to https://github.com/elastic/elasticsearch/pull/84940 (cherry-picked the commit for a test)
Relates to https://github.com/elastic/elasticsearch/pull/61993 (where data stream support was first introduced for CCR)
Resolves https://github.com/elastic/elasticsearch/issues/81751
|
||
|---|---|---|
| .. | ||
| aggregations | ||
| analysis | ||
| autoscaling | ||
| cat | ||
| ccr | ||
| cluster | ||
| commands | ||
| data-management | ||
| data-streams | ||
| docs | ||
| eql | ||
| features/apis | ||
| fleet | ||
| graph | ||
| health | ||
| high-availability | ||
| how-to | ||
| ilm | ||
| images | ||
| index-modules | ||
| indices | ||
| ingest | ||
| licensing | ||
| mapping | ||
| migration | ||
| ml | ||
| modules | ||
| monitoring | ||
| query-dsl | ||
| release-notes | ||
| repositories-metering-api | ||
| rest-api | ||
| rollup | ||
| scripting | ||
| search | ||
| searchable-snapshots | ||
| settings | ||
| setup | ||
| shutdown/apis | ||
| slm/apis | ||
| snapshot-restore | ||
| sql | ||
| tab-widgets | ||
| text-structure/apis | ||
| transform | ||
| troubleshooting | ||
| upgrade | ||
| vectors | ||
| aggregations.asciidoc | ||
| alias.asciidoc | ||
| analysis.asciidoc | ||
| api-conventions.asciidoc | ||
| cat.asciidoc | ||
| cluster.asciidoc | ||
| data-management.asciidoc | ||
| data-rollup-transform.asciidoc | ||
| datatiers.asciidoc | ||
| dependencies-versions.asciidoc | ||
| docs.asciidoc | ||
| gs-index.asciidoc | ||
| high-availability.asciidoc | ||
| how-to.asciidoc | ||
| index-extra-title-page.html | ||
| index-modules.asciidoc | ||
| index.asciidoc | ||
| index.x.asciidoc | ||
| indices.asciidoc | ||
| ingest.asciidoc | ||
| intro.asciidoc | ||
| links.asciidoc | ||
| mapping.asciidoc | ||
| query-dsl.asciidoc | ||
| redirects.asciidoc | ||
| release-notes.asciidoc | ||
| scripting.asciidoc | ||
| search.asciidoc | ||
| setup.asciidoc | ||
| troubleshooting.asciidoc | ||
| upgrade.asciidoc | ||