elasticsearch/docs/reference/ilm/apis/migrate-to-data-tiers.asciidoc
Andrei Dan 3087f164f7
Migrate legacy/v2/component templates away from custom attributes routing (#82472)
This enhances the migrate to data tiers routing API to also iterate over
the existing legacy, composable, and component templates and look if
they define a custom node attribute routing in their settings for either
`index.routing.allocation.require.{nodeAttrName}` or
`index.routing.allocation.include.{nodeAttrName}`. If any does, we
update them to remove all the routings settings for the provided
`nodeAttrName`.

eg. any template with the following setting configuration:
```
"settings": {
  index.routing.allocation.require.data: "warm",
  index.routing.allocation.include.data: "rack1",
  index.routing.allocation.exclude.data: "rack2,rack3"
}
```
will have its settings updated to:
```
"settings": {}
```
2022-01-13 10:45:10 +00:00

186 lines
5.8 KiB
Text

[role="xpack"]
[[ilm-migrate-to-data-tiers]]
=== Migrate to data tiers routing API
++++
<titleabbrev>Migrate indices, ILM policies, and legacy, composable and component templates to data tiers routing</titleabbrev>
++++
Switches the indices, ILM policies, and legacy, composable and component templates from using custom node attributes and
<<shard-allocation-filtering, attribute-based allocation filters>> to using <<data-tiers, data tiers>>, and
optionally deletes one legacy index template.
Using node roles enables {ilm-init} to <<data-tier-migration, automatically move the indices>> between
data tiers.
Migrating away from custom node attributes routing can be manually performed
as indicated in the <<migrate-index-allocation-filters, Migrate index allocation
filters to node roles>> page.
This API provides an automated way of executing three out of the four manual steps listed
in the <<migrate-index-allocation-filters, migration guide>>:
. <<stop-setting-custom-hot-attribute, Stop setting the custom hot attribute on new indices>>
. <<remove-custom-allocation-settings, Remove custom allocation settings from existing {ilm-init} policies>>
. <<set-tier-preference, Replace custom allocation settings from existing indices>> with the corresponding <<tier-preference-allocation-filter,tier preference>>
[[ilm-migrate-to-data-tiers-request]]
==== {api-request-title}
`POST /_ilm/migrate_to_data_tiers`
The API accepts an optional body that allows you to specify:
- The legacy index template name to delete. Defaults to none.
- The name of the custom node attribute used for the indices and ILM policies allocation filtering.
Defaults to `data`.
[[ilm-migrate-to-data-tiers-prereqs]]
==== {api-prereq-title}
* {ilm-init} must be stopped before performing the migration. Use the <<ilm-stop-request, stop ILM API>>
to stop {ilm-init} and <<ilm-get-status-request, get status API>> to wait until the
reported operation mode is `STOPPED`.
[[ilm-migrate-to-data-tiers-query-params]]
==== {api-query-parms-title}
`dry_run`::
(Optional, Boolean)
If `true`, simulates the migration from node attributes based allocation filters to data tiers, but does
not perform the migration. This provides a way to retrieve the indices and ILM policies that need to be
migrated.
Defaults to `false`.
NOTE: When simulating a migration (ie. `dry_run` is `true`) {ilm-init} doesn't need to be stopped.
[[ilm-migrate-to-data-tiers-example]]
==== {api-examples-title}
The following example migrates the indices, ILM policies, legacy templates,
composable, and component templates away from defining custom allocation filtering
using the `custom_attribute_name` node attribute, and deletes the legacy template
with name `global-template` if it exists in the system.
////
[source,console]
----
POST _ilm/stop
PUT _template/global-template
{
"index_patterns": ["migrate-to-tiers-*"],
"settings": {
"index.routing.allocation.require.custom_attribute_name": "hot"
}
}
PUT _template/a-legacy-template
{
"index_patterns": ["legacy-template-migrate-to-tiers-*"],
"settings": {
"index.routing.allocation.require.custom_attribute_name": "hot"
}
}
PUT _index_template/a-composable-template
{
"index_patterns": [ "composable-template-migrate-to-tiers-*" ],
"data_stream": {},
"template" : {
"settings": {
"index.routing.allocation.require.custom_attribute_name": "hot"
}
}
}
PUT _component_template/a-component-template
{
"template" : {
"settings": {
"index.routing.allocation.require.custom_attribute_name": "hot"
}
}
}
PUT warm-index-to-migrate-000001
{
"settings": {
"index.routing.allocation.require.custom_attribute_name": "warm"
}
}
PUT _ilm/policy/policy_with_allocate_action
{
"policy": {
"phases": {
"warm": {
"actions": {
"allocate": {
"require": {
"custom_attribute_name": "warm"
}
}
}
},
"delete": {
"min_age": "30d",
"actions": {
"delete": {}
}
}
}
}
}
----
// TESTSETUP
[source,console]
----
DELETE warm-index-to-migrate-000001
DELETE _ilm/policy/policy_with_allocate_action
DELETE _template/a-legacy-template
DELETE _index_template/a-composable-template
DELETE _component_template/a-component-template
POST _ilm/start
----
// TEARDOWN
////
[source,console]
----------------------------------------------------------------
POST /_ilm/migrate_to_data_tiers
{
"legacy_template_to_delete": "global-template",
"node_attribute": "custom_attribute_name"
}
----------------------------------------------------------------
If the request succeeds, a response like the following will be received:
[source,console-result]
------------------------------------------------------------------------------
{
"dry_run": false,
"removed_legacy_template":"global-template", <1>
"migrated_ilm_policies":["policy_with_allocate_action"], <2>
"migrated_indices":["warm-index-to-migrate-000001"], <3>
"migrated_legacy_templates":["a-legacy-template"], <4>
"migrated_composable_templates":["a-composable-template"], <5>
"migrated_component_templates":["a-component-template"] <6>
}
------------------------------------------------------------------------------
<1> Shows the name of the legacy index template that was deleted. This will be missing
if no legacy index template was deleted.
<2> The ILM policies that were updated.
<3> The indices that were migrated to <<tier-preference-allocation-filter,tier preference>> routing.
<4> The legacy index templates that were updated to not contain custom routing settings for the
provided data attribute.
<5> The composable index templates that were updated to not contain custom routing settings for the
provided data attribute.
<6> The component templates that were updated to not contain custom routing settings for the
provided data attribute.