This enhances the migrate to data tiers routing API to also iterate over
the existing legacy, composable, and component templates and look if
they define a custom node attribute routing in their settings for either
`index.routing.allocation.require.{nodeAttrName}` or
`index.routing.allocation.include.{nodeAttrName}`. If any does, we
update them to remove all the routings settings for the provided
`nodeAttrName`.
eg. any template with the following setting configuration:
```
"settings": {
index.routing.allocation.require.data: "warm",
index.routing.allocation.include.data: "rack1",
index.routing.allocation.exclude.data: "rack2,rack3"
}
```
will have its settings updated to:
```
"settings": {}
```
The migrate to data tiers routing API required ILM to be stopped. This
is fine for "live" runs, but for dry runs this isn't a requirement.
This changes the dry_run to allow the API to run irrespective of the ILM
status.
* Expose the index age in ILM explain output.
ILM already exposes the `age` that ILM will use to transition to the next phase, based on that phase's `min_age`. The `index_age` is based only on the index creation date and it's used to trigger a rollover.
Resolves#64429
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.
Relates to #79309, #31619
* Allow ILM move-to-step without `action` or `name`
This commit enhances ILM's move-to-step API to allow dropping the `name`, or dropping both the
`action` and `name`. For example:
```json
POST /_ilm/move/foo-1
{
"current_step": {
"phase": "hot",
"action": "rollover",
"name": "check-rollover-ready"
},
"next_step": {
"phase": "warm",
"action": "forcemerge"
}
}
```
Will move to the first step in the `forcemerge` action in the `warm` phase (without having to know
the specific step name).
Another example:
```json
POST /_ilm/move/foo-1
{
"current_step": {
"phase": "hot",
"action": "rollover",
"name": "check-rollover-ready"
},
"next_step": {
"phase": "warm"
}
}
```
Will move to the first step in the `warm` phase (without having to know the specific action name).
Bear in mind that the execution order is still entirely an implementation detail, so "first" in the
above sentences means the first step that ILM would execute.
Resolves#58128
* Apply Andrei's wording change (thanks!)
Co-authored-by: Andrei Dan <andrei.dan@elastic.co>
* Log index and policy name when the concrete step key can't be resolved
Co-authored-by: Andrei Dan <andrei.dan@elastic.co>
This adds support for a `dry_run` parameter for the
`_ilm/migrate_to_data_tiers` API. This defaults to `false`, but when
configured to `true` it will simulate the migration of elasticsearch
entities to data tiers based routing, returning the entites that need to
be updated (indices, ILM policies and the legacy index template that'd
be deleted, if any was configured in the request).
This adds the _ilm/migrate_to_data_tiers API to expose the service for
migrating the elasticsearch abstractions (indices, ILM policies and an
optional legacy template to delete) to data tiers routing allocation
(away from custom node attributes)
This commit adds the "in_use_by" object to the response for ILM policies. This map shows the
indices, data streams, and composable templates that use the ILM policy.
An example output may look like:
```json
{
"logs" : {
"version" : 1,
"modified_date" : "2021-06-23T18:42:08.381Z",
"policy" : {
...
},
"in_use_by" : {
"indices" : [".ds-logs-foo-barbaz-2021.06.23-000001", ".ds-logs-foo-other-2021.06.23-000001"],
"data_streams" : ["logs-foo-barbaz", "logs-foo-other"],
"composable_templates" : ["logs"]
}
}
}
```
Resolves#73869
Relates to #70755.
The main changes of this PR are:
Add an optional _meta field to ILM policy.
Add some test code about the change.
Update the doc of Create or update lifecycle policy API.
This adds ILM support for automatically migrating the managed
indices between data tiers.
This proposal makes use of a MigrateAction that is injected
(similar to how the Unfollow action is injected) in phases that
don't define index allocation rules using the AllocateAction or
don't explicitly define the MigrateAction itself (regardless if it's
enabled or disabled).
* [DOCS] Rework conceptual info for ILM.
* Split the actions out of concepts.
* Added xpack role to actions.
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
* Apply suggestions from code review
This commit merges the searchable-snapshots feature branch into master.
See #54803 for the complete list of squashed commits.
Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: Yannick Welsch <yannick@welsch.lu>
Co-authored-by: Lee Hinman <dakrone@users.noreply.github.com>
Co-authored-by: Andrei Dan <andrei.dan@elastic.co>
This adds the infrastructure to be able to retry the execution of retryable
steps and makes the `check-rollover-ready` retryable as an initial step to
make the rollover action more resilient to transient errors.
This commit adds HLRC support and documentation for the SLM Start and
Stop APIs, as well as updating existing documentation where appropriate.
This commit also ensures that the SLM APIs are properly included in the
HLRC documentation.
* Add Snapshot Lifecycle Retention documentation
This commits adds API and general purpose documentation for SLM
retention.
Relates to #43663
* Fix docs tests
* Update default now that #47604 has been merged
* Update docs/reference/ilm/apis/slm-api.asciidoc
Co-Authored-By: Gordon Brown <gordon.brown@elastic.co>
* Update docs/reference/ilm/apis/slm-api.asciidoc
Co-Authored-By: Gordon Brown <gordon.brown@elastic.co>
* Update docs with feedback
This commit adds support for POST requests to the SLM `_execute` API,
because POST is a more appropriate HTTP verb for this action as it is
not idempotent. The docs are also changed to favor POST over PUT,
although PUT is not removed or officially deprecated.
Using arrays of objects with embedded IDs is preferred for new APIs over
using entity IDs as JSON keys. This commit changes the SLM stats API to
use the preferred format.
