mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 01:38:56 -04:00
* [DOCS] Adds documentation for index lifecycle policies * [DOCS] Updated image for policy options to show all menu items * Update create-policy.asciidoc * [DOCS] Incorporated review comments on hot and warm phase * [DOCS] Additional changes to warm phase * [DOCS] Removed the word open in the warm phase
This commit is contained in:
parent
5fd3742f5f
commit
5a3fcbd5be
9 changed files with 162 additions and 0 deletions
BIN
docs/images/index-lifecycle-policies-create.png
Normal file
BIN
docs/images/index-lifecycle-policies-create.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 145 KiB |
BIN
docs/images/index_lifecycle_policies_options.png
Normal file
BIN
docs/images/index_lifecycle_policies_options.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 90 KiB |
BIN
docs/images/index_management_add_policy.png
Normal file
BIN
docs/images/index_management_add_policy.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 189 KiB |
|
@ -19,6 +19,16 @@ include::management/rollups/create_and_manage_rollups.asciidoc[]
|
|||
|
||||
include::management/rollups/visualize_rollup_data.asciidoc[]
|
||||
|
||||
include::management/index-lifecycle-policies/intro-to-lifecycle-policies.asciidoc[]
|
||||
|
||||
include::management/index-lifecycle-policies/create-policy.asciidoc[]
|
||||
|
||||
include::management/index-lifecycle-policies/manage-policy.asciidoc[]
|
||||
|
||||
include::management/index-lifecycle-policies/add-policy-to-index.asciidoc[]
|
||||
|
||||
include::management/index-lifecycle-policies/example-index-lifecycle-policy.asciidoc[]
|
||||
|
||||
include::management/managing-fields.asciidoc[]
|
||||
|
||||
include::management/managing-indices.asciidoc[]
|
||||
|
|
|
@ -0,0 +1,17 @@
|
|||
[[adding-policy-to-index]]
|
||||
=== Adding a policy to an index
|
||||
|
||||
You can add a lifecycle policy to an index and view the status for indices
|
||||
managed by a policy in *Management > {es} > Index Management*. This page lists your
|
||||
{es} indices, which you can filter by lifecycle status and lifecycle phase.
|
||||
|
||||
To add a policy, select the index name and then select *Manage > Add lifecycle policy*.
|
||||
You’ll see the policy name, the phase the index is in, the current
|
||||
action, and if any errors occurred performing that action.
|
||||
|
||||
To remove a policy from an index, select *Manage > Remove lifecycle policy*.
|
||||
|
||||
[role="screenshot"]
|
||||
image::images/index_management_add_policy.png[][UI for adding a policy to an index]
|
||||
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
[[creating-index-lifecycle-policies]]
|
||||
=== Creating an index lifecycle policy
|
||||
|
||||
An index lifecycle policy enables you to define rules over when to perform
|
||||
certain actions, such as a rollover or force merge, on an index. Index lifecycle
|
||||
management automates execution of those actions at the right time.
|
||||
|
||||
When you create an index lifecycle policy, consider the tradeoffs between
|
||||
performance and availability. As you move your index through the lifecycle,
|
||||
you’re likely moving your data to less performant hardware and reducing the
|
||||
number of shards and replicas. It’s important to ensure that the index
|
||||
continues to have enough replicas to prevent data loss in the event of failures.
|
||||
|
||||
*Index Lifecycle Policies* is automatically enabled in {kib}. Go to
|
||||
*Management > {es} > Index Lifecycle Policies*.
|
||||
|
||||
NOTE: If you don’t want to use this feature, you can disable it by setting
|
||||
`xpack.ilm.enabled` to false in your `kibana.yml` configuration file. If you
|
||||
disable *Index Management*, then *Index Lifecycle Policies* is also disabled.
|
||||
|
||||
[role="screenshot"]
|
||||
image::images/index-lifecycle-policies-create.png[][UI for creating an index lifecycle policy]
|
||||
|
||||
You can define up to four phases in the index lifecycle. For each phase, you
|
||||
can enable actions to optimize performance for that phase. Transitioning
|
||||
between phases is based on the age of the index.
|
||||
|
||||
The four phases of an index lifecycle policy are:
|
||||
|
||||
* *Hot.* The index is actively being queried and written to. You can optionally
|
||||
roll over to a new index when the
|
||||
original index reaches a specified size or age. When a rollover occurs, a new
|
||||
index is created, added to the index alias, and designated as the new “hot”
|
||||
index. You can still query the previous indices, but you only ever write to
|
||||
the “hot” index. See {ref}/indices-rollover-index.html[Rollover index] for more information.
|
||||
|
||||
* *Warm.* The index is typically searched at a lower rate than when the data is
|
||||
hot. The index is not used for storing for new data, but might occasionally add
|
||||
late-arriving data, for example, from a Beat that had a network problem that's now fixed.
|
||||
You can optionally shrink the number replicas and move the shards to a
|
||||
different set of nodes with smaller or less performant hardware. You can also
|
||||
reduce the number of primary shards and force merge the index into
|
||||
smaller {ref}/indices-segments.html[segments].
|
||||
|
||||
* *Cold.* The index is no longer being updated and is seldom queried, but is
|
||||
still searchable. If you have a big deployment, you can move it to even
|
||||
less performant hardware. You might also reduce the number of replicas because
|
||||
you expect the data to be queried less frequently.
|
||||
|
||||
* *Delete.* The index is no longer relevant. You can define when it is safe to
|
||||
delete it.
|
||||
|
||||
The index lifecycle always includes an active hot phase. The warm, cold, and
|
||||
delete phases are optional. For example, you might define all four phases for
|
||||
one policy and only a hot and delete phase for another. See {ref}/_actions.html[Actions]
|
||||
for more information on the actions available in each phase.
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
[[example-using-index-lifecycle-policy]]
|
||||
=== Example of using an index lifecycle policy
|
||||
|
||||
A common use case for managing index lifecycle policies is when you’re using
|
||||
{beats-ref}/beats-reference.html[Beats] to continually send time-series data,
|
||||
such as metrics and log data, to {es}. When you create the Beats packages, an
|
||||
index template is installed. The template includes a default policy to apply
|
||||
when new indices are created.
|
||||
|
||||
You can edit the policy in {kib}'s *Index Lifecycle Policies*. For example, you might:
|
||||
|
||||
* Rollover the index when it reaches 50 GB in size or is 30 days old. These
|
||||
settings are the default for the Beats lifecycle policy. This avoids
|
||||
having 1000s of tiny indices. When a rollover occurs, a new “hot” index is
|
||||
created and added to the index alias.
|
||||
|
||||
* Move the index into the warm phase, shrink the index down to a single shard,
|
||||
and force merge to a single segment.
|
||||
|
||||
* After 60 days, move the index into the cold phase and onto less expensive hardware.
|
||||
|
||||
* Delete the index after 90 days.
|
|
@ -0,0 +1,29 @@
|
|||
[[index-lifecycle-policies]]
|
||||
== Index lifecycle policies
|
||||
|
||||
If you're working with time series data, you don't want to continually dump
|
||||
everything into a single index. Instead, you might periodically roll over the
|
||||
data to a new index to keep it from growing so big it's slow and expensive.
|
||||
As the index ages and you query it less frequently, you’ll likely move it to
|
||||
less expensive hardware and reduce the number of shards and replicas.
|
||||
|
||||
To automatically move an index through its lifecycle, you can create a policy
|
||||
to define actions to perform on the index as it ages. Index lifecycle policies
|
||||
are especially useful when working with {beats-ref}/beats-reference.html[Beats]
|
||||
data shippers, which continually
|
||||
send operational data, such as metrics and logs, to Elasticsearch. You can
|
||||
automate a rollover to a new index when the existing index reaches a specified
|
||||
size or age. This ensures that all indices have a similar size instead of having
|
||||
daily indices where size can vary based on the number of Beats and the number
|
||||
of events sent.
|
||||
|
||||
{kib}’s *Index Lifecycle Policies* walks you through the process for creating
|
||||
and configuring a policy. Before using this feature, you should be familiar
|
||||
with index lifecycle management:
|
||||
|
||||
* For an introduction, see
|
||||
{ref}/getting-started-index-lifecycle-management.html[Getting started with index
|
||||
lifecycle management].
|
||||
* To dig into the concepts and technical details, see
|
||||
{ref}/index-lifecycle-management.html[Managing the index lifecycle].
|
||||
* To check out the APIs, see {ref}/index-lifecycle-management-api.html[Index lifecycle management API].
|
|
@ -0,0 +1,27 @@
|
|||
[[managing-index-lifecycle-policies]]
|
||||
=== Managing index lifecycle policies
|
||||
|
||||
Your configured policies appear on the *Index lifecycle policies* page.
|
||||
You can update an existing index lifecycle policy to fix errors or change
|
||||
strategies for newly created indices. To edit a policy, select its name.
|
||||
|
||||
[role="screenshot"]
|
||||
image::images/index_lifecycle_policies_options.png[][UI for viewing and editing an index lifecycle policy]
|
||||
|
||||
In addition, you can:
|
||||
|
||||
* *View indices linked to the policy.* This is important when editing a policy.
|
||||
Any changes you make affect all indices attached to the policy. The settings
|
||||
for the current phase are cached, so the update doesn’t affect that phase. This
|
||||
prevents conflicts when you’re modifying a phase that is currently executing on
|
||||
an index. The changes takes effect when the next phase in the index lifecycle begins.
|
||||
|
||||
* *Add the policy to an index template.* When an index is automatically
|
||||
created using the index template, the policy is applied. If the index is rolled
|
||||
over, the policies for any matching index templates are applied to the newly
|
||||
created index. For more information, see {ref}/indices-templates.html[Index templates].
|
||||
|
||||
* *Delete a policy.* You can’t delete a policy that is currently in use or
|
||||
recover a deleted index.
|
||||
|
||||
|
Loading…
Add table
Add a link
Reference in a new issue