mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-06-28 17:34:17 -04:00
[DOCS] Reworked the shard allocation filtering info. (#36456)
* [DOCS] Reworked the shard allocation filtering info. Closes #36079 * Added multiple index allocation settings example back. * Removed extraneous space
This commit is contained in:
parent
c3a6d1998a
commit
c9e03e6ead
6 changed files with 177 additions and 171 deletions
|
@ -1,29 +1,54 @@
|
|||
[[shard-allocation-filtering]]
|
||||
=== Shard Allocation Filtering
|
||||
=== Index-level shard allocation filtering
|
||||
|
||||
Shard allocation filtering allows you to specify which nodes are allowed
|
||||
to host the shards of a particular index.
|
||||
You can use shard allocation filters to control where {es} allocates shards of
|
||||
a particular index. These per-index filters are applied in conjunction with
|
||||
<<allocation-filtering, cluster-wide allocation filtering>> and
|
||||
<<allocation-awareness, allocation awareness>>.
|
||||
|
||||
NOTE: The per-index shard allocation filters explained below work in
|
||||
conjunction with the cluster-wide allocation filters explained in
|
||||
<<shards-allocation>>.
|
||||
Shard allocation filters can be based on custom node attributes or the built-in
|
||||
`_name`, `host_ip`, `publish_ip`, `_ip`, and `_host` attributes.
|
||||
<<index-lifecycle-management, Index lifecycle management>> uses filters based
|
||||
on custom node attributes to determine how to reallocate shards when moving
|
||||
between phases.
|
||||
|
||||
It is possible to assign arbitrary metadata attributes to each node at
|
||||
startup. For instance, nodes could be assigned a `rack` and a `size`
|
||||
attribute as follows:
|
||||
The `cluster.routing.allocation` settings are dynamic, enabling live indices to
|
||||
be moved from one set of nodes to another. Shards are only relocated if it is
|
||||
possible to do so without breaking another routing constraint, such as never
|
||||
allocating a primary and replica shard on the same node.
|
||||
|
||||
For example, you could use a custom node attribute to indicate a node's
|
||||
performance characteristics and use shard allocation filtering to route shards
|
||||
for a particular index to the most appropriate class of hardware.
|
||||
|
||||
[float]
|
||||
[[index-allocation-filters]]
|
||||
==== Enabling index-level shard allocation filtering
|
||||
|
||||
To filter based on a custom node attribute:
|
||||
|
||||
. Specify the filter characteristics with a custom node attribute in each
|
||||
node's `elasticsearch.yml` configuration file. For example, if you have `small`,
|
||||
`medium`, and `big` nodes, you could add a `size` attribute to filter based
|
||||
on node size.
|
||||
+
|
||||
[source,yaml]
|
||||
--------------------------------------------------------
|
||||
node.attr.size: medium
|
||||
--------------------------------------------------------
|
||||
+
|
||||
You can also set custom attributes when you start a node:
|
||||
+
|
||||
[source,sh]
|
||||
------------------------
|
||||
bin/elasticsearch -Enode.attr.rack=rack1 -Enode.attr.size=big <1>
|
||||
------------------------
|
||||
<1> These attribute settings can also be specified in the `elasticsearch.yml` config file.
|
||||
|
||||
These metadata attributes can be used with the
|
||||
`index.routing.allocation.*` settings to allocate an index to a particular
|
||||
group of nodes. For instance, we can move the index `test` to either `big` or
|
||||
`medium` nodes as follows:
|
||||
|
||||
--------------------------------------------------------
|
||||
`./bin/elasticsearch -Enode.attr.size=medium
|
||||
--------------------------------------------------------
|
||||
|
||||
. Add a routing allocation filter to the index. The `index.routing.allocation`
|
||||
settings support three types of filters: `include`, `exclude`, and `require`.
|
||||
For example, to tell {es} to allocate shards from the `test` index to either
|
||||
`big` or `medium` nodes, use `index.routing.allocation.include`:
|
||||
+
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT test/_settings
|
||||
|
@ -33,24 +58,11 @@ PUT test/_settings
|
|||
------------------------
|
||||
// CONSOLE
|
||||
// TEST[s/^/PUT test\n/]
|
||||
|
||||
Alternatively, we can move the index `test` away from the `small` nodes with
|
||||
an `exclude` rule:
|
||||
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT test/_settings
|
||||
{
|
||||
"index.routing.allocation.exclude.size": "small"
|
||||
}
|
||||
------------------------
|
||||
// CONSOLE
|
||||
// TEST[s/^/PUT test\n/]
|
||||
|
||||
Multiple rules can be specified, in which case all conditions must be
|
||||
satisfied. For instance, we could move the index `test` to `big` nodes in
|
||||
`rack1` with the following:
|
||||
|
||||
+
|
||||
If you specify multiple filters, all conditions must be satisfied for shards to
|
||||
be relocated. For example, to move the `test` index to `big` nodes in `rack1`,
|
||||
you could specify:
|
||||
+
|
||||
[source,js]
|
||||
------------------------
|
||||
PUT test/_settings
|
||||
|
@ -62,10 +74,9 @@ PUT test/_settings
|
|||
// CONSOLE
|
||||
// TEST[s/^/PUT test\n/]
|
||||
|
||||
NOTE: If some conditions cannot be satisfied then shards will not be moved.
|
||||
|
||||
The following settings are _dynamic_, allowing live indices to be moved from
|
||||
one set of nodes to another:
|
||||
[float]
|
||||
[[index-allocation-settings]]
|
||||
==== Index allocation filter settings
|
||||
|
||||
`index.routing.allocation.include.{attribute}`::
|
||||
|
||||
|
@ -82,7 +93,7 @@ one set of nodes to another:
|
|||
Assign the index to a node whose `{attribute}` has _none_ of the
|
||||
comma-separated values.
|
||||
|
||||
These special attributes are also supported:
|
||||
The index allocation settings support the following built-in attributes:
|
||||
|
||||
[horizontal]
|
||||
`_name`:: Match nodes by node name
|
||||
|
@ -91,7 +102,7 @@ These special attributes are also supported:
|
|||
`_ip`:: Match either `_host_ip` or `_publish_ip`
|
||||
`_host`:: Match nodes by hostname
|
||||
|
||||
All attribute values can be specified with wildcards, eg:
|
||||
You can use wildcards when specifying attribute values, for example:
|
||||
|
||||
[source,js]
|
||||
------------------------
|
||||
|
|
|
@ -1,5 +1,5 @@
|
|||
[[allocation-total-shards]]
|
||||
=== Total Shards Per Node
|
||||
=== Total shards per node
|
||||
|
||||
The cluster-level shard allocator tries to spread the shards of a single index
|
||||
across as many nodes as possible. However, depending on how many shards and
|
||||
|
@ -28,6 +28,3 @@ allocated.
|
|||
|
||||
Use with caution.
|
||||
=======================================
|
||||
|
||||
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue