[DOCS] Reworked the shard allocation filtering info. (#36456)

* [DOCS] Reworked the shard allocation filtering info. Closes #36079

* Added multiple index allocation settings example back.

* Removed extraneous space
This commit is contained in:
debadair 2018-12-11 07:44:57 -08:00 committed by GitHub
parent c3a6d1998a
commit c9e03e6ead
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
6 changed files with 177 additions and 171 deletions

View file

@ -1,29 +1,54 @@
[[shard-allocation-filtering]]
=== Shard Allocation Filtering
=== Index-level shard allocation filtering
Shard allocation filtering allows you to specify which nodes are allowed
to host the shards of a particular index.
You can use shard allocation filters to control where {es} allocates shards of
a particular index. These per-index filters are applied in conjunction with
<<allocation-filtering, cluster-wide allocation filtering>> and
<<allocation-awareness, allocation awareness>>.
NOTE: The per-index shard allocation filters explained below work in
conjunction with the cluster-wide allocation filters explained in
<<shards-allocation>>.
Shard allocation filters can be based on custom node attributes or the built-in
`_name`, `host_ip`, `publish_ip`, `_ip`, and `_host` attributes.
<<index-lifecycle-management, Index lifecycle management>> uses filters based
on custom node attributes to determine how to reallocate shards when moving
between phases.
It is possible to assign arbitrary metadata attributes to each node at
startup. For instance, nodes could be assigned a `rack` and a `size`
attribute as follows:
The `cluster.routing.allocation` settings are dynamic, enabling live indices to
be moved from one set of nodes to another. Shards are only relocated if it is
possible to do so without breaking another routing constraint, such as never
allocating a primary and replica shard on the same node.
For example, you could use a custom node attribute to indicate a node's
performance characteristics and use shard allocation filtering to route shards
for a particular index to the most appropriate class of hardware.
[float]
[[index-allocation-filters]]
==== Enabling index-level shard allocation filtering
To filter based on a custom node attribute:
. Specify the filter characteristics with a custom node attribute in each
node's `elasticsearch.yml` configuration file. For example, if you have `small`,
`medium`, and `big` nodes, you could add a `size` attribute to filter based
on node size.
+
[source,yaml]
--------------------------------------------------------
node.attr.size: medium
--------------------------------------------------------
+
You can also set custom attributes when you start a node:
+
[source,sh]
------------------------
bin/elasticsearch -Enode.attr.rack=rack1 -Enode.attr.size=big <1>
------------------------
<1> These attribute settings can also be specified in the `elasticsearch.yml` config file.
These metadata attributes can be used with the
`index.routing.allocation.*` settings to allocate an index to a particular
group of nodes. For instance, we can move the index `test` to either `big` or
`medium` nodes as follows:
--------------------------------------------------------
`./bin/elasticsearch -Enode.attr.size=medium
--------------------------------------------------------
. Add a routing allocation filter to the index. The `index.routing.allocation`
settings support three types of filters: `include`, `exclude`, and `require`.
For example, to tell {es} to allocate shards from the `test` index to either
`big` or `medium` nodes, use `index.routing.allocation.include`:
+
[source,js]
------------------------
PUT test/_settings
@ -33,24 +58,11 @@ PUT test/_settings
------------------------
// CONSOLE
// TEST[s/^/PUT test\n/]
Alternatively, we can move the index `test` away from the `small` nodes with
an `exclude` rule:
[source,js]
------------------------
PUT test/_settings
{
"index.routing.allocation.exclude.size": "small"
}
------------------------
// CONSOLE
// TEST[s/^/PUT test\n/]
Multiple rules can be specified, in which case all conditions must be
satisfied. For instance, we could move the index `test` to `big` nodes in
`rack1` with the following:
+
If you specify multiple filters, all conditions must be satisfied for shards to
be relocated. For example, to move the `test` index to `big` nodes in `rack1`,
you could specify:
+
[source,js]
------------------------
PUT test/_settings
@ -62,10 +74,9 @@ PUT test/_settings
// CONSOLE
// TEST[s/^/PUT test\n/]
NOTE: If some conditions cannot be satisfied then shards will not be moved.
The following settings are _dynamic_, allowing live indices to be moved from
one set of nodes to another:
[float]
[[index-allocation-settings]]
==== Index allocation filter settings
`index.routing.allocation.include.{attribute}`::
@ -82,7 +93,7 @@ one set of nodes to another:
Assign the index to a node whose `{attribute}` has _none_ of the
comma-separated values.
These special attributes are also supported:
The index allocation settings support the following built-in attributes:
[horizontal]
`_name`:: Match nodes by node name
@ -91,7 +102,7 @@ These special attributes are also supported:
`_ip`:: Match either `_host_ip` or `_publish_ip`
`_host`:: Match nodes by hostname
All attribute values can be specified with wildcards, eg:
You can use wildcards when specifying attribute values, for example:
[source,js]
------------------------

View file

@ -1,5 +1,5 @@
[[allocation-total-shards]]
=== Total Shards Per Node
=== Total shards per node
The cluster-level shard allocator tries to spread the shards of a single index
across as many nodes as possible. However, depending on how many shards and
@ -28,6 +28,3 @@ allocated.
Use with caution.
=======================================