diff --git a/docs/plugins/discovery-ec2.asciidoc b/docs/plugins/discovery-ec2.asciidoc index 44acba4752aa..164e3398d7a4 100644 --- a/docs/plugins/discovery-ec2.asciidoc +++ b/docs/plugins/discovery-ec2.asciidoc @@ -241,7 +241,7 @@ The `discovery-ec2` plugin can automatically set the `aws_availability_zone` node attribute to the availability zone of each node. This node attribute allows you to ensure that each shard has copies allocated redundantly across multiple availability zones by using the -{ref}/modules-cluster.html#shard-allocation-awareness[Allocation Awareness] +{ref}/shard-allocation-awareness.html#[Allocation Awareness] feature. In order to enable the automatic definition of the `aws_availability_zone` @@ -333,7 +333,7 @@ labelled as `Moderate` or `Low`. * It is a good idea to distribute your nodes across multiple https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html[availability -zones] and use {ref}/modules-cluster.html#shard-allocation-awareness[shard +zones] and use {ref}/shard-allocation-awareness.html[shard allocation awareness] to ensure that each shard has copies in more than one availability zone. diff --git a/docs/reference/cat/nodeattrs.asciidoc b/docs/reference/cat/nodeattrs.asciidoc index 71a63442df52..6c8093846030 100644 --- a/docs/reference/cat/nodeattrs.asciidoc +++ b/docs/reference/cat/nodeattrs.asciidoc @@ -17,7 +17,7 @@ console. They are _not_ intended for use by applications. For application consumption, use the <>. ==== -Returns information about custom node attributes. +Returns information about <>. [[cat-nodeattrs-api-request]] ==== {api-request-title} diff --git a/docs/reference/cluster.asciidoc b/docs/reference/cluster.asciidoc index b4359c2bf4fb..398ece616fe0 100644 --- a/docs/reference/cluster.asciidoc +++ b/docs/reference/cluster.asciidoc @@ -35,7 +35,7 @@ one of the following: master-eligible nodes, all data nodes, all ingest nodes, all voting-only nodes, all machine learning nodes, and all coordinating-only nodes. * a pair of patterns, using `*` wildcards, of the form `attrname:attrvalue`, - which adds to the subset all nodes with a custom node attribute whose name + which adds to the subset all nodes with a <> whose name and value match the respective patterns. Custom node attributes are configured by setting properties in the configuration file of the form `node.attr.attrname: attrvalue`. diff --git a/docs/reference/commands/node-tool.asciidoc b/docs/reference/commands/node-tool.asciidoc index cdd2bb8f0f9d..265006aa3df1 100644 --- a/docs/reference/commands/node-tool.asciidoc +++ b/docs/reference/commands/node-tool.asciidoc @@ -23,8 +23,8 @@ bin/elasticsearch-node repurpose|unsafe-bootstrap|detach-cluster|override-versio This tool has a number of modes: * `elasticsearch-node repurpose` can be used to delete unwanted data from a - node if it used to be a <> or a - <> but has been repurposed not to have one + node if it used to be a <> or a + <> but has been repurposed not to have one or other of these roles. * `elasticsearch-node remove-settings` can be used to remove persistent settings diff --git a/docs/reference/data-management.asciidoc b/docs/reference/data-management.asciidoc index 7ef021dc6370..849a14e00e69 100644 --- a/docs/reference/data-management.asciidoc +++ b/docs/reference/data-management.asciidoc @@ -43,7 +43,7 @@ Data older than this period can be deleted by {es} at a later time. **Elastic Curator** is a tool that allows you to manage your indices and snapshots using user-defined filters and predefined actions. If ILM provides the functionality to manage your index lifecycle, and you have at least a Basic license, consider using ILM in place of Curator. Many stack components make use of ILM by default. {curator-ref-current}/ilm.html[Learn more]. -NOTE: <> is a deprecated Elasticsearch feature that allows you to manage the amount of data that is stored in your cluster, similar to the downsampling functionality of {ilm-init} and data stream lifecycle. This feature should not be used for new deployments. +NOTE: <> is a deprecated {es} feature that allows you to manage the amount of data that is stored in your cluster, similar to the downsampling functionality of {ilm-init} and data stream lifecycle. This feature should not be used for new deployments. [TIP] ==== diff --git a/docs/reference/data-management/migrate-index-allocation-filters.asciidoc b/docs/reference/data-management/migrate-index-allocation-filters.asciidoc index 85d42e4105a9..ee7d5640d53d 100644 --- a/docs/reference/data-management/migrate-index-allocation-filters.asciidoc +++ b/docs/reference/data-management/migrate-index-allocation-filters.asciidoc @@ -2,7 +2,7 @@ [[migrate-index-allocation-filters]] == Migrate index allocation filters to node roles -If you currently use custom node attributes and +If you currently use <> and <> to move indices through <> in a https://www.elastic.co/blog/implementing-hot-warm-cold-in-elasticsearch-with-index-lifecycle-management[hot-warm-cold architecture], diff --git a/docs/reference/data-store-architecture.asciidoc b/docs/reference/data-store-architecture.asciidoc index 4ee75c15562e..a0d504eb117c 100644 --- a/docs/reference/data-store-architecture.asciidoc +++ b/docs/reference/data-store-architecture.asciidoc @@ -9,10 +9,16 @@ from any node. The topics in this section provides information about the architecture of {es} and how it stores and retrieves data: * <>: Learn about the basic building blocks of an {es} cluster, including nodes, shards, primaries, and replicas. +* <>: Learn about the different roles that nodes can have in an {es} cluster. * <>: Learn how {es} replicates read and write operations across shards and shard copies. * <>: Learn how {es} allocates and balances shards across nodes. +** <>: Learn how to use custom node attributes to distribute shards across different racks or availability zones. +* <>: Learn how {es} caches search requests to improve performance. -- include::nodes-shards.asciidoc[] +include::node-roles.asciidoc[] include::docs/data-replication.asciidoc[leveloffset=-1] -include::modules/shard-ops.asciidoc[] \ No newline at end of file +include::modules/shard-ops.asciidoc[] +include::modules/cluster/allocation_awareness.asciidoc[leveloffset=+1] +include::shard-request-cache.asciidoc[leveloffset=-1] diff --git a/docs/reference/data-streams/downsampling.asciidoc b/docs/reference/data-streams/downsampling.asciidoc index 0b08b0972f9a..10a0241cf073 100644 --- a/docs/reference/data-streams/downsampling.asciidoc +++ b/docs/reference/data-streams/downsampling.asciidoc @@ -72,6 +72,45 @@ the granularity of `cold` archival data to monthly or less. .Downsampled metrics series image::images/data-streams/time-series-downsampled.png[align="center"] +[discrete] +[[downsample-api-process]] +==== The downsampling process + +The downsampling operation traverses the source TSDS index and performs the +following steps: + +. Creates a new document for each value of the `_tsid` field and each +`@timestamp` value, rounded to the `fixed_interval` defined in the downsample +configuration. +. For each new document, copies all <> from the source index to the target index. Dimensions in a +TSDS are constant, so this is done only once per bucket. +. For each <> field, computes aggregations +for all documents in the bucket. Depending on the metric type of each metric +field a different set of pre-aggregated results is stored: + +** `gauge`: The `min`, `max`, `sum`, and `value_count` are stored; `value_count` +is stored as type `aggregate_metric_double`. +** `counter`: The `last_value` is stored. +. For all other fields, the most recent value is copied to the target index. + +[discrete] +[[downsample-api-mappings]] +==== Source and target index field mappings + +Fields in the target, downsampled index are created based on fields in the +original source index, as follows: + +. All fields mapped with the `time-series-dimension` parameter are created in +the target downsample index with the same mapping as in the source index. +. All fields mapped with the `time_series_metric` parameter are created +in the target downsample index with the same mapping as in the source +index. An exception is that for fields mapped as `time_series_metric: gauge` +the field type is changed to `aggregate_metric_double`. +. All other fields that are neither dimensions nor metrics (that is, label +fields), are created in the target downsample index with the same mapping +that they had in the source index. + [discrete] [[running-downsampling]] === Running downsampling on time series data diff --git a/docs/reference/datatiers.asciidoc b/docs/reference/datatiers.asciidoc index c37f54b5c9ca..d7d52e859504 100644 --- a/docs/reference/datatiers.asciidoc +++ b/docs/reference/datatiers.asciidoc @@ -190,7 +190,7 @@ tier]. [[configure-data-tiers-on-premise]] ==== Self-managed deployments -For self-managed deployments, each node's <> is configured +For self-managed deployments, each node's <> is configured in `elasticsearch.yml`. For example, the highest-performance nodes in a cluster might be assigned to both the hot and content tiers: diff --git a/docs/reference/high-availability/cluster-design.asciidoc b/docs/reference/high-availability/cluster-design.asciidoc index 105c8b236b0b..d187db83c43f 100644 --- a/docs/reference/high-availability/cluster-design.asciidoc +++ b/docs/reference/high-availability/cluster-design.asciidoc @@ -87,7 +87,7 @@ the same thing, but it's not necessary to use this feature in such a small cluster. We recommend you set only one of your two nodes to be -<>. This means you can be certain which of your +<>. This means you can be certain which of your nodes is the elected master of the cluster. The cluster can tolerate the loss of the other master-ineligible node. If you set both nodes to master-eligible, two nodes are required for a master election. Since the election will fail if either @@ -164,12 +164,12 @@ cluster that is suitable for production deployments. [[high-availability-cluster-design-three-nodes]] ==== Three-node clusters -If you have three nodes, we recommend they all be <> and +If you have three nodes, we recommend they all be <> and every index that is not a <> should have at least one replica. Nodes are data nodes by default. You may prefer for some indices to have two replicas so that each node has a copy of each shard in those indices. You should also configure each node to be -<> so that any two of them can hold a master +<> so that any two of them can hold a master election without needing to communicate with the third node. Nodes are master-eligible by default. This cluster will be resilient to the loss of any single node. @@ -188,8 +188,8 @@ service provides such a load balancer. Once your cluster grows to more than three nodes, you can start to specialise these nodes according to their responsibilities, allowing you to scale their -resources independently as needed. You can have as many <>, <>, <>, etc. as needed to +resources independently as needed. You can have as many <>, <>, <>, etc. as needed to support your workload. As your cluster grows larger, we recommend using dedicated nodes for each role. This allows you to independently scale resources for each task. diff --git a/docs/reference/ilm/apis/migrate-to-data-tiers.asciidoc b/docs/reference/ilm/apis/migrate-to-data-tiers.asciidoc index 76810170daa1..bbcdd71c45f1 100644 --- a/docs/reference/ilm/apis/migrate-to-data-tiers.asciidoc +++ b/docs/reference/ilm/apis/migrate-to-data-tiers.asciidoc @@ -11,7 +11,7 @@ For the most up-to-date API details, refer to {api-es}/group/endpoint-ilm[{ilm-cap} APIs]. -- -Switches the indices, ILM policies, and legacy, composable and component templates from using custom node attributes and +Switches the indices, ILM policies, and legacy, composable and component templates from using <> and <> to using <>, and optionally deletes one legacy index template. Using node roles enables {ilm-init} to <> between diff --git a/docs/reference/index-modules/allocation/data_tier_allocation.asciidoc b/docs/reference/index-modules/allocation/data_tier_allocation.asciidoc index d08af2100762..2d59e9be31cd 100644 --- a/docs/reference/index-modules/allocation/data_tier_allocation.asciidoc +++ b/docs/reference/index-modules/allocation/data_tier_allocation.asciidoc @@ -13,7 +13,7 @@ This setting corresponds to the data node roles: * <> * <> -NOTE: The <> role is not a valid data tier and cannot be used +NOTE: The <> role is not a valid data tier and cannot be used with the `_tier_preference` setting. The frozen tier stores <> exclusively. diff --git a/docs/reference/index-modules/allocation/filtering.asciidoc b/docs/reference/index-modules/allocation/filtering.asciidoc index 07a2455ca1ef..5da8e254cb4f 100644 --- a/docs/reference/index-modules/allocation/filtering.asciidoc +++ b/docs/reference/index-modules/allocation/filtering.asciidoc @@ -6,7 +6,7 @@ a particular index. These per-index filters are applied in conjunction with <> and <>. -Shard allocation filters can be based on custom node attributes or the built-in +Shard allocation filters can be based on <> or the built-in `_name`, `_host_ip`, `_publish_ip`, `_ip`, `_host`, `_id`, `_tier` and `_tier_preference` attributes. <> uses filters based on custom node attributes to determine how to reallocate shards when moving @@ -114,7 +114,7 @@ The index allocation settings support the following built-in attributes: NOTE: `_tier` filtering is based on <> roles. Only a subset of roles are <> roles, and the generic -<> will match any tier filtering. +<> will match any tier filtering. You can use wildcards when specifying attribute values, for example: diff --git a/docs/reference/indices/downsample-data-stream.asciidoc b/docs/reference/indices/downsample-data-stream.asciidoc index a99d7b246ba4..6354f8e30d25 100644 --- a/docs/reference/indices/downsample-data-stream.asciidoc +++ b/docs/reference/indices/downsample-data-stream.asciidoc @@ -81,6 +81,8 @@ DELETE _index_template/* //// // end::downsample-example[] +Check the <> documentation for an overview, details about the downsampling process, and examples of running downsampling manually and as part of an ILM policy. + [[downsample-api-request]] ==== {api-request-title} @@ -121,44 +123,4 @@ to aggregate the original time series index. For example, `60m` produces a document for each 60 minute (hourly) interval. This follows standard time formatting syntax as used elsewhere in {es}. + -NOTE: Smaller, more granular intervals take up proportionally more space. - -[[downsample-api-process]] -==== The downsampling process - -The downsampling operation traverses the source TSDS index and performs the -following steps: - -. Creates a new document for each value of the `_tsid` field and each -`@timestamp` value, rounded to the `fixed_interval` defined in the downsample -configuration. -. For each new document, copies all <> from the source index to the target index. Dimensions in a -TSDS are constant, so this is done only once per bucket. -. For each <> field, computes aggregations -for all documents in the bucket. Depending on the metric type of each metric -field a different set of pre-aggregated results is stored: - -** `gauge`: The `min`, `max`, `sum`, and `value_count` are stored; `value_count` -is stored as type `aggregate_metric_double`. -** `counter`: The `last_value` is stored. -. For all other fields, the most recent value is copied to the target index. - -[[downsample-api-mappings]] -==== Source and target index field mappings - -Fields in the target, downsampled index are created based on fields in the -original source index, as follows: - -. All fields mapped with the `time-series-dimension` parameter are created in -the target downsample index with the same mapping as in the source index. -. All fields mapped with the `time_series_metric` parameter are created -in the target downsample index with the same mapping as in the source -index. An exception is that for fields mapped as `time_series_metric: gauge` -the field type is changed to `aggregate_metric_double`. -. All other fields that are neither dimensions nor metrics (that is, label -fields), are created in the target downsample index with the same mapping -that they had in the source index. - -Check the <> documentation for an overview and -examples of running downsampling manually and as part of an ILM policy. +NOTE: Smaller, more granular intervals take up proportionally more space. \ No newline at end of file diff --git a/docs/reference/modules/cluster.asciidoc b/docs/reference/modules/cluster.asciidoc index b3eaa5b47c23..cf8e97de5e18 100644 --- a/docs/reference/modules/cluster.asciidoc +++ b/docs/reference/modules/cluster.asciidoc @@ -27,7 +27,23 @@ include::cluster/shards_allocation.asciidoc[] include::cluster/disk_allocator.asciidoc[] -include::cluster/allocation_awareness.asciidoc[] +[[shard-allocation-awareness-settings]] +==== Shard allocation awareness settings + +You can use <> as _awareness attributes_ to enable {es} +to take your physical hardware configuration into account when allocating shards. +If {es} knows which nodes are on the same physical server, in the same rack, or +in the same zone, it can distribute the primary shard and its replica shards to +minimize the risk of losing all shard copies in the event of a failure. <>. + +`cluster.routing.allocation.awareness.attributes`:: + (<>) + The node attributes that {es} should use as awareness attributes. For example, if you have a `rack_id` attribute that specifies the rack in which each node resides, you can set this setting to `rack_id` to ensure that primary and replica shards are not allocated on the same rack. You can specify multiple attributes as a comma-separated list. + +`cluster.routing.allocation.awareness.force.*`:: + (<>) + The shard allocation awareness values that must exist for shards to be reallocated in case of location failure. Learn more about <>. + include::cluster/allocation_filtering.asciidoc[] diff --git a/docs/reference/modules/cluster/allocation_awareness.asciidoc b/docs/reference/modules/cluster/allocation_awareness.asciidoc index 9c6197f9ba40..34164cd364fc 100644 --- a/docs/reference/modules/cluster/allocation_awareness.asciidoc +++ b/docs/reference/modules/cluster/allocation_awareness.asciidoc @@ -1,5 +1,5 @@ [[shard-allocation-awareness]] -==== Shard allocation awareness +== Shard allocation awareness You can use custom node attributes as _awareness attributes_ to enable {es} to take your physical hardware configuration into account when allocating shards. @@ -7,12 +7,7 @@ If {es} knows which nodes are on the same physical server, in the same rack, or in the same zone, it can distribute the primary shard and its replica shards to minimize the risk of losing all shard copies in the event of a failure. -When shard allocation awareness is enabled with the -<> -`cluster.routing.allocation.awareness.attributes` setting, shards are only -allocated to nodes that have values set for the specified awareness attributes. -If you use multiple awareness attributes, {es} considers each attribute -separately when allocating shards. +When shard allocation awareness is enabled with the `cluster.routing.allocation.awareness.attributes` setting, shards are only allocated to nodes that have values set for the specified awareness attributes. If you use multiple awareness attributes, {es} considers each attribute separately when allocating shards. NOTE: The number of attribute values determines how many shard copies are allocated in each location. If the number of nodes in each location is @@ -22,11 +17,11 @@ unassigned. TIP: Learn more about <>. [[enabling-awareness]] -===== Enabling shard allocation awareness +=== Enabling shard allocation awareness To enable shard allocation awareness: -. Specify the location of each node with a custom node attribute. For example, +. Specify the location of each node with a <>. For example, if you want Elasticsearch to distribute shards across different racks, you might use an awareness attribute called `rack_id`. + @@ -94,7 +89,7 @@ copies of a particular shard from being allocated in the same location, you can enable forced awareness. [[forced-awareness]] -===== Forced awareness +=== Forced awareness By default, if one location fails, {es} spreads its shards across the remaining locations. This might be undesirable if the cluster does not have sufficient diff --git a/docs/reference/modules/cluster/allocation_filtering.asciidoc b/docs/reference/modules/cluster/allocation_filtering.asciidoc index e70e43682973..d0d265205904 100644 --- a/docs/reference/modules/cluster/allocation_filtering.asciidoc +++ b/docs/reference/modules/cluster/allocation_filtering.asciidoc @@ -6,7 +6,7 @@ allocates shards from any index. These cluster wide filters are applied in conjunction with <> and <>. -Shard allocation filters can be based on custom node attributes or the built-in +Shard allocation filters can be based on <> or the built-in `_name`, `_host_ip`, `_publish_ip`, `_ip`, `_host`, `_id` and `_tier` attributes. The `cluster.routing.allocation` settings are <>, enabling live indices to @@ -59,9 +59,9 @@ The cluster allocation settings support the following built-in attributes: NOTE: `_tier` filtering is based on <> roles. Only a subset of roles are <> roles, and the generic -<> will match any tier filtering. +<> will match any tier filtering. a subset of roles that are <> roles, but the generic -<> will match any tier filtering. +<> will match any tier filtering. You can use wildcards when specifying attribute values, for example: diff --git a/docs/reference/modules/cluster/disk_allocator.asciidoc b/docs/reference/modules/cluster/disk_allocator.asciidoc index 02cc48c6e27f..8efe4c0132e8 100644 --- a/docs/reference/modules/cluster/disk_allocator.asciidoc +++ b/docs/reference/modules/cluster/disk_allocator.asciidoc @@ -41,6 +41,23 @@ on the affected node drops below the high watermark, {es} automatically removes the write block. Refer to <> to resolve persistent watermark errors. +[NOTE] +.Max headroom settings +=================================================== + +Max headroom settings apply only when watermark settings are percentages or ratios. + +A max headroom value is intended to cap the required free disk space before hitting +the respective watermark. This is useful for servers with larger disks, where a percentage or ratio watermark could translate to an overly large free disk space requirement. In this case, the max headroom can be used to cap the required free disk space amount. + +For example, where `cluster.routing.allocation.disk.watermark.flood_stage` is 95% and `cluster.routing.allocation.disk.watermark.flood_stage.max_headroom` is 100GB, this means that: + +* For a smaller disk, e.g., of 100GB, the flood watermark will hit at 95%, meaning at 5GB of free space, since 5GB is smaller than the 100GB max headroom value. +* For a larger disk, e.g., of 100TB, the flood watermark will hit at 100GB of free space. That is because the 95% flood watermark alone would require 5TB of free disk space, but is capped by the max headroom setting to 100GB. + +Max headroom settings have their default values only if their respective watermark settings are not explicitly set. If watermarks are explicitly set, then the max headroom settings do not have their default values, and need to be explicitly set if they are needed. +=================================================== + [[disk-based-shard-allocation-does-not-balance]] [TIP] ==== @@ -100,18 +117,7 @@ is now `true`. The setting will be removed in a future release. + -- (<>) -Controls the flood stage watermark, which defaults to 95%. {es} enforces a read-only index block (`index.blocks.read_only_allow_delete`) on every index that has one or more shards allocated on the node, and that has at least one disk exceeding the flood stage. This setting is a last resort to prevent nodes from running out of disk space. The index block is automatically released when the disk utilization falls below the high watermark. Similarly to the low and high watermark values, it can alternatively be set to a ratio value, e.g., `0.95`, or an absolute byte value. - -An example of resetting the read-only index block on the `my-index-000001` index: - -[source,console] --------------------------------------------------- -PUT /my-index-000001/_settings -{ - "index.blocks.read_only_allow_delete": null -} --------------------------------------------------- -// TEST[setup:my_index] +Controls the flood stage watermark, which defaults to 95%. {es} enforces a read-only index block (<>) on every index that has one or more shards allocated on the node, and that has at least one disk exceeding the flood stage. This setting is a last resort to prevent nodes from running out of disk space. The index block is automatically released when the disk utilization falls below the high watermark. Similarly to the low and high watermark values, it can alternatively be set to a ratio value, e.g., `0.95`, or an absolute byte value. -- // end::cluster-routing-flood-stage-tag[] @@ -121,10 +127,10 @@ Defaults to 100GB when `cluster.routing.allocation.disk.watermark.flood_stage` is not explicitly set. This caps the amount of free space required. -NOTE: You cannot mix the usage of percentage/ratio values and byte values across +NOTE: You can't mix the usage of percentage/ratio values and byte values across the `cluster.routing.allocation.disk.watermark.low`, `cluster.routing.allocation.disk.watermark.high`, and `cluster.routing.allocation.disk.watermark.flood_stage` settings. Either all values -are set to percentage/ratio values, or all are set to byte values. This enforcement is +must be set to percentage/ratio values, or all must be set to byte values. This is required so that {es} can validate that the settings are internally consistent, ensuring that the low disk threshold is less than the high disk threshold, and the high disk threshold is less than the flood stage threshold. A similar comparison check is done for the max @@ -150,44 +156,6 @@ set. This caps the amount of free space required on dedicated frozen nodes. cluster. Defaults to `30s`. NOTE: Percentage values refer to used disk space, while byte values refer to -free disk space. This can be confusing, since it flips the meaning of high and +free disk space. This can be confusing, because it flips the meaning of high and low. For example, it makes sense to set the low watermark to 10gb and the high -watermark to 5gb, but not the other way around. - -An example of updating the low watermark to at least 100 gigabytes free, a high -watermark of at least 50 gigabytes free, and a flood stage watermark of 10 -gigabytes free, and updating the information about the cluster every minute: - -[source,console] --------------------------------------------------- -PUT _cluster/settings -{ - "persistent": { - "cluster.routing.allocation.disk.watermark.low": "100gb", - "cluster.routing.allocation.disk.watermark.high": "50gb", - "cluster.routing.allocation.disk.watermark.flood_stage": "10gb", - "cluster.info.update.interval": "1m" - } -} --------------------------------------------------- - -Concerning the max headroom settings for the watermarks, please note -that these apply only in the case that the watermark settings are percentages/ratios. -The aim of a max headroom value is to cap the required free disk space before hitting -the respective watermark. This is especially useful for servers with larger -disks, where a percentage/ratio watermark could translate to a big free disk space requirement, -and the max headroom can be used to cap the required free disk space amount. -As an example, let us take the default settings for the flood watermark. -It has a 95% default value, and the flood max headroom setting has a default value of 100GB. -This means that: - -* For a smaller disk, e.g., of 100GB, the flood watermark will hit at 95%, meaning at 5GB -of free space, since 5GB is smaller than the 100GB max headroom value. -* For a larger disk, e.g., of 100TB, the flood watermark will hit at 100GB of free space. -That is because the 95% flood watermark alone would require 5TB of free disk space, but -that is capped by the max headroom setting to 100GB. - -Finally, the max headroom settings have their default values only if their respective watermark -settings are not explicitly set (thus, they have their default percentage values). -If watermarks are explicitly set, then the max headroom settings do not have their default values, -and would need to be explicitly set if they are desired. +watermark to 5gb, but not the other way around. \ No newline at end of file diff --git a/docs/reference/modules/cluster/misc.asciidoc b/docs/reference/modules/cluster/misc.asciidoc index 75eaca88c66b..b66ac1fdb0cc 100644 --- a/docs/reference/modules/cluster/misc.asciidoc +++ b/docs/reference/modules/cluster/misc.asciidoc @@ -1,6 +1,9 @@ [[misc-cluster-settings]] === Miscellaneous cluster settings +[[cluster-name]] +include::{es-ref-dir}/setup/important-settings/cluster-name.asciidoc[] + [discrete] [[cluster-read-only]] ==== Metadata diff --git a/docs/reference/modules/discovery/bootstrapping.asciidoc b/docs/reference/modules/discovery/bootstrapping.asciidoc index 5120c1d17e69..a885f1633ea4 100644 --- a/docs/reference/modules/discovery/bootstrapping.asciidoc +++ b/docs/reference/modules/discovery/bootstrapping.asciidoc @@ -2,7 +2,7 @@ === Bootstrapping a cluster Starting an Elasticsearch cluster for the very first time requires the initial -set of <> to be explicitly defined on one or +set of <> to be explicitly defined on one or more of the master-eligible nodes in the cluster. This is known as _cluster bootstrapping_. This is only required the first time a cluster starts up. Freshly-started nodes that are joining a running cluster obtain this diff --git a/docs/reference/modules/discovery/publishing.asciidoc b/docs/reference/modules/discovery/publishing.asciidoc index af664585085c..2bb031e6a986 100644 --- a/docs/reference/modules/discovery/publishing.asciidoc +++ b/docs/reference/modules/discovery/publishing.asciidoc @@ -1,5 +1,23 @@ +[[cluster-state-overview]] +=== Cluster state + +The _cluster state_ is an internal data structure which keeps track of a +variety of information needed by every node, including: + +* The identity and attributes of the other nodes in the cluster + +* Cluster-wide settings + +* Index metadata, including the mapping and settings for each index + +* The location and status of every shard copy in the cluster + +The elected master node ensures that every node in the cluster has a copy of +the same cluster state. The <> lets you retrieve a +representation of this internal state for debugging or diagnostic purposes. + [[cluster-state-publishing]] -=== Publishing the cluster state +==== Publishing the cluster state The elected master node is the only node in a cluster that can make changes to the cluster state. The elected master node processes one batch of cluster state @@ -58,3 +76,16 @@ speed of the storage on each master-eligible node, as well as the reliability and latency of the network interconnections between all nodes in the cluster. You must therefore ensure that the storage and networking available to the nodes in your cluster are good enough to meet your performance goals. + +[[dangling-index]] +==== Dangling indices + +When a node joins the cluster, if it finds any shards stored in its local +data directory that do not already exist in the cluster state, it will consider +those shards to belong to a "dangling" index. You can list, import or +delete dangling indices using the <>. + +NOTE: The API cannot offer any guarantees as to whether the imported data +truly represents the latest state of the data when the index was still part +of the cluster. \ No newline at end of file diff --git a/docs/reference/modules/discovery/voting.asciidoc b/docs/reference/modules/discovery/voting.asciidoc index 04cae9d02ab6..f4bd4756d897 100644 --- a/docs/reference/modules/discovery/voting.asciidoc +++ b/docs/reference/modules/discovery/voting.asciidoc @@ -2,7 +2,7 @@ === Voting configurations Each {es} cluster has a _voting configuration_, which is the set of -<> whose responses are counted when making +<> whose responses are counted when making decisions such as electing a new master or committing a new cluster state. Decisions are made only after a majority (more than half) of the nodes in the voting configuration respond. diff --git a/docs/reference/modules/gateway.asciidoc b/docs/reference/modules/gateway.asciidoc index bf7e6de64f09..35a5ef6ddec0 100644 --- a/docs/reference/modules/gateway.asciidoc +++ b/docs/reference/modules/gateway.asciidoc @@ -1,10 +1,11 @@ [[modules-gateway]] === Local gateway settings +[[dangling-indices]] The local gateway stores the cluster state and shard data across full cluster restarts. -The following _static_ settings, which must be set on every <>, +The following _static_ settings, which must be set on every <>, control how long a freshly elected master should wait before it tries to recover the <> and the cluster's data. @@ -36,17 +37,4 @@ These settings can be configured in `elasticsearch.yml` as follows: gateway.expected_data_nodes: 3 gateway.recover_after_time: 600s gateway.recover_after_data_nodes: 3 --------------------------------------------------- - -[[dangling-indices]] -==== Dangling indices - -When a node joins the cluster, if it finds any shards stored in its local -data directory that do not already exist in the cluster, it will consider -those shards to belong to a "dangling" index. You can list, import or -delete dangling indices using the <>. - -NOTE: The API cannot offer any guarantees as to whether the imported data -truly represents the latest state of the data when the index was still part -of the cluster. +-------------------------------------------------- \ No newline at end of file diff --git a/docs/reference/modules/indices/fielddata.asciidoc b/docs/reference/modules/indices/fielddata.asciidoc index 1383bf74d6d4..688685c0a224 100644 --- a/docs/reference/modules/indices/fielddata.asciidoc +++ b/docs/reference/modules/indices/fielddata.asciidoc @@ -5,10 +5,6 @@ The field data cache contains <> and <>. This behavior can be configured. @@ -20,16 +16,12 @@ at the cost of rebuilding the cache as needed. If the circuit breaker limit is reached, further requests that increase the cache size will be prevented. In this case you should manually <>. +TIP: You can monitor memory usage for field data as well as the field data circuit +breaker using +the <> or the <>. + `indices.fielddata.cache.size`:: (<>) The max size of the field data cache, eg `38%` of node heap space, or an absolute value, eg `12GB`. Defaults to unbounded. If you choose to set it, -it should be smaller than <> limit. - -[discrete] -[[fielddata-monitoring]] -==== Monitoring field data - -You can monitor memory usage for field data as well as the field data circuit -breaker using -the <> or the <>. +it should be smaller than <> limit. \ No newline at end of file diff --git a/docs/reference/modules/indices/request_cache.asciidoc b/docs/reference/modules/indices/request_cache.asciidoc index 4d4d349c685a..f6ad65245836 100644 --- a/docs/reference/modules/indices/request_cache.asciidoc +++ b/docs/reference/modules/indices/request_cache.asciidoc @@ -1,4 +1,4 @@ -[[shard-request-cache]] +[[shard-request-cache-settings]] === Shard request cache settings When a search request is run against an index or against many indices, each @@ -10,139 +10,16 @@ The shard-level request cache module caches the local results on each shard. This allows frequently used (and potentially heavy) search requests to return results almost instantly. The requests cache is a very good fit for the logging use case, where only the most recent index is being actively updated -- -results from older indices will be served directly from the cache. +results from older indices will be served directly from the cache. You can use shard request cache settings to control the size and expiration of the cache. -[IMPORTANT] -=================================== - -By default, the requests cache will only cache the results of search requests -where `size=0`, so it will not cache `hits`, -but it will cache `hits.total`, <>, and -<>. - -Most queries that use `now` (see <>) cannot be cached. - -Scripted queries that use the API calls which are non-deterministic, such as -`Math.random()` or `new Date()` are not cached. -=================================== - -[discrete] -==== Cache invalidation - -The cache is smart -- it keeps the same _near real-time_ promise as uncached -search. - -Cached results are invalidated automatically whenever the shard refreshes to -pick up changes to the documents or when you update the mapping. In other -words you will always get the same results from the cache as you would for an -uncached search request. - -The longer the refresh interval, the longer that cached entries will remain -valid even if there are changes to the documents. If the cache is full, the -least recently used cache keys will be evicted. - -The cache can be expired manually with the <>: - -[source,console] ------------------------- -POST /my-index-000001,my-index-000002/_cache/clear?request=true ------------------------- -// TEST[s/^/PUT my-index-000001\nPUT my-index-000002\n/] - -[discrete] -==== Enabling and disabling caching - -The cache is enabled by default, but can be disabled when creating a new -index as follows: - -[source,console] ------------------------------ -PUT /my-index-000001 -{ - "settings": { - "index.requests.cache.enable": false - } -} ------------------------------ - -It can also be enabled or disabled dynamically on an existing index with the -<> API: - -[source,console] ------------------------------ -PUT /my-index-000001/_settings -{ "index.requests.cache.enable": true } ------------------------------ -// TEST[continued] - - -[discrete] -==== Enabling and disabling caching per request - -The `request_cache` query-string parameter can be used to enable or disable -caching on a *per-request* basis. If set, it overrides the index-level setting: - -[source,console] ------------------------------ -GET /my-index-000001/_search?request_cache=true -{ - "size": 0, - "aggs": { - "popular_colors": { - "terms": { - "field": "colors" - } - } - } -} ------------------------------ -// TEST[continued] - -Requests where `size` is greater than 0 will not be cached even if the request cache is -enabled in the index settings. To cache these requests you will need to use the -query-string parameter detailed here. - -[discrete] -==== Cache key - -A hash of the whole JSON body is used as the cache key. This means that if the JSON -changes -- for instance if keys are output in a different order -- then the -cache key will not be recognised. - -TIP: Most JSON libraries support a _canonical_ mode which ensures that JSON -keys are always emitted in the same order. This canonical mode can be used in -the application to ensure that a request is always serialized in the same way. +To learn more about the shard request cache, see <>. [discrete] ==== Cache settings -The cache is managed at the node level, and has a default maximum size of `1%` -of the heap. This can be changed in the `config/elasticsearch.yml` file with: +`indices.requests.cache.size`:: +(<>) The maximum size of the cache, as a percentage of the heap. Default: `1%`. -[source,yaml] --------------------------------- -indices.requests.cache.size: 2% --------------------------------- +`indices.requests.cache.expire`:: +(<>) The TTL for cached results. Stale results are automatically invalidated when the index is refreshed, so you shouldn't need to use this setting. -Also, you can use the +indices.requests.cache.expire+ setting to specify a TTL -for cached results, but there should be no reason to do so. Remember that -stale results are automatically invalidated when the index is refreshed. This -setting is provided for completeness' sake only. - -[discrete] -==== Monitoring cache usage - -The size of the cache (in bytes) and the number of evictions can be viewed -by index, with the <> API: - -[source,console] ------------------------- -GET /_stats/request_cache?human ------------------------- - -or by node with the <> API: - -[source,console] ------------------------- -GET /_nodes/stats/indices/request_cache?human ------------------------- diff --git a/docs/reference/modules/network.asciidoc b/docs/reference/modules/network.asciidoc index 1e4c5a21d386..2ea4dcb9b18f 100644 --- a/docs/reference/modules/network.asciidoc +++ b/docs/reference/modules/network.asciidoc @@ -286,3 +286,22 @@ include::remote-cluster-network.asciidoc[] include::network/tracers.asciidoc[] include::network/threading.asciidoc[] + +[[readiness-tcp-port]] +==== TCP readiness port + +preview::[] + +If configured, a node can open a TCP port when the node is in a ready state. A node is deemed +ready when it has successfully joined a cluster. In a single node configuration, the node is +said to be ready, when it's able to accept requests. + +To enable the readiness TCP port, use the `readiness.port` setting. The readiness service will bind to +all host addresses. + +If the node leaves the cluster, or the <> is used to mark the node +for shutdown, the readiness port is immediately closed. + +A successful connection to the readiness TCP port signals that the {es} node is ready. When a client +connects to the readiness port, the server simply terminates the socket connection. No data is sent back +to the client. If a client cannot connect to the readiness port, the node is not ready. \ No newline at end of file diff --git a/docs/reference/modules/node.asciidoc b/docs/reference/modules/node.asciidoc index 022e8b5d1e2f..e8dd995623a1 100644 --- a/docs/reference/modules/node.asciidoc +++ b/docs/reference/modules/node.asciidoc @@ -1,5 +1,5 @@ [[modules-node]] -=== Nodes +=== Node settings Any time that you start an instance of {es}, you are starting a _node_. A collection of connected nodes is called a <>. If you @@ -18,24 +18,33 @@ TIP: The performance of an {es} node is often limited by the performance of the Review our recommendations for optimizing your storage for <> and <>. +[[node-name-settings]] +==== Node name setting + +include::{es-ref-dir}/setup/important-settings/node-name.asciidoc[] + [[node-roles]] -==== Node roles +==== Node role settings You define a node's roles by setting `node.roles` in `elasticsearch.yml`. If you set `node.roles`, the node is only assigned the roles you specify. If you don't set `node.roles`, the node is assigned the following roles: -* `master` -* `data` +* [[master-node]]`master` +* [[data-node]]`data` * `data_content` * `data_hot` * `data_warm` * `data_cold` * `data_frozen` * `ingest` -* `ml` +* [[ml-node]]`ml` * `remote_cluster_client` -* `transform` +* [[transform-node]]`transform` + +The following additional roles are available: + +* `voting_only` [IMPORTANT] ==== @@ -65,386 +74,7 @@ As the cluster grows and in particular if you have large {ml} jobs or {ctransforms}, consider separating dedicated master-eligible nodes from dedicated data nodes, {ml} nodes, and {transform} nodes. -<>:: - -A node that has the `master` role, which makes it eligible to be -<>, which controls the cluster. - -<>:: - -A node that has one of several data roles. Data nodes hold data and perform data -related operations such as CRUD, search, and aggregations. A node with a generic `data` role can fill any of the specialized data node roles. - -<>:: - -A node that has the `ingest` role. Ingest nodes are able to apply an -<> to a document in order to transform and enrich the -document before indexing. With a heavy ingest load, it makes sense to use -dedicated ingest nodes and to not include the `ingest` role from nodes that have -the `master` or `data` roles. - -<>:: - -A node that has the `remote_cluster_client` role, which makes it eligible to act -as a remote client. - -<>:: - -A node that has the `ml` role. If you want to use {ml-features}, there must be -at least one {ml} node in your cluster. For more information, see -<> and {ml-docs}/index.html[Machine learning in the {stack}]. - -<>:: - -A node that has the `transform` role. If you want to use {transforms}, there -must be at least one {transform} node in your cluster. For more information, see -<> and <>. - -[NOTE] -[[coordinating-node]] -.Coordinating node -=============================================== - -Requests like search requests or bulk-indexing requests may involve data held -on different data nodes. A search request, for example, is executed in two -phases which are coordinated by the node which receives the client request -- -the _coordinating node_. - -In the _scatter_ phase, the coordinating node forwards the request to the data -nodes which hold the data. Each data node executes the request locally and -returns its results to the coordinating node. In the _gather_ phase, the -coordinating node reduces each data node's results into a single global -result set. - -Every node is implicitly a coordinating node. This means that a node that has -an explicit empty list of roles via `node.roles` will only act as a coordinating -node, which cannot be disabled. As a result, such a node needs to have enough -memory and CPU in order to deal with the gather phase. - -=============================================== - -[[master-node]] -==== Master-eligible node - -The master node is responsible for lightweight cluster-wide actions such as -creating or deleting an index, tracking which nodes are part of the cluster, -and deciding which shards to allocate to which nodes. It is important for -cluster health to have a stable master node. - -Any master-eligible node that is not a <> may -be elected to become the master node by the <>. - -IMPORTANT: Master nodes must have a `path.data` directory whose contents -persist across restarts, just like data nodes, because this is where the -cluster metadata is stored. The cluster metadata describes how to read the data -stored on the data nodes, so if it is lost then the data stored on the data -nodes cannot be read. - -[[dedicated-master-node]] -===== Dedicated master-eligible node - -It is important for the health of the cluster that the elected master node has -the resources it needs to fulfill its responsibilities. If the elected master -node is overloaded with other tasks then the cluster will not operate well. The -most reliable way to avoid overloading the master with other tasks is to -configure all the master-eligible nodes to be _dedicated master-eligible nodes_ -which only have the `master` role, allowing them to focus on managing the -cluster. Master-eligible nodes will still also behave as -<> that route requests from clients to -the other nodes in the cluster, but you should _not_ use dedicated master nodes -for this purpose. - -A small or lightly-loaded cluster may operate well if its master-eligible nodes -have other roles and responsibilities, but once your cluster comprises more -than a handful of nodes it usually makes sense to use dedicated master-eligible -nodes. - -To create a dedicated master-eligible node, set: - -[source,yaml] -------------------- -node.roles: [ master ] -------------------- - -[[voting-only-node]] -===== Voting-only master-eligible node - -A voting-only master-eligible node is a node that participates in -<> but which will not act as the cluster's -elected master node. In particular, a voting-only node can serve as a tiebreaker -in elections. - -It may seem confusing to use the term "master-eligible" to describe a -voting-only node since such a node is not actually eligible to become the master -at all. This terminology is an unfortunate consequence of history: -master-eligible nodes are those nodes that participate in elections and perform -certain tasks during cluster state publications, and voting-only nodes have the -same responsibilities even if they can never become the elected master. - -To configure a master-eligible node as a voting-only node, include `master` and -`voting_only` in the list of roles. For example to create a voting-only data -node: - -[source,yaml] -------------------- -node.roles: [ data, master, voting_only ] -------------------- - -IMPORTANT: Only nodes with the `master` role can be marked as having the -`voting_only` role. - -High availability (HA) clusters require at least three master-eligible nodes, at -least two of which are not voting-only nodes. Such a cluster will be able to -elect a master node even if one of the nodes fails. - -Voting-only master-eligible nodes may also fill other roles in your cluster. -For instance, a node may be both a data node and a voting-only master-eligible -node. A _dedicated_ voting-only master-eligible nodes is a voting-only -master-eligible node that fills no other roles in the cluster. To create a -dedicated voting-only master-eligible node, set: - -[source,yaml] -------------------- -node.roles: [ master, voting_only ] -------------------- - -Since dedicated voting-only nodes never act as the cluster's elected master, -they may require less heap and a less powerful CPU than the true master nodes. -However all master-eligible nodes, including voting-only nodes, are on the -critical path for <>. Cluster state updates are usually independent of -performance-critical workloads such as indexing or searches, but they are -involved in management activities such as index creation and rollover, mapping -updates, and recovery after a failure. The performance characteristics of these -activities are a function of the speed of the storage on each master-eligible -node, as well as the reliability and latency of the network interconnections -between the elected master node and the other nodes in the cluster. You must -therefore ensure that the storage and networking available to the nodes in your -cluster are good enough to meet your performance goals. - -[[data-node]] -==== Data nodes - -Data nodes hold the shards that contain the documents you have indexed. Data -nodes handle data related operations like CRUD, search, and aggregations. -These operations are I/O-, memory-, and CPU-intensive. It is important to -monitor these resources and to add more data nodes if they are overloaded. - -The main benefit of having dedicated data nodes is the separation of the master -and data roles. - -In a multi-tier deployment architecture, you use specialized data roles to -assign data nodes to specific tiers: `data_content`,`data_hot`, `data_warm`, -`data_cold`, or `data_frozen`. A node can belong to multiple tiers. - -If you want to include a node in all tiers, or if your cluster does not use multiple tiers, then you can use the generic `data` role. - -include::../how-to/shard-limits.asciidoc[] - -WARNING: If you assign a node to a specific tier using a specialized data role, then you shouldn't also assign it the generic `data` role. The generic `data` role takes precedence over specialized data roles. - -[[generic-data-node]] -===== Generic data node - -Generic data nodes are included in all content tiers. - -To create a dedicated generic data node, set: -[source,yaml] ----- -node.roles: [ data ] ----- - -[[data-content-node]] -===== Content data node - -Content data nodes are part of the content tier. -include::{es-ref-dir}/datatiers.asciidoc[tag=content-tier] - -To create a dedicated content node, set: -[source,yaml] ----- -node.roles: [ data_content ] ----- - -[[data-hot-node]] -===== Hot data node - -Hot data nodes are part of the hot tier. -include::{es-ref-dir}/datatiers.asciidoc[tag=hot-tier] - -To create a dedicated hot node, set: -[source,yaml] ----- -node.roles: [ data_hot ] ----- - -[[data-warm-node]] -===== Warm data node - -Warm data nodes are part of the warm tier. -include::{es-ref-dir}/datatiers.asciidoc[tag=warm-tier] - -To create a dedicated warm node, set: -[source,yaml] ----- -node.roles: [ data_warm ] ----- - -[[data-cold-node]] -===== Cold data node - -Cold data nodes are part of the cold tier. -include::{es-ref-dir}/datatiers.asciidoc[tag=cold-tier] - -To create a dedicated cold node, set: -[source,yaml] ----- -node.roles: [ data_cold ] ----- - -[[data-frozen-node]] -===== Frozen data node - -Frozen data nodes are part of the frozen tier. -include::{es-ref-dir}/datatiers.asciidoc[tag=frozen-tier] - -To create a dedicated frozen node, set: -[source,yaml] ----- -node.roles: [ data_frozen ] ----- - -[[node-ingest-node]] -==== Ingest node - -Ingest nodes can execute pre-processing pipelines, composed of one or more -ingest processors. Depending on the type of operations performed by the ingest -processors and the required resources, it may make sense to have dedicated -ingest nodes, that will only perform this specific task. - -To create a dedicated ingest node, set: - -[source,yaml] ----- -node.roles: [ ingest ] ----- - -[[coordinating-only-node]] -==== Coordinating only node - -If you take away the ability to be able to handle master duties, to hold data, -and pre-process documents, then you are left with a _coordinating_ node that -can only route requests, handle the search reduce phase, and distribute bulk -indexing. Essentially, coordinating only nodes behave as smart load balancers. - -Coordinating only nodes can benefit large clusters by offloading the -coordinating node role from data and master-eligible nodes. They join the -cluster and receive the full <>, like every other -node, and they use the cluster state to route requests directly to the -appropriate place(s). - -WARNING: Adding too many coordinating only nodes to a cluster can increase the -burden on the entire cluster because the elected master node must await -acknowledgement of cluster state updates from every node! The benefit of -coordinating only nodes should not be overstated -- data nodes can happily -serve the same purpose. - -To create a dedicated coordinating node, set: - -[source,yaml] ----- -node.roles: [ ] ----- - -[[remote-node]] -==== Remote-eligible node - -A remote-eligible node acts as a cross-cluster client and connects to -<>. Once connected, you can search -remote clusters using <>. You can also sync -data between clusters using <>. - -[source,yaml] ----- -node.roles: [ remote_cluster_client ] ----- - -[[ml-node]] -==== [xpack]#Machine learning node# - -{ml-cap} nodes run jobs and handle {ml} API requests. For more information, see -<>. - -To create a dedicated {ml} node, set: - -[source,yaml] ----- -node.roles: [ ml, remote_cluster_client] ----- - -The `remote_cluster_client` role is optional but strongly recommended. -Otherwise, {ccs} fails when used in {ml} jobs or {dfeeds}. If you use {ccs} in -your {anomaly-jobs}, the `remote_cluster_client` role is also required on all -master-eligible nodes. Otherwise, the {dfeed} cannot start. See <>. - -[[transform-node]] -==== [xpack]#{transform-cap} node# - -{transform-cap} nodes run {transforms} and handle {transform} API requests. For -more information, see <>. - -To create a dedicated {transform} node, set: - -[source,yaml] ----- -node.roles: [ transform, remote_cluster_client ] ----- - -The `remote_cluster_client` role is optional but strongly recommended. -Otherwise, {ccs} fails when used in {transforms}. See <>. - -[[change-node-role]] -==== Changing the role of a node - -Each data node maintains the following data on disk: - -* the shard data for every shard allocated to that node, -* the index metadata corresponding with every shard allocated to that node, and -* the cluster-wide metadata, such as settings and index templates. - -Similarly, each master-eligible node maintains the following data on disk: - -* the index metadata for every index in the cluster, and -* the cluster-wide metadata, such as settings and index templates. - -Each node checks the contents of its data path at startup. If it discovers -unexpected data then it will refuse to start. This is to avoid importing -unwanted <> which can lead -to a red cluster health. To be more precise, nodes without the `data` role will -refuse to start if they find any shard data on disk at startup, and nodes -without both the `master` and `data` roles will refuse to start if they have any -index metadata on disk at startup. - -It is possible to change the roles of a node by adjusting its -`elasticsearch.yml` file and restarting it. This is known as _repurposing_ a -node. In order to satisfy the checks for unexpected data described above, you -must perform some extra steps to prepare a node for repurposing when starting -the node without the `data` or `master` roles. - -* If you want to repurpose a data node by removing the `data` role then you - should first use an <> to safely - migrate all the shard data onto other nodes in the cluster. - -* If you want to repurpose a node to have neither the `data` nor `master` roles - then it is simplest to start a brand-new node with an empty data path and the - desired roles. You may find it safest to use an - <> to migrate the shard data elsewhere - in the cluster first. - -If it is not possible to follow these extra steps then you may be able to use -the <> tool to delete any -excess data that prevents a node from starting. +To learn more about the available node roles, see <>. [discrete] === Node data path settings @@ -495,6 +125,25 @@ modify the contents of the data directory. The data directory contains no executables so a virus scan will only find false positives. // end::modules-node-data-path-warning-tag[] +[[custom-node-attributes]] +==== Custom node attributes + +If needed, you can add custom attributes to a node. These attributes can be used to <>, or to group nodes together for <>. + +[TIP] +=============================================== +You can also set a node attribute using the `-E` command line argument when you start a node: + +[source,sh] +-------------------------------------------------------- +./bin/elasticsearch -Enode.attr.rack_id=rack_one +-------------------------------------------------------- +=============================================== + +`node.attr.`:: + (<>) + A custom attribute that you can assign to a node. For example, you might assign a `rack_id` attribute to each node to ensure that primary and replica shards are not allocated on the same rack. You can specify multiple attributes as a comma-separated list. + [discrete] [[other-node-settings]] === Other node settings @@ -504,4 +153,4 @@ including: * <> * <> -* <> +* <> \ No newline at end of file diff --git a/docs/reference/modules/remote-clusters.asciidoc b/docs/reference/modules/remote-clusters.asciidoc index ca1c507aa4ed..87078c0f1956 100644 --- a/docs/reference/modules/remote-clusters.asciidoc +++ b/docs/reference/modules/remote-clusters.asciidoc @@ -80,7 +80,7 @@ The _gateway nodes_ selection depends on the following criteria: + * *version*: Remote nodes must be compatible with the cluster they are registered to. -* *role*: By default, any non-<> node can act as a +* *role*: By default, any non-<> node can act as a gateway node. Dedicated master nodes are never selected as gateway nodes. * *attributes*: You can define the gateway nodes for a cluster by setting <> to `true`. diff --git a/docs/reference/modules/shard-ops.asciidoc b/docs/reference/modules/shard-ops.asciidoc index 66ceebcfa031..93d6b6d3468f 100644 --- a/docs/reference/modules/shard-ops.asciidoc +++ b/docs/reference/modules/shard-ops.asciidoc @@ -25,7 +25,7 @@ By default, the primary and replica shard copies for an index can be allocated t You can control how shard copies are allocated using the following settings: -- <>: Use these settings to control how shard copies are allocated and balanced across the entire cluster. For example, you might want to allocate nodes availability zones, or prevent certain nodes from being used so you can perform maintenance. +- <>: Use these settings to control how shard copies are allocated and balanced across the entire cluster. For example, you might want to <>, or prevent certain nodes from being used so you can perform maintenance. - <>: Use these settings to control how the shard copies for a specific index are allocated. For example, you might want to allocate an index to a node in a specific data tier, or to an node with specific attributes. @@ -80,4 +80,4 @@ When a shard copy is relocated, it is created as a new shard copy on the target You can control how and when shard copies are relocated. For example, you can adjust the rebalancing settings that control when shard copies are relocated to balance the cluster, or the high watermark for disk-based shard allocation that can trigger relocation. These settings are part of the <>. -Shard relocation operations also respect shard allocation and recovery settings. \ No newline at end of file +Shard relocation operations also respect shard allocation and recovery settings. \ No newline at end of file diff --git a/docs/reference/monitoring/index.asciidoc b/docs/reference/monitoring/index.asciidoc index 1b83f4c11ba5..82e1447ba8a1 100644 --- a/docs/reference/monitoring/index.asciidoc +++ b/docs/reference/monitoring/index.asciidoc @@ -9,6 +9,7 @@ performance of your {es} cluster. * <> * <> +* <> * <> * <> * <> @@ -23,6 +24,8 @@ include::overview.asciidoc[] include::how-monitoring-works.asciidoc[] +include::{es-ref-dir}/setup/logging-config.asciidoc[] + include::production.asciidoc[] include::configuring-elastic-agent.asciidoc[] diff --git a/docs/reference/node-roles.asciidoc b/docs/reference/node-roles.asciidoc new file mode 100644 index 000000000000..e8c1d9143a38 --- /dev/null +++ b/docs/reference/node-roles.asciidoc @@ -0,0 +1,437 @@ +[[node-roles-overview]] +== Node roles + +Any time that you start an instance of {es}, you are starting a _node_. A +collection of connected nodes is called a <>. If you +are running a single node of {es}, then you have a cluster of one node. All nodes know about all the other nodes in the cluster and can forward client +requests to the appropriate node. + +Each node performs one or more roles. Roles control the behavior of the node in the cluster. + +[discrete] +[[set-node-roles]] +=== Set node roles + +You define a node's roles by setting `node.roles` in <>. If you set `node.roles`, the node is only assigned the roles you specify. If you don't set `node.roles`, the node is assigned the following roles: + +* `master` +* `data` +* `data_content` +* `data_hot` +* `data_warm` +* `data_cold` +* `data_frozen` +* `ingest` +* `ml` +* `remote_cluster_client` +* `transform` + +[IMPORTANT] +==== +If you set `node.roles`, ensure you specify every node role your cluster needs. +Every cluster requires the following node roles: + +* `master` +* {blank} ++ +-- +`data_content` and `data_hot` + +OR + +`data` +-- + +Some {stack} features also require specific node roles: + +- {ccs-cap} and {ccr} require the `remote_cluster_client` role. +- {stack-monitor-app} and ingest pipelines require the `ingest` role. +- {fleet}, the {security-app}, and {transforms} require the `transform` role. + The `remote_cluster_client` role is also required to use {ccs} with these + features. +- {ml-cap} features, such as {anomaly-detect}, require the `ml` role. +==== + +As the cluster grows and in particular if you have large {ml} jobs or +{ctransforms}, consider separating dedicated master-eligible nodes from +dedicated data nodes, {ml} nodes, and {transform} nodes. + +[discrete] +[[change-node-role]] +=== Change the role of a node + +Each data node maintains the following data on disk: + +* the shard data for every shard allocated to that node, +* the index metadata corresponding with every shard allocated to that node, and +* the cluster-wide metadata, such as settings and index templates. + +Similarly, each master-eligible node maintains the following data on disk: + +* the index metadata for every index in the cluster, and +* the cluster-wide metadata, such as settings and index templates. + +Each node checks the contents of its data path at startup. If it discovers +unexpected data then it will refuse to start. This is to avoid importing +unwanted <> which can lead +to a red cluster health. To be more precise, nodes without the `data` role will +refuse to start if they find any shard data on disk at startup, and nodes +without both the `master` and `data` roles will refuse to start if they have any +index metadata on disk at startup. + +It is possible to change the roles of a node by adjusting its +`elasticsearch.yml` file and restarting it. This is known as _repurposing_ a +node. In order to satisfy the checks for unexpected data described above, you +must perform some extra steps to prepare a node for repurposing when starting +the node without the `data` or `master` roles. + +* If you want to repurpose a data node by removing the `data` role then you + should first use an <> to safely + migrate all the shard data onto other nodes in the cluster. + +* If you want to repurpose a node to have neither the `data` nor `master` roles + then it is simplest to start a brand-new node with an empty data path and the + desired roles. You may find it safest to use an + <> to migrate the shard data elsewhere + in the cluster first. + +If it is not possible to follow these extra steps then you may be able to use +the <> tool to delete any +excess data that prevents a node from starting. + +[discrete] +[[node-roles-list]] +=== Available node roles + +The following is a list of the roles that a node can perform in a cluster. A node can have one or more roles. + +* <> (`master`): A node that is eligible to be +<>, which controls the cluster. + +* <> (`data`, `data_content`, `data_hot`, `data_warm`, `data_cold`, `data_frozen`): A node that has one of several data roles. Data nodes hold data and perform data related operations such as CRUD, search, and aggregations. You might use multiple data roles in a cluster so you can implement <>. + +* <> (`ingest`): Ingest nodes are able to apply an <> to a document in order to transform and enrich the document before indexing. With a heavy ingest load, it makes sense to use dedicated ingest nodes and to not include the `ingest` role from nodes that have the `master` or `data` roles. + +* <> (`remote_cluster_client`): A node that is eligible to act as a remote client. + +* <> (`ml`): A node that can run {ml-features}. If you want to use {ml-features}, there must be at least one {ml} node in your cluster. For more information, see <> and {ml-docs}/index.html[Machine learning in the {stack}]. + +* <> (`transform`): A node that can perform {transforms}. If you want to use {transforms}, there must be at least one {transform} node in your cluster. For more information, see <> and <>. + +[NOTE] +[[coordinating-node]] +.Coordinating node +=============================================== + +Requests like search requests or bulk-indexing requests may involve data held +on different data nodes. A search request, for example, is executed in two +phases which are coordinated by the node which receives the client request -- +the _coordinating node_. + +In the _scatter_ phase, the coordinating node forwards the request to the data +nodes which hold the data. Each data node executes the request locally and +returns its results to the coordinating node. In the _gather_ phase, the +coordinating node reduces each data node's results into a single global +result set. + +Every node is implicitly a coordinating node. This means that a node that has +an explicit empty list of roles in the `node.roles` setting will only act as a coordinating +node, which cannot be disabled. As a result, such a node needs to have enough +memory and CPU in order to deal with the gather phase. + +=============================================== + +[discrete] + +[[master-node-role]] +==== Master-eligible node + +The master node is responsible for lightweight cluster-wide actions such as +creating or deleting an index, tracking which nodes are part of the cluster, +and deciding which shards to allocate to which nodes. It is important for +cluster health to have a stable master node. + +Any master-eligible node that is not a <> may +be elected to become the master node by the <>. + +IMPORTANT: Master nodes must have a `path.data` directory whose contents +persist across restarts, just like data nodes, because this is where the +cluster metadata is stored. The cluster metadata describes how to read the data +stored on the data nodes, so if it is lost then the data stored on the data +nodes cannot be read. + +[discrete] +[[dedicated-master-node]] +===== Dedicated master-eligible node + +It is important for the health of the cluster that the elected master node has +the resources it needs to fulfill its responsibilities. If the elected master +node is overloaded with other tasks then the cluster will not operate well. The +most reliable way to avoid overloading the master with other tasks is to +configure all the master-eligible nodes to be _dedicated master-eligible nodes_ +which only have the `master` role, allowing them to focus on managing the +cluster. Master-eligible nodes will still also behave as +<> that route requests from clients to +the other nodes in the cluster, but you should _not_ use dedicated master nodes +for this purpose. + +A small or lightly-loaded cluster may operate well if its master-eligible nodes +have other roles and responsibilities, but once your cluster comprises more +than a handful of nodes it usually makes sense to use dedicated master-eligible +nodes. + +To create a dedicated master-eligible node, set: + +[source,yaml] +------------------- +node.roles: [ master ] +------------------- + +[discrete] +[[voting-only-node]] +===== Voting-only master-eligible node + +A voting-only master-eligible node is a node that participates in +<> but which will not act as the cluster's +elected master node. In particular, a voting-only node can serve as a tiebreaker +in elections. + +It may seem confusing to use the term "master-eligible" to describe a +voting-only node since such a node is not actually eligible to become the master +at all. This terminology is an unfortunate consequence of history: +master-eligible nodes are those nodes that participate in elections and perform +certain tasks during cluster state publications, and voting-only nodes have the +same responsibilities even if they can never become the elected master. + +To configure a master-eligible node as a voting-only node, include `master` and +`voting_only` in the list of roles. For example to create a voting-only data +node: + +[source,yaml] +------------------- +node.roles: [ data, master, voting_only ] +------------------- + +IMPORTANT: Only nodes with the `master` role can be marked as having the +`voting_only` role. + +High availability (HA) clusters require at least three master-eligible nodes, at +least two of which are not voting-only nodes. Such a cluster will be able to +elect a master node even if one of the nodes fails. + +Voting-only master-eligible nodes may also fill other roles in your cluster. +For instance, a node may be both a data node and a voting-only master-eligible +node. A _dedicated_ voting-only master-eligible nodes is a voting-only +master-eligible node that fills no other roles in the cluster. To create a +dedicated voting-only master-eligible node, set: + +[source,yaml] +------------------- +node.roles: [ master, voting_only ] +------------------- + +Since dedicated voting-only nodes never act as the cluster's elected master, +they may require less heap and a less powerful CPU than the true master nodes. +However all master-eligible nodes, including voting-only nodes, are on the +critical path for <>. Cluster state updates are usually independent of +performance-critical workloads such as indexing or searches, but they are +involved in management activities such as index creation and rollover, mapping +updates, and recovery after a failure. The performance characteristics of these +activities are a function of the speed of the storage on each master-eligible +node, as well as the reliability and latency of the network interconnections +between the elected master node and the other nodes in the cluster. You must +therefore ensure that the storage and networking available to the nodes in your +cluster are good enough to meet your performance goals. + +[discrete] +[[data-node-role]] +==== Data nodes + +Data nodes hold the shards that contain the documents you have indexed. Data +nodes handle data related operations like CRUD, search, and aggregations. +These operations are I/O-, memory-, and CPU-intensive. It is important to +monitor these resources and to add more data nodes if they are overloaded. + +The main benefit of having dedicated data nodes is the separation of the master +and data roles. + +In a multi-tier deployment architecture, you use specialized data roles to +assign data nodes to specific tiers: `data_content`,`data_hot`, `data_warm`, +`data_cold`, or `data_frozen`. A node can belong to multiple tiers. + +If you want to include a node in all tiers, or if your cluster does not use multiple tiers, then you can use the generic `data` role. + +include::{es-ref-dir}/how-to/shard-limits.asciidoc[] + +WARNING: If you assign a node to a specific tier using a specialized data role, then you shouldn't also assign it the generic `data` role. The generic `data` role takes precedence over specialized data roles. + +[discrete] +[[generic-data-node]] +===== Generic data node + +Generic data nodes are included in all content tiers. A node with a generic `data` role can fill any of the specialized data node roles. + +To create a dedicated generic data node, set: +[source,yaml] +---- +node.roles: [ data ] +---- + +[discrete] +[[data-content-node]] +===== Content data node + +Content data nodes are part of the content tier. +include::{es-ref-dir}/datatiers.asciidoc[tag=content-tier] + +To create a dedicated content node, set: +[source,yaml] +---- +node.roles: [ data_content ] +---- + +[discrete] +[[data-hot-node]] +===== Hot data node + +Hot data nodes are part of the hot tier. +include::{es-ref-dir}/datatiers.asciidoc[tag=hot-tier] + +To create a dedicated hot node, set: +[source,yaml] +---- +node.roles: [ data_hot ] +---- + +[discrete] +[[data-warm-node]] +===== Warm data node + +Warm data nodes are part of the warm tier. +include::{es-ref-dir}/datatiers.asciidoc[tag=warm-tier] + +To create a dedicated warm node, set: +[source,yaml] +---- +node.roles: [ data_warm ] +---- + +[discrete] +[[data-cold-node]] +===== Cold data node + +Cold data nodes are part of the cold tier. +include::{es-ref-dir}/datatiers.asciidoc[tag=cold-tier] + +To create a dedicated cold node, set: +[source,yaml] +---- +node.roles: [ data_cold ] +---- + +[discrete] +[[data-frozen-node]] +===== Frozen data node + +Frozen data nodes are part of the frozen tier. +include::{es-ref-dir}/datatiers.asciidoc[tag=frozen-tier] + +To create a dedicated frozen node, set: +[source,yaml] +---- +node.roles: [ data_frozen ] +---- + +[discrete] +[[node-ingest-node]] +==== Ingest node + +Ingest nodes can execute pre-processing pipelines, composed of one or more +ingest processors. Depending on the type of operations performed by the ingest +processors and the required resources, it may make sense to have dedicated +ingest nodes, that will only perform this specific task. + +To create a dedicated ingest node, set: + +[source,yaml] +---- +node.roles: [ ingest ] +---- + +[discrete] +[[coordinating-only-node]] +==== Coordinating only node + +If you take away the ability to be able to handle master duties, to hold data, +and pre-process documents, then you are left with a _coordinating_ node that +can only route requests, handle the search reduce phase, and distribute bulk +indexing. Essentially, coordinating only nodes behave as smart load balancers. + +Coordinating only nodes can benefit large clusters by offloading the +coordinating node role from data and master-eligible nodes. They join the +cluster and receive the full <>, like every other +node, and they use the cluster state to route requests directly to the +appropriate place(s). + +WARNING: Adding too many coordinating only nodes to a cluster can increase the +burden on the entire cluster because the elected master node must await +acknowledgement of cluster state updates from every node! The benefit of +coordinating only nodes should not be overstated -- data nodes can happily +serve the same purpose. + +To create a dedicated coordinating node, set: + +[source,yaml] +---- +node.roles: [ ] +---- + +[discrete] +[[remote-node]] +==== Remote-eligible node + +A remote-eligible node acts as a cross-cluster client and connects to +<>. Once connected, you can search +remote clusters using <>. You can also sync +data between clusters using <>. + +[source,yaml] +---- +node.roles: [ remote_cluster_client ] +---- + +[discrete] +[[ml-node-role]] +==== [xpack]#Machine learning node# + +{ml-cap} nodes run jobs and handle {ml} API requests. For more information, see +<>. + +To create a dedicated {ml} node, set: + +[source,yaml] +---- +node.roles: [ ml, remote_cluster_client] +---- + +The `remote_cluster_client` role is optional but strongly recommended. +Otherwise, {ccs} fails when used in {ml} jobs or {dfeeds}. If you use {ccs} in +your {anomaly-jobs}, the `remote_cluster_client` role is also required on all +master-eligible nodes. Otherwise, the {dfeed} cannot start. See <>. + +[discrete] +[[transform-node-role]] +==== [xpack]#{transform-cap} node# + +{transform-cap} nodes run {transforms} and handle {transform} API requests. For +more information, see <>. + +To create a dedicated {transform} node, set: + +[source,yaml] +---- +node.roles: [ transform, remote_cluster_client ] +---- + +The `remote_cluster_client` role is optional but strongly recommended. +Otherwise, {ccs} fails when used in {transforms}. See <>. \ No newline at end of file diff --git a/docs/reference/path-settings-overview.asciidoc b/docs/reference/path-settings-overview.asciidoc new file mode 100644 index 000000000000..0740b9769c9b --- /dev/null +++ b/docs/reference/path-settings-overview.asciidoc @@ -0,0 +1,112 @@ +[[path-settings-overview]] +=== Path settings + +include::{es-ref-dir}/setup/important-settings/path-settings.asciidoc[] + +[[multiple-data-paths]] +==== Multiple data paths +deprecated::[7.13.0] + +If needed, you can specify multiple paths in `path.data`. {es} stores the node's +data across all provided paths but keeps each shard's data on the same path. + +{es} does not balance shards across a node's data paths. High disk +usage in a single path can trigger a <> for the entire node. If triggered, {es} will not add shards to +the node, even if the node’s other paths have available disk space. If you need +additional disk space, we recommend you add a new node rather than additional +data paths. + +include::{es-ref-dir}/tab-widgets/multi-data-path-widget.asciidoc[] + +[[mdp-migrate]] +===== Migrate from multiple data paths + +Support for multiple data paths was deprecated in 7.13 and will be removed +in a future release. + +As an alternative to multiple data paths, you can create a filesystem which +spans multiple disks with a hardware virtualisation layer such as RAID, or a +software virtualisation layer such as Logical Volume Manager (LVM) on Linux or +Storage Spaces on Windows. If you wish to use multiple data paths on a single +machine then you must run one node for each data path. + +If you currently use multiple data paths in a +{ref}/high-availability-cluster-design.html[highly available cluster] then you +can migrate to a setup that uses a single path for each node without downtime +using a process similar to a +{ref}/restart-cluster.html#restart-cluster-rolling[rolling restart]: shut each +node down in turn and replace it with one or more nodes each configured to use +a single data path. In more detail, for each node that currently has multiple +data paths you should follow the following process. In principle you can +perform this migration during a rolling upgrade to 8.0, but we recommend +migrating to a single-data-path setup before starting to upgrade. + +1. Take a snapshot to protect your data in case of disaster. + +2. Optionally, migrate the data away from the target node by using an +{ref}/modules-cluster.html#cluster-shard-allocation-filtering[allocation filter]: ++ +[source,console] +-------------------------------------------------- +PUT _cluster/settings +{ + "persistent": { + "cluster.routing.allocation.exclude._name": "target-node-name" + } +} +-------------------------------------------------- ++ +You can use the {ref}/cat-allocation.html[cat allocation API] to track progress +of this data migration. If some shards do not migrate then the +{ref}/cluster-allocation-explain.html[cluster allocation explain API] will help +you to determine why. + +3. Follow the steps in the +{ref}/restart-cluster.html#restart-cluster-rolling[rolling restart process] +up to and including shutting the target node down. + +4. Ensure your cluster health is `yellow` or `green`, so that there is a copy +of every shard assigned to at least one of the other nodes in your cluster. + +5. If applicable, remove the allocation filter applied in the earlier step. ++ +[source,console] +-------------------------------------------------- +PUT _cluster/settings +{ + "persistent": { + "cluster.routing.allocation.exclude._name": null + } +} +-------------------------------------------------- + +6. Discard the data held by the stopped node by deleting the contents of its +data paths. + +7. Reconfigure your storage. For instance, combine your disks into a single +filesystem using LVM or Storage Spaces. Ensure that your reconfigured storage +has sufficient space for the data that it will hold. + +8. Reconfigure your node by adjusting the `path.data` setting in its +`elasticsearch.yml` file. If needed, install more nodes each with their own +`path.data` setting pointing at a separate data path. + +9. Start the new nodes and follow the rest of the +{ref}/restart-cluster.html#restart-cluster-rolling[rolling restart process] for +them. + +10. Ensure your cluster health is `green`, so that every shard has been +assigned. + +You can alternatively add some number of single-data-path nodes to your +cluster, migrate all your data over to these new nodes using +{ref}/modules-cluster.html#cluster-shard-allocation-filtering[allocation filters], +and then remove the old nodes from the cluster. This approach will temporarily +double the size of your cluster so it will only work if you have the capacity to +expand your cluster like this. + +If you currently use multiple data paths but your cluster is not highly +available then you can migrate to a non-deprecated configuration by taking +a snapshot, creating a new cluster with the desired configuration and restoring +the snapshot into it. diff --git a/docs/reference/search/multi-search-template-api.asciidoc b/docs/reference/search/multi-search-template-api.asciidoc index 010320c6b05e..2fdb412575eb 100644 --- a/docs/reference/search/multi-search-template-api.asciidoc +++ b/docs/reference/search/multi-search-template-api.asciidoc @@ -85,7 +85,7 @@ cross-cluster search requests. Defaults to `true`. `max_concurrent_searches`:: (Optional, integer) Maximum number of concurrent searches the API can run. -Defaults to +max(1, (# of <> * +Defaults to +max(1, (# of <> * min(<>, 10)))+. `rest_total_hits_as_int`:: diff --git a/docs/reference/search/multi-search.asciidoc b/docs/reference/search/multi-search.asciidoc index ea2dd5977933..6adcc62e5ec4 100644 --- a/docs/reference/search/multi-search.asciidoc +++ b/docs/reference/search/multi-search.asciidoc @@ -97,7 +97,7 @@ include::{es-ref-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailabl `max_concurrent_searches`:: (Optional, integer) Maximum number of concurrent searches the multi search API can execute. Defaults -to +max(1, (# of <> * min(<>, 10)))+. +to +max(1, (# of <> * min(<>, 10)))+. `max_concurrent_shard_requests`:: + diff --git a/docs/reference/settings/ml-settings.asciidoc b/docs/reference/settings/ml-settings.asciidoc index 1077a63b0024..08163b1391f2 100644 --- a/docs/reference/settings/ml-settings.asciidoc +++ b/docs/reference/settings/ml-settings.asciidoc @@ -19,6 +19,8 @@ at all. // end::ml-settings-description-tag[] +TIP: To control memory usage used by {ml} jobs, you can use the <>. + [discrete] [[general-ml-settings]] ==== General machine learning settings @@ -67,7 +69,7 @@ limitations as described <>. The inference cache exists in the JVM heap on each ingest node. The cache affords faster processing times for the `inference` processor. The value can be a static byte sized value (such as `2gb`) or a percentage of total allocated -heap. Defaults to `40%`. See also <>. +heap. Defaults to `40%`. See also <>. [[xpack-interference-model-ttl]] // tag::interference-model-ttl-tag[] @@ -249,11 +251,4 @@ nodes in your cluster, you shouldn't use this setting. + If this setting is `true` it also affects the default value for `xpack.ml.max_model_memory_limit`. In this case `xpack.ml.max_model_memory_limit` -defaults to the largest size that could be assigned in the current cluster. - -[discrete] -[[model-inference-circuit-breaker]] -==== {ml-cap} circuit breaker settings - -The relevant circuit breaker settings can be found in the <>. - +defaults to the largest size that could be assigned in the current cluster. \ No newline at end of file diff --git a/docs/reference/setup.asciidoc b/docs/reference/setup.asciidoc index 80828fdbfbb0..922f1bdba4d1 100644 --- a/docs/reference/setup.asciidoc +++ b/docs/reference/setup.asciidoc @@ -27,6 +27,8 @@ the only resource-intensive application on the host or container. For example, you might run {metricbeat} alongside {es} for cluster statistics, but a resource-heavy {ls} deployment should be on its own host. +// alphabetized + include::run-elasticsearch-locally.asciidoc[] include::setup/install.asciidoc[] @@ -47,30 +49,28 @@ include::settings/ccr-settings.asciidoc[] include::modules/discovery/discovery-settings.asciidoc[] +include::settings/data-stream-lifecycle-settings.asciidoc[] + include::modules/indices/fielddata.asciidoc[] +include::modules/gateway.asciidoc[] + include::settings/health-diagnostic-settings.asciidoc[] include::settings/ilm-settings.asciidoc[] -include::settings/data-stream-lifecycle-settings.asciidoc[] - include::modules/indices/index_management.asciidoc[] include::modules/indices/recovery.asciidoc[] include::modules/indices/indexing_buffer.asciidoc[] +include::settings/inference-settings.asciidoc[] + include::settings/license-settings.asciidoc[] -include::modules/gateway.asciidoc[] - -include::setup/logging-config.asciidoc[] - include::settings/ml-settings.asciidoc[] -include::settings/inference-settings.asciidoc[] - include::settings/monitoring-settings.asciidoc[] include::modules/node.asciidoc[] @@ -79,6 +79,8 @@ include::modules/network.asciidoc[] include::modules/indices/query_cache.asciidoc[] +include::{es-ref-dir}/path-settings-overview.asciidoc[] + include::modules/indices/search-settings.asciidoc[] include::settings/security-settings.asciidoc[] diff --git a/docs/reference/setup/add-nodes.asciidoc b/docs/reference/setup/add-nodes.asciidoc index ba749782c092..941a3e6c40f7 100644 --- a/docs/reference/setup/add-nodes.asciidoc +++ b/docs/reference/setup/add-nodes.asciidoc @@ -48,7 +48,7 @@ For more information about discovery and shard allocation, refer to As nodes are added or removed Elasticsearch maintains an optimal level of fault tolerance by automatically updating the cluster's _voting configuration_, which -is the set of <> whose responses are counted +is the set of <> whose responses are counted when making decisions such as electing a new master or committing a new cluster state. diff --git a/docs/reference/setup/advanced-configuration.asciidoc b/docs/reference/setup/advanced-configuration.asciidoc index 2a7ccc56742d..73b210ea559b 100644 --- a/docs/reference/setup/advanced-configuration.asciidoc +++ b/docs/reference/setup/advanced-configuration.asciidoc @@ -1,13 +1,7 @@ [[advanced-configuration]] -=== Advanced configuration - -Modifying advanced settings is generally not recommended and could negatively -impact performance and stability. Using the {es}-provided defaults -is recommended in most circumstances. +=== Set JVM options [[set-jvm-options]] -==== Set JVM options - If needed, you can override the default JVM options by adding custom options files (preferred) or setting the `ES_JAVA_OPTS` environment variable. @@ -21,10 +15,14 @@ Where you put the JVM options files depends on the type of installation: * Docker: Bind mount custom JVM options files into `/usr/share/elasticsearch/config/jvm.options.d/`. +CAUTION: Setting your own JVM options is generally not recommended and could negatively +impact performance and stability. Using the {es}-provided defaults +is recommended in most circumstances. + NOTE: Do not modify the root `jvm.options` file. Use files in `jvm.options.d/` instead. [[jvm-options-syntax]] -===== JVM options syntax +==== JVM options syntax A JVM options file contains a line-delimited list of JVM arguments. Arguments are preceded by a dash (`-`). @@ -66,7 +64,7 @@ and ignored. Lines that aren't commented out and aren't recognized as valid JVM arguments are rejected and {es} will fail to start. [[jvm-options-env]] -===== Use environment variables to set JVM options +==== Use environment variables to set JVM options In production, use JVM options files to override the default settings. In testing and development environments, @@ -155,23 +153,11 @@ options. We do not recommend using `ES_JAVA_OPTS` in production. NOTE: If you are running {es} as a Windows service, you can change the heap size using the service manager. See <>. -[[readiness-tcp-port]] -===== Enable the Elasticsearch TCP readiness port - -preview::[] - -If configured, a node can open a TCP port when the node is in a ready state. A node is deemed -ready when it has successfully joined a cluster. In a single node configuration, the node is -said to be ready, when it's able to accept requests. - -To enable the readiness TCP port, use the `readiness.port` setting. The readiness service will bind to -all host addresses. - -If the node leaves the cluster, or the <> is used to mark the node -for shutdown, the readiness port is immediately closed. - -A successful connection to the readiness TCP port signals that the {es} node is ready. When a client -connects to the readiness port, the server simply terminates the socket connection. No data is sent back -to the client. If a client cannot connect to the readiness port, the node is not ready. +[[heap-dump-path]] +include::important-settings/heap-dump-path.asciidoc[leveloffset=-1] +[[gc-logging]] +include::important-settings/gc-logging.asciidoc[leveloffset=-1] +[[error-file-path]] +include::important-settings/error-file.asciidoc[leveloffset=-1] \ No newline at end of file diff --git a/docs/reference/setup/important-settings.asciidoc b/docs/reference/setup/important-settings.asciidoc index 03c891af7074..26f9c79cb669 100644 --- a/docs/reference/setup/important-settings.asciidoc +++ b/docs/reference/setup/important-settings.asciidoc @@ -19,10 +19,20 @@ of items which *must* be considered before using your cluster in production: Our {ess-trial}[{ecloud}] service configures these items automatically, making your cluster production-ready by default. +[[path-settings]] +[discrete] +==== Path settings + include::important-settings/path-settings.asciidoc[] +Elasticsearch offers a deprecated setting that allows you to specify multiple paths in `path.data`. +To learn about this setting, and how to migrate away from it, refer to <>. + include::important-settings/cluster-name.asciidoc[] +[[node-name]] +[discrete] +==== Node name setting include::important-settings/node-name.asciidoc[] include::important-settings/network-host.asciidoc[] diff --git a/docs/reference/setup/important-settings/cluster-name.asciidoc b/docs/reference/setup/important-settings/cluster-name.asciidoc index 3f1516f21de1..6d489eee76cf 100644 --- a/docs/reference/setup/important-settings/cluster-name.asciidoc +++ b/docs/reference/setup/important-settings/cluster-name.asciidoc @@ -1,4 +1,3 @@ -[[cluster-name]] [discrete] ==== Cluster name setting diff --git a/docs/reference/setup/important-settings/error-file.asciidoc b/docs/reference/setup/important-settings/error-file.asciidoc index ca95ded78d53..2f654002d51f 100644 --- a/docs/reference/setup/important-settings/error-file.asciidoc +++ b/docs/reference/setup/important-settings/error-file.asciidoc @@ -1,4 +1,3 @@ -[[error-file-path]] [discrete] ==== JVM fatal error log setting diff --git a/docs/reference/setup/important-settings/gc-logging.asciidoc b/docs/reference/setup/important-settings/gc-logging.asciidoc index 3534e1335c9f..873c85d58d91 100644 --- a/docs/reference/setup/important-settings/gc-logging.asciidoc +++ b/docs/reference/setup/important-settings/gc-logging.asciidoc @@ -1,4 +1,3 @@ -[[gc-logging]] [discrete] ==== GC logging settings @@ -20,9 +19,8 @@ To see further options not contained in the original JEP, see https://docs.oracle.com/en/java/javase/13/docs/specs/man/java.html#enable-logging-with-the-jvm-unified-logging-framework[Enable Logging with the JVM Unified Logging Framework]. -[[gc-logging-examples]] [discrete] -==== Examples +===== Examples Change the default GC log output location to `/opt/my-app/gc.log` by creating `$ES_HOME/config/jvm.options.d/gc.options` with some sample diff --git a/docs/reference/setup/important-settings/heap-dump-path.asciidoc b/docs/reference/setup/important-settings/heap-dump-path.asciidoc index 8f01379842a9..8b06ae752f36 100644 --- a/docs/reference/setup/important-settings/heap-dump-path.asciidoc +++ b/docs/reference/setup/important-settings/heap-dump-path.asciidoc @@ -1,4 +1,3 @@ -[[heap-dump-path]] [discrete] ==== JVM heap dump path setting diff --git a/docs/reference/setup/important-settings/node-name.asciidoc b/docs/reference/setup/important-settings/node-name.asciidoc index eda3052d119c..f1260844a054 100644 --- a/docs/reference/setup/important-settings/node-name.asciidoc +++ b/docs/reference/setup/important-settings/node-name.asciidoc @@ -1,7 +1,3 @@ -[[node-name]] -[discrete] -==== Node name setting - {es} uses `node.name` as a human-readable identifier for a particular instance of {es}. This name is included in the response of many APIs. The node name defaults to the hostname of the machine when diff --git a/docs/reference/setup/important-settings/path-settings.asciidoc b/docs/reference/setup/important-settings/path-settings.asciidoc index a0a444ca5090..002e08e2dc74 100644 --- a/docs/reference/setup/important-settings/path-settings.asciidoc +++ b/docs/reference/setup/important-settings/path-settings.asciidoc @@ -1,7 +1,3 @@ -[[path-settings]] -[discrete] -==== Path settings - {es} writes the data you index to indices and data streams to a `data` directory. {es} writes its own application logs, which contain information about cluster health and operations, to a `logs` directory. @@ -20,113 +16,4 @@ Supported `path.data` and `path.logs` values vary by platform: include::{es-ref-dir}/tab-widgets/customize-data-log-path-widget.asciidoc[] -include::{es-ref-dir}/modules/node.asciidoc[tag=modules-node-data-path-warning-tag] - -[discrete] -==== Multiple data paths -deprecated::[7.13.0] - -If needed, you can specify multiple paths in `path.data`. {es} stores the node's -data across all provided paths but keeps each shard's data on the same path. - -{es} does not balance shards across a node's data paths. High disk -usage in a single path can trigger a <> for the entire node. If triggered, {es} will not add shards to -the node, even if the node’s other paths have available disk space. If you need -additional disk space, we recommend you add a new node rather than additional -data paths. - -include::{es-ref-dir}/tab-widgets/multi-data-path-widget.asciidoc[] - -[discrete] -[[mdp-migrate]] -==== Migrate from multiple data paths - -Support for multiple data paths was deprecated in 7.13 and will be removed -in a future release. - -As an alternative to multiple data paths, you can create a filesystem which -spans multiple disks with a hardware virtualisation layer such as RAID, or a -software virtualisation layer such as Logical Volume Manager (LVM) on Linux or -Storage Spaces on Windows. If you wish to use multiple data paths on a single -machine then you must run one node for each data path. - -If you currently use multiple data paths in a -{ref}/high-availability-cluster-design.html[highly available cluster] then you -can migrate to a setup that uses a single path for each node without downtime -using a process similar to a -{ref}/restart-cluster.html#restart-cluster-rolling[rolling restart]: shut each -node down in turn and replace it with one or more nodes each configured to use -a single data path. In more detail, for each node that currently has multiple -data paths you should follow the following process. In principle you can -perform this migration during a rolling upgrade to 8.0, but we recommend -migrating to a single-data-path setup before starting to upgrade. - -1. Take a snapshot to protect your data in case of disaster. - -2. Optionally, migrate the data away from the target node by using an -{ref}/modules-cluster.html#cluster-shard-allocation-filtering[allocation filter]: -+ -[source,console] --------------------------------------------------- -PUT _cluster/settings -{ - "persistent": { - "cluster.routing.allocation.exclude._name": "target-node-name" - } -} --------------------------------------------------- -+ -You can use the {ref}/cat-allocation.html[cat allocation API] to track progress -of this data migration. If some shards do not migrate then the -{ref}/cluster-allocation-explain.html[cluster allocation explain API] will help -you to determine why. - -3. Follow the steps in the -{ref}/restart-cluster.html#restart-cluster-rolling[rolling restart process] -up to and including shutting the target node down. - -4. Ensure your cluster health is `yellow` or `green`, so that there is a copy -of every shard assigned to at least one of the other nodes in your cluster. - -5. If applicable, remove the allocation filter applied in the earlier step. -+ -[source,console] --------------------------------------------------- -PUT _cluster/settings -{ - "persistent": { - "cluster.routing.allocation.exclude._name": null - } -} --------------------------------------------------- - -6. Discard the data held by the stopped node by deleting the contents of its -data paths. - -7. Reconfigure your storage. For instance, combine your disks into a single -filesystem using LVM or Storage Spaces. Ensure that your reconfigured storage -has sufficient space for the data that it will hold. - -8. Reconfigure your node by adjusting the `path.data` setting in its -`elasticsearch.yml` file. If needed, install more nodes each with their own -`path.data` setting pointing at a separate data path. - -9. Start the new nodes and follow the rest of the -{ref}/restart-cluster.html#restart-cluster-rolling[rolling restart process] for -them. - -10. Ensure your cluster health is `green`, so that every shard has been -assigned. - -You can alternatively add some number of single-data-path nodes to your -cluster, migrate all your data over to these new nodes using -{ref}/modules-cluster.html#cluster-shard-allocation-filtering[allocation filters], -and then remove the old nodes from the cluster. This approach will temporarily -double the size of your cluster so it will only work if you have the capacity to -expand your cluster like this. - -If you currently use multiple data paths but your cluster is not highly -available then you can migrate to a non-deprecated configuration by taking -a snapshot, creating a new cluster with the desired configuration and restoring -the snapshot into it. +include::{es-ref-dir}/modules/node.asciidoc[tag=modules-node-data-path-warning-tag] \ No newline at end of file diff --git a/docs/reference/setup/logging-config.asciidoc b/docs/reference/setup/logging-config.asciidoc index e382bbdacb46..04e9ba3f0bef 100644 --- a/docs/reference/setup/logging-config.asciidoc +++ b/docs/reference/setup/logging-config.asciidoc @@ -1,5 +1,5 @@ [[logging]] -=== Logging +== Elasticsearch application logging You can use {es}'s application logs to monitor your cluster and diagnose issues. If you run {es} as a service, the default location of the logs varies based on @@ -11,7 +11,7 @@ If you run {es} from the command line, {es} prints logs to the standard output (`stdout`). [discrete] -[[loggin-configuration]] +[[logging-configuration]] === Logging configuration IMPORTANT: Elastic strongly recommends using the Log4j 2 configuration that is shipped by default. @@ -304,6 +304,7 @@ The user ID is included in the `X-Opaque-ID` field in deprecation JSON logs. Deprecation logs can be indexed into `.logs-deprecation.elasticsearch-default` data stream `cluster.deprecation_indexing.enabled` setting is set to true. +[discrete] ==== Deprecation logs throttling :es-rate-limiting-filter-java-doc: {elasticsearch-javadoc}/org/elasticsearch/common/logging/RateLimitingFilter.html Deprecation logs are deduplicated based on a deprecated feature key diff --git a/docs/reference/shard-request-cache.asciidoc b/docs/reference/shard-request-cache.asciidoc new file mode 100644 index 000000000000..ec79dfb531bd --- /dev/null +++ b/docs/reference/shard-request-cache.asciidoc @@ -0,0 +1,134 @@ +[[shard-request-cache]] +=== The shard request cache + +When a search request is run against an index or against many indices, each +involved shard executes the search locally and returns its local results to +the _coordinating node_, which combines these shard-level results into a +``global'' result set. + +The shard-level request cache module caches the local results on each shard. +This allows frequently used (and potentially heavy) search requests to return +results almost instantly. The requests cache is a very good fit for the logging +use case, where only the most recent index is being actively updated -- +results from older indices will be served directly from the cache. + +You can control the size and expiration of the cache at the node level using the <>. + +[IMPORTANT] +=================================== + +By default, the requests cache will only cache the results of search requests +where `size=0`, so it will not cache `hits`, +but it will cache `hits.total`, <>, and +<>. + +Most queries that use `now` (see <>) cannot be cached. + +Scripted queries that use the API calls which are non-deterministic, such as +`Math.random()` or `new Date()` are not cached. +=================================== + +[discrete] +==== Cache invalidation + +The cache is smart -- it keeps the same _near real-time_ promise as uncached +search. + +Cached results are invalidated automatically whenever the shard refreshes to +pick up changes to the documents or when you update the mapping. In other +words you will always get the same results from the cache as you would for an +uncached search request. + +The longer the refresh interval, the longer that cached entries will remain +valid even if there are changes to the documents. If the cache is full, the +least recently used cache keys will be evicted. + +The cache can be expired manually with the <>: + +[source,console] +------------------------ +POST /my-index-000001,my-index-000002/_cache/clear?request=true +------------------------ +// TEST[s/^/PUT my-index-000001\nPUT my-index-000002\n/] + +[discrete] +==== Enabling and disabling caching + +The cache is enabled by default, but can be disabled when creating a new +index as follows: + +[source,console] +----------------------------- +PUT /my-index-000001 +{ + "settings": { + "index.requests.cache.enable": false + } +} +----------------------------- + +It can also be enabled or disabled dynamically on an existing index with the +<> API: + +[source,console] +----------------------------- +PUT /my-index-000001/_settings +{ "index.requests.cache.enable": true } +----------------------------- +// TEST[continued] + + +[discrete] +==== Enabling and disabling caching per request + +The `request_cache` query-string parameter can be used to enable or disable +caching on a *per-request* basis. If set, it overrides the index-level setting: + +[source,console] +----------------------------- +GET /my-index-000001/_search?request_cache=true +{ + "size": 0, + "aggs": { + "popular_colors": { + "terms": { + "field": "colors" + } + } + } +} +----------------------------- +// TEST[continued] + +Requests where `size` is greater than 0 will not be cached even if the request cache is +enabled in the index settings. To cache these requests you will need to use the +query-string parameter detailed here. + +[discrete] +==== Cache key + +A hash of the whole JSON body is used as the cache key. This means that if the JSON +changes -- for instance if keys are output in a different order -- then the +cache key will not be recognised. + +TIP: Most JSON libraries support a _canonical_ mode which ensures that JSON +keys are always emitted in the same order. This canonical mode can be used in +the application to ensure that a request is always serialized in the same way. + +[discrete] +==== Monitoring cache usage + +The size of the cache (in bytes) and the number of evictions can be viewed +by index, with the <> API: + +[source,console] +------------------------ +GET /_stats/request_cache?human +------------------------ + +or by node with the <> API: + +[source,console] +------------------------ +GET /_nodes/stats/indices/request_cache?human +------------------------ diff --git a/docs/reference/snapshot-restore/apis/restore-snapshot-api.asciidoc b/docs/reference/snapshot-restore/apis/restore-snapshot-api.asciidoc index 2f2c7fcd8ebd..89cd0f96915b 100644 --- a/docs/reference/snapshot-restore/apis/restore-snapshot-api.asciidoc +++ b/docs/reference/snapshot-restore/apis/restore-snapshot-api.asciidoc @@ -75,7 +75,7 @@ POST /_snapshot/my_repository/my_snapshot/_restore // tag::restore-prereqs[] * You can only restore a snapshot to a running cluster with an elected -<>. The snapshot's repository must be +<>. The snapshot's repository must be <> and available to the cluster. * The snapshot and cluster versions must be compatible. See diff --git a/docs/reference/snapshot-restore/take-snapshot.asciidoc b/docs/reference/snapshot-restore/take-snapshot.asciidoc index 711fcfe4cc48..1ae2258c7da8 100644 --- a/docs/reference/snapshot-restore/take-snapshot.asciidoc +++ b/docs/reference/snapshot-restore/take-snapshot.asciidoc @@ -46,7 +46,7 @@ taking snapshots at different time intervals. include::register-repository.asciidoc[tag=kib-snapshot-prereqs] * You can only take a snapshot from a running cluster with an elected -<>. +<>. * A snapshot repository must be <> and available to the cluster. diff --git a/docs/reference/transform/setup.asciidoc b/docs/reference/transform/setup.asciidoc index dab357546d93..3171086e43d6 100644 --- a/docs/reference/transform/setup.asciidoc +++ b/docs/reference/transform/setup.asciidoc @@ -11,7 +11,7 @@ To use {transforms}, you must have: -* at least one <>, +* at least one <>, * management features visible in the {kib} space, and * security privileges that: + diff --git a/docs/reference/upgrade/disable-shard-alloc.asciidoc b/docs/reference/upgrade/disable-shard-alloc.asciidoc index f69a67309525..4672ea65446b 100644 --- a/docs/reference/upgrade/disable-shard-alloc.asciidoc +++ b/docs/reference/upgrade/disable-shard-alloc.asciidoc @@ -5,7 +5,7 @@ starting to replicate the shards on that node to other nodes in the cluster, which can involve a lot of I/O. Since the node is shortly going to be restarted, this I/O is unnecessary. You can avoid racing the clock by <> of replicas before -shutting down <>: +shutting down <>: [source,console] --------------------------------------------------