elasticsearch/docs/reference
Lee Hinman 5fca68a155
Add telemetry for data tiers (#63031)
This commit adds telemetry for our data tier formalization. This telemetry helps determine the
topology of the cluster with regard to the content, hot, warm, & cold tiers/roles.

An example of the telemetry looks like:

```
GET /_xpack/usage?human
{
  ...
  "data_tiers" : {
    "available" : true,
    "enabled" : true,
    "data_warm" : {
      ...
    },
    "data_cold" : {
      ...
    },
    "data_content" : {
      "node_count" : 1,
      "index_count" : 6,
      "total_shard_count" : 6,
      "primary_shard_count" : 6,
      "doc_count" : 71,
      "total_size" : "59.6kb",
      "total_size_bytes" : 61110,
      "primary_size" : "59.6kb",
      "primary_size_bytes" : 61110,
      "primary_shard_size_avg" : "9.9kb",
      "primary_shard_size_avg_bytes" : 10185,
      "primary_shard_size_median" : "8kb",
      "primary_shard_size_median_bytes" : 8254,
      "primary_shard_size_mad" : "7.2kb",
      "primary_shard_size_mad_bytes" : 7391
    },
    "data_hot" : {
       ...
    }
  }
}
```

The fields are as follows:

- node_count :: number of nodes with this tier/role
- index_count :: number of indices on this tier
- total_shard_count :: total number of shards for all nodes in this tier
- primary_shard_count :: number of primary shards for all nodes in this tier
- doc_count :: number of documents for all nodes in this tier
- total_size_bytes :: total number of bytes for all shards for all nodes in this tier
- primary_size_bytes :: number of bytes for all primary shards on all nodes in this tier
- primary_shard_size_avg_bytes :: average shard size for primary shard in this tier
- primary_shard_size_median_bytes :: median shard size for primary shard in this tier
- primary_shard_size_mad_bytes :: [median absolute deviation](https://en.wikipedia.org/wiki/Median_absolute_deviation) of shard size for primary shard in this tier

Relates to #60848
2020-10-01 08:35:10 -06:00
..
aggregations [DOCS] Add experimental tag to inference processor and bucket aggregation (#63023) 2020-09-30 07:20:38 -07:00
analysis [DOCS] Clarify that v2.0+ hyphenation files aren't supported (#60579) (#63072) 2020-09-30 09:28:23 -04:00
autoscaling Autoscaling decision return absolute capacity (#61575) 2020-09-18 20:44:20 +02:00
cat [DOCS] Replace Wikipedia links with attribute (#61171) 2020-08-17 09:44:24 -04:00
ccr Removing erroneous screen capture. (#62561) 2020-09-17 12:22:37 -04:00
cluster Add index.routing.allocation.prefer._tier setting (#62589) 2020-09-18 14:49:59 -06:00
commands [DOCS] Fix elasticsearch-croneval chunking (#63008) 2020-09-29 09:53:20 -04:00
data-streams Add "synthetics-*-*" templates for synthetics fleet data (#62193) 2020-09-14 16:37:21 -06:00
docs Add "synthetics-*-*" templates for synthetics fleet data (#62193) 2020-09-14 16:37:21 -06:00
eql [DOCS] EQL: date_nanos timestamp is not supported (#63101) 2020-09-30 17:31:24 -04:00
graph [DOCS] Document xpack.graph.enabled setting (#60073) 2020-08-21 12:37:05 -04:00
high-availability [DOCS] Replace Wikipedia links with attribute (#61171) 2020-08-17 09:44:24 -04:00
how-to [DOCS] Document shard sizing guide (#61942) 2020-09-28 09:24:40 -04:00
ilm [DOCS] Fix link to Filebeat docs (#62519) 2020-09-17 09:24:50 -04:00
images SQL: Update documentation on Tableau integration (#60890) 2020-08-17 12:10:15 +02:00
index-modules [DOCS] Document shard sizing guide (#61942) 2020-09-28 09:24:40 -04:00
indices Unmute rollover docs test (#62603) 2020-09-18 10:51:44 +02:00
ingest Add country_name to the default properties of geoip ingest processor (#62915) 2020-09-30 14:06:51 -05:00
licensing [DOCS] Swap [float] for [discrete] (#60124) 2020-07-23 11:48:22 -04:00
mapping Remove TypeFieldMapper (#62838) 2020-09-30 15:47:29 +01:00
migration Remove mapping boost parameter entirely (#62639) 2020-09-23 14:28:59 +01:00
ml [ML] Implement AucRoc metric for classification (#60502) 2020-09-30 08:56:23 +02:00
modules [DOCS] Remove duplicate disk.threshold_enabled setting. (#62924) 2020-09-29 08:58:46 -04:00
monitoring [DOCS] Add realm limitations for monitoring clusters (#62714) 2020-09-22 09:35:18 -07:00
query-dsl Search - add case insensitive flag for "term" family of queries (#61596) 2020-09-18 17:17:08 +01:00
release-notes [DOCS] Swap [float] for [discrete] (#60124) 2020-07-23 11:48:22 -04:00
repositories-metering-api Add repositories metering API (#60371) 2020-09-08 10:44:54 +02:00
rest-api Add telemetry for data tiers (#63031) 2020-10-01 08:35:10 -06:00
rollup [DOCS] Fix typo in rollup groups docs (#62269) (#62315) 2020-09-14 10:42:47 -04:00
scripting [DOCS] Fix ingest script compilation rate and cache size (#61468) 2020-08-24 10:26:38 -04:00
search Docs: Support """ in console-response (#62876) 2020-09-28 10:49:12 -04:00
searchable-snapshots/apis Remove REST APIs documentation for experimental Searchable Snapshot APIs (#62217) 2020-09-10 15:46:41 +02:00
settings Update ilm-settings.asciidoc 2020-09-25 16:25:00 +03:00
setup [DOCS] Fix typo in JVM options 2020-08-27 08:42:17 -04:00
slm [DOCS] Fix typo in SLM docs (#62591) 2020-09-18 12:31:05 -04:00
snapshot-restore Remove Outdated Snapshot Docs (#61684) 2020-08-31 15:52:49 +02:00
sql SQL: Implement FORMAT function (#55454) 2020-09-21 15:32:11 +02:00
transform [DOCS] Adds limitation item about using scripts in transforms (#63021) 2020-09-30 15:16:10 +02:00
upgrade [DOCS] Note remote reindex is not fwd compatible (#60425) 2020-07-30 09:08:59 -04:00
vectors [DOCS] Update my-index examples (#60132) 2020-07-27 14:46:39 -04:00
aggregations.asciidoc [DOCS] Swap [float] for [discrete] (#60124) 2020-07-23 11:48:22 -04:00
analysis.asciidoc [DOCS] Swap [float] for [discrete] (#60124) 2020-07-23 11:48:22 -04:00
api-conventions.asciidoc [DOCS] Fix elasticsearch-croneval chunking (#63008) 2020-09-29 09:53:20 -04:00
cat.asciidoc AwaitsFix for #51619 2020-08-06 09:59:56 +01:00
cluster.asciidoc Password-protected Keystore Feature Branch PR (#51123) 2020-01-27 19:51:39 -05:00
data-rollup-transform.asciidoc [DOCS] Changes level offset of transform pages (#60066) 2020-07-22 10:55:48 -07:00
docs.asciidoc [DOCS] Remove heading offsets for REST APIs (#44568) 2019-07-19 14:35:36 -04:00
frozen-indices.asciidoc [DOCS] Replace twitter dataset in docs (#60604) 2020-08-03 12:49:56 -04:00
getting-started.asciidoc [DOCS] Swap [float] for [discrete] (#60124) 2020-07-23 11:48:22 -04:00
glossary.asciidoc [DOCS] Fix hyphenation for "time series" (#61472) 2020-08-24 10:34:41 -04:00
gs-index.asciidoc [DOCS] Adding index file for GS "mini book". 2017-07-18 13:44:08 -07:00
high-availability.asciidoc [DOCS] Add docs for designing resilient clusters (#47233) 2020-06-05 11:48:44 -04:00
how-to.asciidoc [DOCS] Document shard sizing guide (#61942) 2020-09-28 09:24:40 -04:00
index-extra-title-page.html [DOCS] Add index-extra-title-page.html for direct HTML migration (#50189) 2019-12-13 12:44:12 -05:00
index-modules.asciidoc [DOCS] Update CCR docs to focus on Kibana (#60555) 2020-08-17 15:36:54 -04:00
index.asciidoc [DOCS] Combine Search your data files (#61477) 2020-08-24 11:22:56 -04:00
index.x.asciidoc [DOCS] Removes redundant index.asciidoc files (#30707) 2018-05-18 11:05:40 -07:00
indices.asciidoc [DOCS] Split delete index template API docs (#62074) (#62168) 2020-09-09 10:13:13 -04:00
ingest.asciidoc [DOCS] Update my-index examples (#60132) 2020-07-27 14:46:39 -04:00
intro.asciidoc [DOCS] Fix hyphenation for "time series" (#61472) 2020-08-24 10:34:41 -04:00
links.asciidoc [DOCS] Fix security links in machine learning APIs (#60098) 2020-07-23 12:14:56 -07:00
mapping.asciidoc Remove a redundant section on field data types. (#61821) 2020-09-02 15:00:01 -07:00
query-dsl.asciidoc [DOCS] Add redirects for wildcard and constant keyword (#61815) 2020-09-01 15:32:35 -04:00
redirects.asciidoc [DOCS] Fix elasticsearch-croneval chunking (#63008) 2020-09-29 09:53:20 -04:00
release-notes.asciidoc [DOCS] Adds placeholders for v8 highlights, breaking changes, release notes (#38641) 2019-02-08 15:48:42 -08:00
scripting.asciidoc [DOCS] Swap [float] for [discrete] (#60124) 2020-07-23 11:48:22 -04:00
search.asciidoc [DOCS] Display point in time API docs (#61527) 2020-08-25 11:03:41 -04:00
setup.asciidoc [DOCS] http -> https, remove outdated plugin docs (#60380) 2020-07-31 15:58:38 -04:00
upgrade.asciidoc [DOCS] Add tip for upgrade assistant (#55027) 2020-05-05 09:49:22 -04:00