Commit graph

403 commits

Author SHA1 Message Date
Liam Thompson
9a62dba53c
[DOCS] Remove remaining beta flags for RCS (#108201) 2024-05-03 09:12:37 +02:00
David Turner
f9c414c5e3
Fix up hot threads timeout docs (#107692)
The hot threads API does not support a `?master_timeout` parameter, and
the `?timeout` parameter is not an ack timeout and defaults to an
infinite wait. This commit fixes the incorrect docs.
2024-04-22 08:45:00 -04:00
Howard
dcb248a013
Add node stats effective watermark thresholds docs (#107668)
Relates https://github.com/elastic/elasticsearch/pull/107244
2024-04-21 15:31:11 +01:00
Liam Thompson
33a71e3289
[DOCS] Refactor book-scoped variables in docs/reference/index.asciidoc (#107413)
* Remove `es-test-dir` book-scoped variable

* Remove `plugins-examples-dir` book-scoped variable

* Remove `:dependencies-dir:` and `:xes-repo-dir:` book-scoped variables

- In `index.asciidoc`, two variables (`:dependencies-dir:` and `:xes-repo-dir:`) were removed.
- In `sql/index.asciidoc`, the `:sql-tests:` path was updated to fuller path
- In `esql/index.asciidoc`, the `:esql-tests:` path was updated idem

* Replace `es-repo-dir` with `es-ref-dir`

* Move `:include-xpack: true` to few files that use it, remove from index.asciidoc
2024-04-17 14:37:07 +02:00
Artem Prigoda
6a300509cd
Add metric for calculating index flush time excluding waiting on locks (#107196)
Add a new `total_time_excluding_waiting_on_lock metric` to the index flush stats that measures the flushing time excluding waiting on the flush lock. This metrics provides a more granular view on flush performance and without the overhead of flush throttling.

Resolves ES-7201
2024-04-12 15:04:08 +02:00
Volodymyr Krasnikov
0faac52ac4
Metric for rejected indexing primary operations (per document) (#107080)
* Fix number of rejected primary operations

* Update docs/changelog/107080.yaml

* Update test

* Add metric 'es.indexing.primary_operations.document.rejections.ratio' + test

* rm useless changelog

* update docs

* use -1 as a default version on unsupported version
2024-04-07 21:09:58 -07:00
David Turner
7ee63dfd51
Expand docs on reset desired balance API (#106921)
Explains its purpose and surrounding context a little more, including a
note that this should never be necessary (i.e. if you find you need it,
that's a bug).
2024-04-02 04:18:59 -04:00
Parker Timmins
e59dd0b60e
Add total size in bytes to doc stats (#106840) 2024-03-29 09:40:37 -05:00
Yang Wang
5632380ecd
[Doc] Trivial correction for shard allocator choice (#106216)
Relates: #105894
2024-03-12 18:34:22 +11:00
Ievgen Degtiarenko
5e52059947
Add allocation stats (#105894)
This change attempts to add allocation section to the node stats in
order to simplify unbalanced clusters debugging. It is required for
https://github.com/elastic/elasticsearch/pull/97561
2024-03-11 11:07:56 -04:00
Stef Nestor
18a509a18f
(DOC+) Node Stats fs.available reflects XFS quotas (#106085)
Moving https://github.com/elastic/elasticsearch/pull/103472 here.

---

👋 howdy, team!

Could we include "XFS quotas" as an example for "depending on OS or process level restrictions" for this doc's searchability for users to better understand how to investigate this potential lever's impact?

TIA!
2024-03-08 10:19:27 -05:00
Liam Thompson
52b027e44b
[DOCS] Fix bullet in get-desired-balance ref (#105819)
Fix formatting
2024-02-29 09:32:50 +01:00
Lorenzo Dematté
0e328dbfc8
DesiredNode: deprecate node_version field and make it optional (unused) in current parser (#104209)
Deprecated node_version field, made it optional(unused) in new parser
Added deprecation warning handler for mixed cluster
Split tests for old vs. current format
2024-01-29 14:09:45 +01:00
Ievgen Degtiarenko
70ecb12556
Add undesired shard count (#101426)
This change add undesired shard (ones that are allocated not on the desired
node) counts to the api output.
2023-11-02 09:44:48 +01:00
Simon Cooper
3f32affbb6
Add component info versions to node info in a pluggable way (#99631)
This adds a `ComponentVersionNumber` service interface for modules to provide version numbers for individual components to be reported inside node info. Initial implementations for `MlConfigVersion` and `TransformConfigVersion` are provided.
2023-09-21 17:08:43 +01:00
Simon Cooper
8726f1653f
Add IndexVersion to node info (#99515)
This adds index_version field to node info, so we can read it in bwc tests to determine the index version we are upgrading from
2023-09-14 10:09:51 +01:00
Yang Wang
ebe4fe9f15
[Doc] Add links to the new API key based remote cluster page (#99115)
This PR adds links to the new API key based remote cluster page in
multiple places.

Relates: #98330
2023-09-01 06:08:49 -04:00
Dianna Hohensee
a25e176692
Add node "roles" to allocation explain response (#98550)
Report node "roles" in the /_cluster/allocation/explain response.
Nodes with limited sets of roles may affect shard distribution in ways
users did not originally consider, so it is helpful to surface this
information along with node allocation decision explanations.
2023-08-23 08:30:35 -04:00
Yang Wang
b337f9b6f3
[Docs] Misc doc update for RCS 2.0 (#98472)
This PR adds docs for the following items: * Remote indices privileges *
Remote cluster network settings * Remote cluster security settings * New
privileges * New response field for RemoteInfo API

List of preview pages: * [Remote indices in defining
roles](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/defining-roles.html#roles-remote-indices-priv)
* [Remote indices in PutRole
API](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-api-put-role.html#security-api-put-role-request-body)
* [Remote cluster server SSL
settings](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-settings.html#_remote_cluster_server_api_key_based_model_tlsssl_settings)
* [Remote cluster client SSL
settings](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-settings.html#_remote_cluster_client_api_key_based_model_tlsssl_settings)
* [Remote cluster network
settings](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/modules-network.html#remote-cluster-network-settings)
and
[here](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/modules-network.html#common-network-settings)
* [Remote cluster credentials
setting](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/remote-clusters-settings.html)
* [New
privileges](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-privileges.html)
* [New response field for RemoteInfo
API](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/cluster-remote-info.html#cluster-remote-info-api-response-body)
2023-08-15 20:11:21 -04:00
Jim Ferenczi
a5d21ce800
Add the total dense vector count in the indices stats output (#98275)
This change adds the total dense vector count to the output of the indices stats.
This is useful for observability in order to track the number of indexed vectors
in a cluster.

---------

Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
2023-08-11 23:17:38 +09:00
Carlos Delgado
8e64359fb1
Change cluster stats synonyms keys (#98126) 2023-08-04 17:28:12 +02:00
Abdon Pijpelink
efc0cb5422
[DOCS] Node stats API: fix descriptions of 'cache_size' and 'cache_count' (#98092) 2023-08-03 09:45:22 +02:00
Carlos Delgado
e300319afb
Synonym Rules APIs docs (#98064) 2023-08-02 14:11:30 +02:00
Carlos Delgado
c0a99baef5
Add synonyms sets information to cluster stats (#97900) 2023-07-27 21:25:24 +02:00
David Turner
ddd4ba5e30
Fix docs for explaining unassigned shards (#97538)
Today the `current_node` parameter is given in several sample requests
illustrating how to explain an unassigned shard using the cluster
allocation explain API. This doesn't make sense, an unassigned shard has
no `current_node`. This commit removes the misleading parameter in these
cases.
2023-07-11 08:01:12 +01:00
David Turner
ac0c316764
Add note on troubleshooting laggy cancellations (#97485)
Today we document that tasks may not react to cancellations immediately,
but in practice it's surprising to users and kind of a bug if they run
for too long after being cancelled. This commit adds a little extra
detail about the information to collect to troubleshoot such a
situation.
2023-07-10 11:52:36 +01:00
Ievgen Degtiarenko
801eaba095
Fix delete-desired-balance doc (#96978)
This change adds a proper API description instead of one that was
accidentally copied from the other endpoint.
2023-06-27 10:12:15 +02:00
Pablo Alcantar Morales
508f3c4214
Marking Cluster _info API as experimental (#97020) 2023-06-22 17:59:44 +02:00
Volodymyr Krasnikov
7abe8cb974
Add repo throttle metrics to node stats api response (#96678)
* Add repo throttle metrics to node stats api response

* Update docs/changelog/96678.yaml

* Change x-content output structure

* Fix test after merge from main

* Follow PR comments

* minor fixes

* minor fixes 2

* Introduce new TransportVersion (V_8_500_010)

* Fix yaml test

* Follow PR comments

* Make stats datapoints human readable

* Follow common pattern for human readable output

* Bump up TransportVersion
2023-06-13 09:04:36 -07:00
Pablo Alcantar Morales
5a623ee3bf
Add script information to the cluster info endpoint (#96613)
Add a new target (`script`) to the `/_info` API. It consolidates all the script information from the cluster nodes and returns a summary at the cluster level (compared with `_nodes/stats/script` it lacks the `<node>` dimension).
2023-06-12 10:10:21 +02:00
Pablo Alcantar Morales
88419af5c0
fix incorrect definition of target (#96707) 2023-06-08 16:34:00 +02:00
Pablo Alcantar Morales
f5d47ccb0d
Add thread_pool information to the cluster info endpoint (#96407)
Add a new target (thread_pool) to the /_info API. It consolidates all the thread pools information from the cluster nodes and returns a summary at the cluster level (compared with _nodes/stats/thread_pool it lacks the <node> dimension)
2023-06-01 10:47:39 +02:00
Pablo Alcantar Morales
244da063ca
Add ingest information to the cluster info endpoint (#96328) 2023-05-26 09:28:12 +02:00
debadair
777598d602
[DOCS] Remove redirect pages (#88738)
* [DOCS] Remove manual redirects

* [DOCS] Removed refs to modules-discovery-hosts-providers

* [DOCS] Fixed broken internal refs

* Fixing bad cross links in ES book, and adding redirects.asciidoc[] back into docs/reference/index.asciidoc.

* Update docs/reference/search/point-in-time-api.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/setup/restart-cluster.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/sql/endpoints/translate.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update docs/reference/snapshot-restore/restore-snapshot.asciidoc

Co-authored-by: James Rodewig <james.rodewig@elastic.co>

* Update repository-azure.asciidoc

* Update node-tool.asciidoc

* Update repository-azure.asciidoc

---------

Co-authored-by: amyjtechwriter <61687663+amyjtechwriter@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Amy Jonsson <amy.jonsson@elastic.co>
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2023-05-24 12:32:46 +01:00
Pablo Alcantar Morales
64e32679c8
Generalise new cluster info endpoint (#96259) 2023-05-23 16:30:56 +02:00
Pablo Alcantar Morales
dc5d0546e9
New HTTP info (/_info/http) endpoint (#96198)
Adding a new endpoint under `_info/http`. This endpoint summarises the HTTP info of all the nodes into one big response, at cluster level. Compared with `_nodes/stats`, it lacks the nodes dimension.
2023-05-22 07:43:23 +02:00
Jean-Pierre Matsumoto
87b8f1cf73
Bad ref to 'node_id' parameter in Task Mgt doc (#90380)
* Bad ref to 'node_id' parameter in Task Mgt doc

Here in documentation:
https://www.elastic.co/guide/en/elasticsearch/reference/8.4/tasks.html#tasks-api-query-params

Name of parameter to filter node is `nodes`.

* Change 'node_id' into 'nodes' and add 'nodes' as a common param

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-05-03 11:55:29 +02:00
Martijn van Groningen
6566bb4075
Add global ordinal info to stats APIs. (#94500)
This change adds:
* Total global ordinal build time for all fields and per field.
* Max shard value count per field. The value count is per shard and of the shard with the highest count. Reporting value on index level or across indices is too expensive to report or keep track of.

This is added to common stats, which
is exposed in several stats APIs.

The following api call:

```
GET /_nodes/stats?filter_path=nodes.*.indices.fielddata&fields=key,key2
```

Returns:

```
{
    "nodes": {
        "pcMNy4GsQ8ef6Rw-bI2EFg": {
            "indices": {
                "fielddata": {
                    "memory_size_in_bytes": 2552,
                    "evictions": 0,
                    "fields": {
                        "key2": {
                            "memory_size_in_bytes": 1320
                        },
                        "key": {
                            "memory_size_in_bytes": 1232
                        }
                    },
                    "global_ordinals": {
                        "build_time_in_millis": 8,
                        "fields": {
                            "key2": {
                                "build_time_in_millis": 4,
                                "shard_max_value_count": 4
                            },
                            "key": {
                                "build_time_in_millis": 4,
                                "shard_max_value_count": 4
                            }
                        }
                    }
                }
            }
        }
    }
}
```
2023-04-24 10:45:27 +02:00
Ievgen Degtiarenko
c2c0ced9b1
Reset desired balance (#94525)
This introduces an endpoint to reset the desired balance.
It could be used if computed balance diverged from the actual one a lot 
to start a new computation from the current state.
2023-04-20 08:03:48 +02:00
David Turner
4ef9965d47
Report transport message size per action (#94543)
Adds to the transport stats a histogram of transport message sizes for
each transport action.

Closes https://github.com/elastic/elasticsearch/issues/88151
2023-03-28 12:05:18 -04:00
Simon Cooper
dc6ccbbe08
Add transport_version to node info JSON (#94669) 2023-03-24 09:00:01 +00:00
Ievgen Degtiarenko
3221d1d199
Make relocatingNodeIsDesired nullable when not relocating (#94279) 2023-03-20 12:49:16 +01:00
Ievgen Degtiarenko
21adad38a3
Add data tier to /_internal/desired-balance (#94496) 2023-03-20 10:44:55 +01:00
David Turner
e43e7c2f4a
Improve transport stats histogram (#93598)
- omits empty buckets at the start and end of the histogram
- includes human-readable representation of the bucket boundaries if `?human` specified
2023-03-17 18:01:58 -04:00
Ievgen Degtiarenko
90d017ae09
Add cluster_info to the GET /_internal/desired-balance endpoint output (#94272) 2023-03-06 13:57:55 +01:00
Abdon Pijpelink
c5b1d997d1
[DOCS] Add 'total' object for io_stats in nodes stats response (#93854) 2023-02-23 13:08:42 +01:00
David Turner
a7e2430b79
Include node ID in balance API (#93823)
Today we report node stats by name, but the desired nodes work in terms
of node IDs. This commit adds a mapping between node name and ID to make
the output easier to interpret.
2023-02-15 08:39:59 -05:00
Pooya Salehi
ee56ea8c82
Clarify wait_for_completion doc in the Task API (#93754) 2023-02-13 18:29:14 +01:00
David Turner
9c8c9528ad
Add cluster stats re. snapshot activity (#93680)
Shows how many ongoing snapshots/clones/deletions/etc. there are, and
summarises the shard-level status too for progress tracking.
2023-02-13 07:48:14 +00:00
Ievgen Degtiarenko
ab5ae88919
Expose forecasted and actual disk usage per tier and node (#93497) 2023-02-06 13:34:18 +01:00