This adds some more counts for dense_vector field mapping stats. This
allows for seeing the number of mappings with a given element type,
similarity, or index type.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Add remote cluster stats to _cluster/stats
* Implement remote cluster stats polling
* Add docs for the include_remotes part
(cherry picked from commit b26d81c713)
* Note in docs about incorrect IO stats when running in docker
* Update docs/reference/cluster/nodes-stats.asciidoc
* Requested PR changes to wording
* Update docs/reference/cluster/nodes-stats.asciidoc
---------
Co-authored-by: David Turner <david.turner@elastic.co>
This PR adds a count of currently unassigned primary shards to both the
`/_cat/health` and `/_cluster/health` endpoints. This is to aid cluster
administrators in estimating the time remaining for a cluster to go from
RED to YELLOW status as per enchancement request #111727.
Tests and doc updates are in place with this PR and manual testing with
`./gradlew run` has been conducted on the endpoints to ensure correct
output.
## Known Limitations * Testing * Due to limitations in the YAML REST
test framework skip functionality, YAML REST tests for this endpoint are
disabled when running a mixed version cluster by using a cluster version
number synthetic feature to skip when any member of the cluster is not
at a version greater than when this change is due to be introduced
Adds to the `GET _cluster/stats` endpoint information about the snapshot
repositories in use, including their types, whether they are read-only
or read-write, and for Azure repositories the kind of credentials in
use.
* (Doc+) Link API to parent Doc part1
---------
Co-authored-by: shainaraskas <shaina.raskas@elastic.co>
Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>
This change returns the total number of fields at the segment level,
allowing for a more accurate estimate of the memory used by Lucene. The
new estimate is expected to be closer to the actual memory usage than
the current estimate using the index-level field count, due to the
non-trivial overhead incurred by each Lucene segment. Two new fields are
introduced: total_segment_fields, which is the total number of fields at
the segment level, and average_fields_per_segment. The overhead per
field in segments with fewer fields is larger than in segments with many
fields.
* Add SparseVectorStats
* Update to use mappings in engine
* Update to be unique to primary shards
* Fix doc
* Fix null error in test
* Cleanup
* fix yaml
* remove comment
* add version to yaml
* Revert whitespace changes to stats doc
* fix yml test
* Checkstyle
* Fix NPE in test
* Update docs/changelog/108793.yaml
* Add link to sparse_vector field type in docs
* PR feedback
* Flesh out test a bit more
* PR feedback - alphabetize placement in docs
* Fix doc change
Change the ingest byte stats to always be returned
whether or not they have a value of 0. Add human readable
form of byte stats. Update docs to reflect changes.
Current ingest byte stat fields could easily be confused.
Add more descriptive name to make it clear that they do not
count all docs processed by the pipeline.
Add ingested_in_bytes and produced_in_bytes stats to pipeline ingest stats.
These track how many bytes are ingested and produced by a given pipeline.
For efficiency, these stats are recorded for the first pipeline to process a
document. Thus, if a pipeline is called as a final pipeline after a default pipeline,
as a pipeline processor, and after a reroute request, a document will not
contribute to the stats for that pipeline. If a given pipeline has 0 bytes recorded
for both of these stats, due to not being the first pipeline to run any doc, these
stats will not appear in the pipeline's entry in ingest stats.
The hot threads API does not support a `?master_timeout` parameter, and
the `?timeout` parameter is not an ack timeout and defaults to an
infinite wait. This commit fixes the incorrect docs.
* Remove `es-test-dir` book-scoped variable
* Remove `plugins-examples-dir` book-scoped variable
* Remove `:dependencies-dir:` and `:xes-repo-dir:` book-scoped variables
- In `index.asciidoc`, two variables (`:dependencies-dir:` and `:xes-repo-dir:`) were removed.
- In `sql/index.asciidoc`, the `:sql-tests:` path was updated to fuller path
- In `esql/index.asciidoc`, the `:esql-tests:` path was updated idem
* Replace `es-repo-dir` with `es-ref-dir`
* Move `:include-xpack: true` to few files that use it, remove from index.asciidoc
Add a new `total_time_excluding_waiting_on_lock metric` to the index flush stats that measures the flushing time excluding waiting on the flush lock. This metrics provides a more granular view on flush performance and without the overhead of flush throttling.
Resolves ES-7201
* Fix number of rejected primary operations
* Update docs/changelog/107080.yaml
* Update test
* Add metric 'es.indexing.primary_operations.document.rejections.ratio' + test
* rm useless changelog
* update docs
* use -1 as a default version on unsupported version
Explains its purpose and surrounding context a little more, including a
note that this should never be necessary (i.e. if you find you need it,
that's a bug).
Moving https://github.com/elastic/elasticsearch/pull/103472 here.
---
👋 howdy, team!
Could we include "XFS quotas" as an example for "depending on OS or process level restrictions" for this doc's searchability for users to better understand how to investigate this potential lever's impact?
TIA!
Deprecated node_version field, made it optional(unused) in new parser
Added deprecation warning handler for mixed cluster
Split tests for old vs. current format
This adds a `ComponentVersionNumber` service interface for modules to provide version numbers for individual components to be reported inside node info. Initial implementations for `MlConfigVersion` and `TransformConfigVersion` are provided.
Report node "roles" in the /_cluster/allocation/explain response.
Nodes with limited sets of roles may affect shard distribution in ways
users did not originally consider, so it is helpful to surface this
information along with node allocation decision explanations.
This change adds the total dense vector count to the output of the indices stats.
This is useful for observability in order to track the number of indexed vectors
in a cluster.
---------
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
