The addition of the logger requires several updates to tests to deal with the possible warning, or muting if there is not way to specify an allowed (but not mandatory) warning
👋 howdy, team!
1. Related to https://github.com/elastic/dev/issues/2631, highlights customers are usually seeking `heap.percent` instead of `ram.percent`
2. Aligns the claimed "(Default)" columns in doc to what returned for v8.15.1 test cluster
This PR adds a count of currently unassigned primary shards to both the
`/_cat/health` and `/_cluster/health` endpoints. This is to aid cluster
administrators in estimating the time remaining for a cluster to go from
RED to YELLOW status as per enchancement request #111727.
Tests and doc updates are in place with this PR and manual testing with
`./gradlew run` has been conducted on the endpoints to ensure correct
output.
## Known Limitations * Testing * Due to limitations in the YAML REST
test framework skip functionality, YAML REST tests for this endpoint are
disabled when running a mixed version cluster by using a cluster version
number synthetic feature to skip when any member of the cluster is not
at a version greater than when this change is due to be introduced
Handle the "output memory allocator bytes" field if and only if it is present in the model size stats, as reported by the C++ backend.
This PR _must_ be merged prior to the corresponding `ml-cpp` one, to keep CI tests happy.
* Add SparseVectorStats
* Update to use mappings in engine
* Update to be unique to primary shards
* Fix doc
* Fix null error in test
* Cleanup
* fix yaml
* remove comment
* add version to yaml
* Revert whitespace changes to stats doc
* fix yml test
* Checkstyle
* Fix NPE in test
* Update docs/changelog/108793.yaml
* Add link to sparse_vector field type in docs
* PR feedback
* Flesh out test a bit more
* PR feedback - alphabetize placement in docs
* Fix doc change
* Remove `es-test-dir` book-scoped variable
* Remove `plugins-examples-dir` book-scoped variable
* Remove `:dependencies-dir:` and `:xes-repo-dir:` book-scoped variables
- In `index.asciidoc`, two variables (`:dependencies-dir:` and `:xes-repo-dir:`) were removed.
- In `sql/index.asciidoc`, the `:sql-tests:` path was updated to fuller path
- In `esql/index.asciidoc`, the `:esql-tests:` path was updated idem
* Replace `es-repo-dir` with `es-ref-dir`
* Move `:include-xpack: true` to few files that use it, remove from index.asciidoc
👋 howdy, team!
Will you kindly consider adding `shards` as an alias to
`shard_stats.total_count` column for CAT Nodes to match its naming from
CAT Allocation? (The tests returned clean without changes which I hadn't
expected, so please let me know if I missed something.)
**Example**: To avoid running the default CAT Nodes & CAT Allocation
separately, you can run CAT Nodes
```
GET _cat/nodes?v&s=master,name&h=name,id,master,node.role,cpu,heap.percent,disk.*,sstc,uptime
```
Where `sstc` is `shards` from CAT Allocation. This is a 👶 API (+ its
doc) change to make the output more intuitive.
In #96994 we added a column for node roles to the `GET _cat/allocation`
API but left it hidden by default to avoid changing behaviour. In fact
it's ok to change the behaviour of the CAT APIs since they're only
intended for human consumption, and it's awfully useful to see the node
roles in this API response, so this commit makes this column display by
default.
* Add 'dataset' size to cat indices and cat shards
This adds the `dataset` computed size for the `/_cat/indices` and `/_cat/shards` APIs. This new
column is reported by default.
Resolves#95092
This change adds the total dense vector count to the output of the indices stats.
This is useful for observability in order to track the number of indexed vectors
in a cluster.
---------
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
Now that we have the estimated field mappings heap overhead
in nodes stats, we can refer to them in the guide for sizing
data nodes appropriately.
Relates to #86639
The ingest attachment processor is currently available as a plugin. This
commit moves the processor to the default distribution so it is always
available.
* correct way of getting node heap size
in [[shard-count-recommendation]], we explain that the number of shards should be at most 20 shards per GB of heap.
but the command to get relevant heap size should be _cat/nodes?v=true&h=heap.max and not _cat/nodes?v=true&h=heap.current . The latter gives the current memory consumption, which is alway moving. Here we need to consider the max allocated heap size (-Xmx)
* Adds heap.max to valid columns
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Removing the mock nio transport and replacing its usage with the netty transport to make tests
with a more realistic transport implementation. This way improves the real world coverage for
the Netty transport, makes our tests more realistic and saves lots of code.
In particular, coverage on the rather complicated throttling/chunking in the netty message handler
is really ice to have.
The downside of this change is that we lose the slow transport thread warnings that the mock transport
outputs. This isn't a big deal these days in my opinion as we have slow logging in other places
now that makes up for this (we didn't when initially adding the slow logging) and that contains
far more detailed information on what exactly was slow.
Other than that, the mock transport does not come with any features we don't also have in the Netty
transport at this point.
Today we indicate that the `unassigned.reason` field in various APIs
indicates the reason why a shard is unassigned. This isn't really true,
it tells you some information about the event that caused the shard to
_become_ unassigned (or which most recently changed its routing table
entry while remaining unassigned) but tells you almost nothing about why
the shard _is now_ unassigned and how to fix it. That's what the
allocation explain API is for. This commit clarifies this point in the
docs.
Closes#80892
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.
Relates to #79309, #31619
We document that `GET /_index_template/...` accepts a comma-separated
list of template names but in fact today this API accepts only a single
name or pattern. Likewise `GET /_cat/templates/...` (at least it didn't
until #78829 but that's not released yet). This commit fixes the docs to
indicate these APIs accept only a single template name and also adds
some extra validation to reject requests containing a `,` since such a
request cannot match any actual templates.
It also adjusts `GET /_cat/templates` to use the filtering built into
`TransportGetComposableIndexTemplateAction` rather than retrieving all
templates and then filtering them on the coordinating node.
PRs #73062 and #73043 repurposed the `alias` anchor for a new guide for index
and data stream aliases. Previously, this anchor was used for our field alias
documentation.
Repurposing the anchor has caused continuity errors for users selecting
different versions of the ES docs. It could also cause confusion for users with
a `/current/` link to the `alias` page.
This updates the anchor for the alias guide and adds a redirect page to
disambiguate the `alias` anchor.
It also fixes a bread crumb issue for redirects following the 'Modifying your
Data' redirect page.
Closes#77034.
Adds support for the include_unloaded_segments flag in node stats, which helps with understanding resource usage of
shared_cache-style searchable snapshots on a per-node basis.
Refactoring of cat transform to show more relevant information. The current cat transform shows a
lot of configuration details, however cat should show operationally useful information. This PR
changes the defaults and also adds when transform did a search last.
