* [DOCS] Typo in Search speed
The PR https://github.com/elastic/elasticsearch/pull/89782 introduced some broken tags to leak in the text
* Fix tags
* Make all headings discrete
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
This commit removes the experimental tag from kNN search docs and makes some
docs improvements:
* Add a prominent warning about memory usage in the kNN search guide
* Link to the performance tuning guide from the main guide
* Clarify the memory requirements section in the tuning guide
Now that we have the estimated field mappings heap overhead
in nodes stats, we can refer to them in the guide for sizing
data nodes appropriately.
Relates to #86639
So that they are visible in NodeIndicesStats only at the node and index (but not shard) levels. Also visible in the _cat/nodes table. And make an exact count yaml REST test.
Introduce max headroom settings for the low, high, and flood disk watermark stages, similar to the existing max headroom setting for the flood stage of the frozen tier. Introduce new max headrooms in HealthMetadata and in ReactiveStorageDeciderService. Add multiple tests in DiskThresholdDeciderUnitTests, DiskThresholdDeciderTests and DiskThresholdMonitorTests. Moreover, addition & subtraction for ByteSizeValue, and min.
We encountered a case where a substantial fraction of the heap usage was
due to per-segment-per-field `FieldInfo` objects, particularly
`FieldInfo#name`. This commit adds a note to the sizing docs about this
overhead.
When using LVM or software raid on Linux the kernel, or specific
distribution rules, may use higher ergonomic defaults for the
readahead of resulting block device(s). This can adversely affect
search performance due to high page cache thrashing, in search
heavy scenarios when mmap is involved.
Add a clarification section in the docs raising awareness about this
value and preferring the lower default.
This guidance does not apply any longer.
The overhead per shard has been significantly reduced in recent versions
and removed rule of thumb will be too pessimistic in many if not
most cases and might be too optimistic in other specific ones.
=> Replace guidance with rule of thumb per field count on data nodes and
rule of thumb by index count (which is far more relevant nowadays than
shards) for master nodes.
relates #77466
Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: Henning Andersen <33268011+henningandersen@users.noreply.github.com>
* correct way of getting node heap size
in [[shard-count-recommendation]], we explain that the number of shards should be at most 20 shards per GB of heap.
but the command to get relevant heap size should be _cat/nodes?v=true&h=heap.max and not _cat/nodes?v=true&h=heap.current . The latter gives the current memory consumption, which is alway moving. Here we need to consider the max allocated heap size (-Xmx)
* Adds heap.max to valid columns
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Today we don't really say anything about the requirements for the data
path in terms of correctness, and we specifically say to avoid NFS for
performance reasons. This isn't wholly accurate: some NFS
implementations work just fine. This commit documents a more balanced
position on local vs remote storage.
Asciidoc source blocks are to be delimited with four dashes. This adds missing delimiters, and updates some that contained only three dashes. It matters for parsing purposes.
Today the _Size your shards_ docs focus on shard size and count, but in
fact index count and field count are also important. This commit expands
these docs a bit to cover this observation too.
Today we indicate that the `unassigned.reason` field in various APIs
indicates the reason why a shard is unassigned. This isn't really true,
it tells you some information about the event that caused the shard to
_become_ unassigned (or which most recently changed its routing table
entry while remaining unassigned) but tells you almost nothing about why
the shard _is now_ unassigned and how to fix it. That's what the
allocation explain API is for. This commit clarifies this point in the
docs.
Closes#80892
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
* [DOCS] Update ES quick start for security ON by default
* Remove code.asciidoc, which is part of the overall doc build now
* Update node names for cleanup
* Add note with links to tools
* Add --net elastic network
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
This PR changes uses of transient cluster settings to
persistent cluster settings.
The PR also deprecates the transient settings usage.
Relates to #49540
In production, we recommend you run {es} on a dedicated host or as a primary service.
This adds that best practice to our setup documentation.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
The current shrink API snippet doesn't show you how to remove replicas or reduce primary shards.
Rather than duplicate those instructions from the shrink API docs, this removes the snippet. A link to the shrink API and shrink ILM action docs is already provided.
It also updates a delete index API snippet to avoid wildcards. Wildcard expansion for the delete index API is disabled by default in 8.0.
* Mention match_only_text in disk usage docs
Previously we explained how to manually disable norms, freqs, and positions. We
now have a ready-made solution in the new `match_only_text` field type.
* Fixing typo and minor grammar changes
Co-authored-by: Adam Locke <adam.locke@elastic.co>
On the "Size your shards" page, the shard size recommendation assumes a time
series use case. Similarly, users shouldn't count nearly empty and rarely used
Enterprise Search system indices against the recommended shard count limit.
Closes#76328.
Changes:
* Revises the size your shards guide to use a 50GB shard guideline. This better aligns with our default in the ILM policy UI.
* Updates the language to indicate that the 50GB shard guideline is not a hard limit. Larger shards may work depending on the network and use case.
Reverts some changes added in #71367.
