elasticsearch/docs/reference/release-notes/8.6.0.asciidoc

[[release-notes-8.6.0]]
== {es} version 8.6.0

Also see <<breaking-changes-8.6,Breaking changes in 8.6>>.

[[known-issues-8.6.0]]
[float]
=== Known issues

include::8.4.0.asciidoc[tag=file-based-settings-deadlock-known-issue]

include::8.0.0.asciidoc[tag=jackson-filtering-bug]

include::8.4.0.asciidoc[tag=ingest-processor-log4j-cluster-instability-known-issue]

// tag::reconciliation-imbalance-known-issue[]
* Shard rebalancing may temporarily unbalance cluster
+
From 8.6.0 onwards the default shard rebalancing algorithm will compute the
final desired balance and then make shard movements to reconcile the current
state of the cluster with the desired state. However the order in which the
shard movements take place may be skewed towards certain nodes, causing the
cluster to become temporarily unbalanced while the reconciliation is ongoing.
As always, once a node reaches a disk watermark it will not accept any
additional shards, but this skew may result in nodes reaching their disk
watermarks more often than expected in normal operation. Once the
reconciliation process completes, the cluster will be balanced again.
+
To avoid this problem, upgrade to 8.8.0 or later.
// end::reconciliation-imbalance-known-issue[]

[[bug-8.6.0]]
[float]
=== Bug fixes

Aggregations::
* GeoBoundsAggregations reject sub aggregations {es-pull}91073[#91073] (issue: {es-issue}91072[#91072])

Allocation::
* Avoid NPE when disassociateDeadNodes is executed for a node present in the desired balance {es-pull}91659[#91659]
* Check `NodesShutdownMetadata` type before assuming restart {es-pull}90807[#90807]

Authentication::
* Fix time unit for connection request timeout of JWKs reload {es-pull}92080[#92080]
* Improve performance for role mapping with DNs {es-pull}92074[#92074]
* Improve robustness of `JwkSet` reloading {es-pull}92081[#92081]
* Support stored authentication headers prior to version 6.7 {es-pull}92221[#92221]

Authorization::
* Make adding auth info to REST responses more robust {es-pull}92168[#92168]
* Security remove datemath special handling {es-pull}91047[#91047]

Cluster Coordination::
* Fix `TransportMasterNodeAction` holding a CS reference needlessly {es-pull}90724[#90724] (issue: {es-issue}89220[#89220])
* Include last-committed data in publication {es-pull}92259[#92259] (issue: {es-issue}90158[#90158])
* Unsafe bootstrap memory optimization {es-pull}92493[#92493]

EQL::
* EQL sequences: support join on multi-values {es-pull}89965[#89965]

Graph::
* Fix potential issue with graph api's timed out field in response {es-pull}91006[#91006]

Health::
* Don't account for the unassigned reason when diagnosing NO_VALID_SHARD_COPY {es-pull}92416[#92416]
* Fix NPE when evaluating the disk health for non-data nodes {es-pull}92643[#92643]
* Use https in the short URLs for the `shards_availability` indicator {es-pull}92310[#92310]

Indices APIs::
* Trigger index settings providers when updating component templates {es-pull}91615[#91615] (issue: {es-issue}91592[#91592])

Infra/Core::
* Check reserved state in Metadata.isGlobalStateEquals {es-pull}92124[#92124]
* Datastream unavailable exception metadata {es-pull}91461[#91461]
* Fix `BytesRefArray` on append empty `BytesRef` {es-pull}91364[#91364]
* Fix index expression options for requests with a single name or pattern {es-pull}91231[#91231]
* Force init of Unbox in log4j {es-pull}92377[#92377] (issue: {es-issue}91964[#91964])
* In file based settings, wait until security index is ready for role mappings {es-pull}92173[#92173] (issue: {es-issue}91939[#91939])
* Index expression exclusions never trigger "not found" {es-pull}90902[#90902]
* Update error states from inside the main state executor {es-pull}90346[#90346] (issue: {es-issue}90337[#90337])

Infra/Scripting::
* Fix compile with hex literals ending with d/f {es-pull}91501[#91501] (issue: {es-issue}88614[#88614])

Ingest Node::
* Fixing a race condition in `EnrichCoordinatorProxyAction` that can leave an item stuck in its queue {es-pull}90688[#90688] (issue: {es-issue}90598[#90598])

Machine Learning::
* Copy more settings when creating DF analytics destination index {es-pull}91546[#91546] (issue: {es-issue}89795[#89795])
* Fix for 'No statistics' error message {ml-pull}2410[#2410]
* Fix for 'No counts available' error message {ml-pull}2414[#2414]
* Guard against input sequences that are too long for Question Answering models {es-pull}91924[#91924]
* Improve performance of closing files before spawning {ml-pull}2424[#2424]
* Skip remote clusters when performing up front privileges validation for datafeeds {es-pull}91895[#91895] (issue: {es-issue}87832[#87832])
* Support fields with commas in data frame analytics `analyzed_fields` {es-pull}91710[#91710] (issue: {es-issue}72541[#72541])
* Validate rule filters are present on open anomaly detection api {es-pull}92207[#92207]


Mapping::
* Consolidate field name validation when parsing mappings and documents {es-pull}91328[#91328]
* Fix handling empty key case in the terms aggregation {es-pull}90822[#90822]

Monitoring::
* Fix logstash loadavg (xpack cases) {es-pull}90494[#90494]
* [Stack Monitoring] Update ES module mappings {es-pull}90649[#90649]

Network::
* Clean up on exception while chunking XContent {es-pull}92024[#92024]
* Fix Chunked APIs sending incorrect responses to HEAD requests {es-pull}92042[#92042] (issue: {es-issue}92032[#92032])
* Reject connection attempts while closing {es-pull}92465[#92465]

SQL::
* Fix NPE on logging when not tracking total hits {es-pull}92425[#92425]

Search::
* Allow different decay values depending on the score function {es-pull}91195[#91195] (issue: {es-issue}78887[#78887])
* Fix timing bug with DFS profiling {es-pull}92421[#92421]

Snapshot/Restore::
* Simplify and optimize deduplication of `RepositoryData` for a non-caching repository instance {es-pull}91851[#91851] (issue: {es-issue}89952[#89952])

Store::
* Fix numOpenOutputs and modCount in ByteSizeCachingDirectory {es-pull}92440[#92440] (issue: {es-issue}92434[#92434])

Transform::
* Skip remote clusters when performing up front privileges validation {es-pull}91788[#91788]

Vector Search::
* Make `knn` search requests fully cancellable {es-pull}90612[#90612]

[[deprecation-8.6.0]]
[float]
=== Deprecations

Allocation::
* Deprecate state field in /_cluster/reroute response {es-pull}90399[#90399]
* Ensure balance threshold is at least 1 {es-pull}92100[#92100]

Ingest Node::
* Deprecate 'remove_binary' default of false for ingest attachment processor {es-pull}90460[#90460]

Mapping::
* Deprecate silently ignoring type, fields, copy_to and boost in metadata field definition {es-pull}90989[#90989] (issue: {es-issue}35389[#35389])

[[enhancement-8.6.0]]
[float]
=== Enhancements

Allocation::
* Clear up forecasted write load and shard size from previous write index during rollovers {es-pull}91590[#91590]
* Forecast average shard size during rollovers {es-pull}91561[#91561]
* Forecast write load during rollovers {es-pull}91425[#91425]
* Improve shard balancing {es-pull}91603[#91603]
* Introduce desired-balance allocator {es-pull}91343[#91343]
* Limit shard realocation retries {es-pull}90296[#90296]
* Prevalidate node removal API {es-pull}88952[#88952]
* Set default `cluster.routing.allocation.balance.disk_usage` {es-pull}91951[#91951]
* Store write load in the `IndexMetadata` during data streams rollovers {es-pull}91019[#91019]
* Update the default `cluster.routing.allocation.balance.disk_usage` {es-pull}92065[#92065]
* `DesiredBalance:` expose it via _internal/desired_balance {es-pull}91038[#91038] (issue: {es-issue}90583[#90583])

Authorization::
* [Fleet] Added logs-elastic_agent* read privileges to `kibana_system` {es-pull}91701[#91701]

CRUD::
* Keep track of average shard write load {es-pull}90768[#90768] (issue: {es-issue}90102[#90102])

Geo::
* Centroid aggregation for cartesian points and shapes {es-pull}89216[#89216] (issue: {es-issue}90156[#90156])
* Improve H3#hexRing logic and add H3#areNeighborCells method {es-pull}91140[#91140]
* Move SpatialUtils to geo library {es-pull}88088[#88088] (issue: {es-issue}86607[#86607])
* Reduce number of object allocations in H3#geoToH3 and speed up computations {es-pull}91492[#91492]
* Support `cartesian_bounds` aggregation on point and shape {es-pull}91298[#91298] (issue: {es-issue}90157[#90157])

ILM+SLM::
* ILM: Get policy support wildcard name {es-pull}89238[#89238]

Infra/Core::
* Handle APM global labels as affix setting {es-pull}91438[#91438] (issue: {es-issue}91278[#91278])
* Improve date math exclusions in expressions {es-pull}90298[#90298]
* Introduce a phase to use String.equals on constant strings, rather than def equality {es-pull}91362[#91362] (issue: {es-issue}91235[#91235])
* More actionable error for ancient indices {es-pull}91243[#91243]
* Operator/index templates {es-pull}90143[#90143]
* Operator/ingest {es-pull}89735[#89735]
* Transport threads and `_hot_threads` {es-pull}90482[#90482] (issue: {es-issue}90334[#90334])
* Upgrade XContent to Jackson 2.14.0 and enable Fast Double Parser {es-pull}90553[#90553]

Infra/Plugins::
* Create placeholder plugin when loading stable plugins {es-pull}90870[#90870]
* Example stable plugin {es-pull}90805[#90805]
* Make `extendedPlugins,` `HasNativeController` and `moduleName` optional in plugin descriptor {es-pull}90835[#90835]
* Rename `NamedComponent` name parameter to value {es-pull}91306[#91306]

Infra/Scripting::
* Use an explicit null check for null receivers in painless, rather than an NPE {es-pull}91347[#91347] (issue: {es-issue}91236[#91236])

Machine Learning::
* Add a filter parameter to frequent items {es-pull}91137[#91137]
* Add a regex to the output of the `categorize_text` aggregation {es-pull}90723[#90723]
* Add ability to filter and sort buckets by `change_point` numeric values {es-pull}91299[#91299]
* Add api to update trained model deployment `number_of_allocations` {es-pull}90728[#90728]
* Alias timestamp to @timestamp in anomaly detection results index {es-pull}90812[#90812]
* Allow `model_aliases` to be used with Pytorch trained models {es-pull}91296[#91296]
* Allow overriding timestamp field to null in file structure finder {es-pull}90764[#90764]
* Audit a message every day the datafeed has seen no data {es-pull}91774[#91774]
* Low priority trained model deployments {es-pull}91234[#91234] (issue: {es-issue}91024[#91024])
* Provide additional information about anomaly score factors {es-pull}90675[#90675]

Mapping::
* Don't create IndexCaps objects when recording unmapped fields {es-pull}90806[#90806] (issue: {es-issue}90796[#90796])
* aggregate metric double add a max min validation {es-pull}90381[#90381]

Recovery::
* Remove resize index settings once shards are started {es-pull}90391[#90391] (issue: {es-issue}90127[#90127])

Rollup::
* Test downsample runtime fields and security {es-pull}90593[#90593]

Search::
* Add LimitedOffsetsEnum to Limited offset token {es-pull}86110[#86110] (issue: {es-issue}86109[#86109])
* Add profiling and documentation for dfs phase {es-pull}90536[#90536] (issue: {es-issue}89713[#89713])
* Bulk merge field-caps responses using mapping hash {es-pull}86323[#86323]
* Enhance nested depth tracking when parsing queries {es-pull}90425[#90425]
* Expose telemetry about search usage {es-pull}91528[#91528]
* Return docs when using nested mappings in archive indices {es-pull}90585[#90585] (issue: {es-issue}90523[#90523])
* Use `IndexOrDocValues` query for IP range queries {es-pull}90303[#90303] (issue: {es-issue}83658[#83658])

Snapshot/Restore::
* Increase snaphot pool max size to 10 {es-pull}90282[#90282] (issue: {es-issue}89608[#89608])
* Tie snapshot speed to node bandwidth settings {es-pull}91021[#91021] (issue: {es-issue}57023[#57023])

Store::
* Allow plugins to wrap Lucene directories created by the `IndexModule` {es-pull}91556[#91556]

TLS::
* Add certificate start/expiry dates to SSL Diagnostic message {es-pull}89461[#89461]

TSDB::
* Generate 'index.routing_path' from dynamic mapping templates {es-pull}90552[#90552] (issue: {es-issue}90528[#90528])
* Support malformed numbers in synthetic `_source` {es-pull}90428[#90428]
* Support synthetic `_source` for `_doc_count` field {es-pull}91465[#91465]
* Synthetic _source: support `field` in many cases {es-pull}89950[#89950]
* Synthetic `_source`: `ignore_malformed` for `ip` {es-pull}90038[#90038]
* Synthetic `_source`: support `wildcard` field {es-pull}90196[#90196]

Transform::
* Add a health section to transform stats {es-pull}90760[#90760]
* Support `aggregate_metric_double` field type in transform aggregations {es-pull}91045[#91045]

Vector Search::
* Add profiling information for knn vector queries {es-pull}90200[#90200]

[[feature-8.6.0]]
[float]
=== New features

Distributed::
* Add "index" and "search" node roles with feature flag and setting {es-pull}90993[#90993]

EQL::
* EQL samples {es-pull}91312[#91312]

Health::
* Use chunked encoding for `RestGetHealthAction` {es-pull}91515[#91515] (issue: {es-issue}90223[#90223])
* [HealthAPI] Use the `RestCancellableNodeClient` infrastructure {es-pull}91587[#91587]

Machine Learning::
* Make `categorize_text` aggregation GA {es-pull}88600[#88600]

Vector Search::
* Add fielddata and scripting support for byte-sized vectors {es-pull}91184[#91184]
* Add support for indexing byte-sized knn vectors {es-pull}90774[#90774]

[[regression-8.6.0]]
[float]
=== Regressions

Infra/Core::
* Revert "Remove `ImmutableOpenMap` from snapshot services" {es-pull}90287[#90287]

[[upgrade-8.6.0]]
[float]
=== Upgrades

Infra/Logging::
* Upgrade to log4j 2.19.0 {es-pull}90589[#90589] (issue: {es-issue}90584[#90584])

Network::
* Upgrade to Netty 4.1.82.Final {es-pull}90604[#90604]
* Upgrade to Netty 4.1.84 {es-pull}91271[#91271]

Snapshot/Restore::
* Upgrade GCS SDK to 2.13.1 {es-pull}92327[#92327]