elasticsearch/docs/reference/release-notes/8.11.0.asciidoc

[[release-notes-8.11.0]]
== {es} version 8.11.0

Also see <<breaking-changes-8.11,Breaking changes in 8.11>>.

[[breaking-8.11.0]]
[float]
=== Breaking changes

Infra/Core::
* Remove `transport_versions` from cluster state API {es-pull}99223[#99223]

[[known-issues-8.11.0]]
[float]
=== Known issues
include::8.10.3.asciidoc[tag=no-preventive-gc-issue]

[[bug-8.11.0]]
[float]
=== Bug fixes

Aggregations::
* Adjust `DateHistogram's` bucket accounting to be iteratively {es-pull}101012[#101012]
* Allow parsing on non-string routing fields {es-pull}97729[#97729]
* Support runtime fields in synthetic source {es-pull}99796[#99796] (issue: {es-issue}98287[#98287])

Allocation::
* Consider node shutdown in `DataTierAllocationDecider` {es-pull}98824[#98824] (issue: {es-issue}97207[#97207])

Application::
* Align look-back with client-side cache {es-pull}101264[#101264]
* Increase K/V look-back time interval {es-pull}101205[#101205]
* Provide stable resampling {es-pull}101255[#101255]
* [Profiling] Tighten resource creation check {es-pull}99873[#99873]

Authorization::
* Allow `enrich_user` to read/view enrich indices {es-pull}100707[#100707]
* Grant editor and viewer access to profiling {es-pull}100594[#100594]

CCR::
* CCR: Use local cluster state request {es-pull}100323[#100323]

CRUD::
* Change `GetFromTranslog` to indices action {es-pull}99300[#99300]
* Wait for cluster to recover before resolving index template {es-pull}99797[#99797]

Cluster Coordination::
* Reset `GatewayService` flags before reroute {es-pull}98653[#98653] (issue: {es-issue}98606[#98606])

Data streams::
* DSL waits for the tsdb time boundaries to lapse {es-pull}100470[#100470] (issue: {es-issue}99696[#99696])
* Propagate cancellation in `DataTiersUsageTransportAction` {es-pull}100253[#100253]
* [DSL] skip deleting indices that have in-progress downsampling operations {es-pull}101495[#101495]

Downsampling::
* Make downsample target index replicas configurable {es-pull}99712[#99712]

ES|QL::
* "params" correctly parses the values including an optional "type" {es-pull}99310[#99310] (issue: {es-issue}99294[#99294])
* Account for an exception being thrown when building a `BytesRefArrayBlock` {es-pull}99726[#99726] (issue: {es-issue}99472[#99472])
* Add arithmetic operators {es-pull}98628[#98628]
* Add identity check in Block equality {es-pull}100377[#100377] (issue: {es-issue}100374[#100374])
* Adds Enrich implicit `match_fields` to `field_caps` call {es-pull}101456[#101456] (issue: {es-issue}101328[#101328])
* Better management of not stored TEXT fiels with synthetic source {es-pull}99695[#99695]
* Continue resolving attributes for Eval {es-pull}99601[#99601] (issue: {es-issue}99576[#99576])
* Create a Vector when needed for IN {es-pull}99382[#99382] (issue: {es-issue}99347[#99347])
* ESQL: Fix unreleased block in topn {es-pull}101648[#101648] (issue: {es-issue}101588[#101588])
* ESQL: check type before casting {es-pull}101492[#101492] (issue: {es-issue}101489[#101489])
* Fix NPE when aggregating literals {es-pull}99827[#99827]
* Fix escaping of backslash in LIKE operator {es-pull}101120[#101120] (issue: {es-issue}101106[#101106])
* Fix eval of functions on foldable literals {es-pull}101438[#101438] (issue: {es-issue}101425[#101425])
* Fix non-null value being returned for unsupported data types in `ValueSources` {es-pull}100656[#100656] (issue: {es-issue}100048[#100048])
* Graceful handling of non-bool condition in the filter {es-pull}100645[#100645] (issues: {es-issue}100049[#100049], {es-issue}100409[#100409])
* Handle queries with non-existing enrich policies and no field {es-pull}100647[#100647] (issue: {es-issue}100593[#100593])
* Implement serialization of `InvalidMappedField` {es-pull}98972[#98972] (issue: {es-issue}98851[#98851])
* Improve verifier error for incorrect agg declaration {es-pull}100650[#100650] (issue: {es-issue}100641[#100641])
* Limit how many bytes `concat()` can process {es-pull}100360[#100360]
* Make DISSECT parameter `append_separator` case insensitive {es-pull}101358[#101358] (issue: {es-issue}101138[#101138])
* Page shouldn't close a block twice {es-pull}100370[#100370] (issues: {es-issue}100356[#100356], {es-issue}100365[#100365])
* Preserve intermediate aggregation output in local relation {es-pull}100866[#100866] (issue: {es-issue}100807[#100807])
* Properly handle multi-values in fold() and date math {es-pull}100766[#100766] (issue: {es-issue}100497[#100497])
* Remove aliasing inside Eval {es-pull}100238[#100238] (issue: {es-issue}100174[#100174])
* Resilience to non-indexed fields {es-pull}99588[#99588] (issue: {es-issue}99506[#99506])
* Skip synthetic attributes when planning the physical fragment {es-pull}99188[#99188] (issue: {es-issue}99170[#99170])
* Support date and time intervals as input params {es-pull}101001[#101001] (issue: {es-issue}99570[#99570])
* Support queries that don't return underlying fields {es-pull}98759[#98759] (issue: {es-issue}98404[#98404])
* Use exact attributes for data source extraction {es-pull}99874[#99874] (issue: {es-issue}99183[#99183])
* `mv_expand` pushes down limit and project and keep the limit after it untouched {es-pull}100782[#100782] (issues: {es-issue}99971[#99971], {es-issue}100774[#100774])
* support metric tsdb fields while querying index patterns {es-pull}100351[#100351] (issue: {es-issue}100144[#100144])

Geo::
* Use `NamedWritable` to enable `GeoBoundingBox` serialisation {es-pull}99163[#99163] (issue: {es-issue}99089[#99089])

Health::
* Fix NPE in `StableMasterHealthIndicatorService` {es-pull}98635[#98635]
* Health report infrastructure doesn't trip the circuit breakers {es-pull}101629[#101629]
* Propagate cancellation in `GetHealthAction` {es-pull}100273[#100273]

Highlighting::
* Correctly handle `ScriptScoreQuery` in plain highlighter {es-pull}99804[#99804] (issue: {es-issue}99700[#99700])
* Disable `weight_matches` when kNN query is present {es-pull}101713[#101713]

ILM+SLM::
* Compute SLM retention from `RepositoryData` {es-pull}100092[#100092] (issue: {es-issue}99953[#99953])
* `WaitForSnapshotStep` verifies if the index belongs to the latest snapshot of that SLM policy {es-pull}100911[#100911]

Infra/Core::
* Add `java.net.NetPermission` to APM module's permissions {es-pull}99474[#99474]
* Don't update system index mappings in mixed clusters {es-pull}101778[#101778] (issues: {es-issue}101331[#101331], {es-issue}99778[#99778])
* Revert "Kibana system index does not allow user templates to affect it" {es-pull}98888[#98888]
* Specify correct current `IndexVersion` after 8.10 release {es-pull}98574[#98574] (issue: {es-issue}98555[#98555])
* Tracing: Use `doPriv` when working with spans, use `SpanId` {es-pull}100232[#100232]

Infra/Scripting::
* Improve painless error wrapping {es-pull}100872[#100872]

Ingest Node::
* Improving tika handling {es-pull}101486[#101486]
* Update enrich execution to only set index false on fields that support it {es-pull}98038[#98038] (issue: {es-issue}98019[#98019])

Machine Learning::
* Avoid risk of OOM in datafeeds when memory is constrained {es-pull}98915[#98915] (issue: {es-issue}89769[#89769])
* Fix for inference requests being sent to every node with a model allocation. If there are more nodes than items in the original request then empty requests were sent. {es-pull}100388[#100388] (issue: {es-issue}100180[#100180])
* Preserve order of inference results when calling the _infer API with multiple inputs on a model deployment with more than one allocation the output results order was not guaranteed to match the input order. The fix ensures the output order matches the input order. {es-pull}100143[#100143]
* Remove noisy 'Could not find trained model' message {es-pull}100760[#100760]
* Safely drain deployment request queues before allowing node to shutdown {es-pull}98406[#98406]
* Use the correct writable name for model assignment metadata in mixed version clusters. Prevents a node failure due to IllegalArgumentException Unknown NamedWriteable [trained_model_assignment] {es-pull}100886[#100886]
* Wait to gracefully stop deployments until alternative allocation exists {es-pull}99107[#99107]

Mapping::
* Automatically disable `ignore_malformed` on datastream `@timestamp` fields {es-pull}99346[#99346]
* Correct behaviour of `ContentPath::remove()` {es-pull}98332[#98332] (issue: {es-issue}98327[#98327])
* Fix merges of mappings with `subobjects: false` for composable index templates {es-pull}97317[#97317] (issue: {es-issue}96768[#96768])
* Percolator to support parsing script score query with params {es-pull}101051[#101051] (issue: {es-issue}97377[#97377])

Network::
* Do not report failure after connections are made {es-pull}99117[#99117]

Percolator::
* Fix percolator query for stored queries that expand on wildcard field names {es-pull}98878[#98878]

Query Languages::
* Preserve subfields for unsupported types {es-pull}100875[#100875] (issue: {es-issue}100869[#100869])

Recovery::
* Fix interruption of `markAllocationIdAsInSync` {es-pull}100610[#100610] (issues: {es-issue}96578[#96578], {es-issue}100589[#100589])

Search::
* Consistent scores for multi-term `SourceConfirmedTestQuery` {es-pull}100846[#100846] (issue: {es-issue}98712[#98712])
* Fix UnsignedLong field range query gt "0" can get the result equal to 0 {es-pull}98843[#98843]
* Fix `advanceExact` for doc values from sources {es-pull}99685[#99685]
* Fork response-sending in `OpenPointInTimeAction` {es-pull}99222[#99222]
* [CI] `SearchResponseTests#testSerialization` failing resolved {es-pull}100020[#100020] (issue: {es-issue}100005[#100005])
* fix fuzzy query rewrite parameter not work {es-pull}97642[#97642]

Security::
* Fix NullPointerException in RotableSecret {es-pull}100779[#100779] (issue: {es-issue}99759[#99759])

Snapshot/Restore::
* Fix race condition in `SnapshotsService` {es-pull}101652[#101652]
* Fix snapshot double finalization {es-pull}101497[#101497]
* Fix thread context in `getRepositoryData` {es-pull}99627[#99627]
* Frozen index input clone copy cache file {es-pull}98930[#98930]
* Make S3 anti-contention delay configurable {es-pull}101245[#101245]
* More robust timeout for repo analysis {es-pull}101184[#101184] (issue: {es-issue}101182[#101182])
* Register `repository_s3` settings {es-pull}101344[#101344]
* Reinstate `RepositoryData` BwC {es-pull}100447[#100447]

TSDB::
* Don't ignore empty index template that have no template definition {es-pull}98840[#98840] (issue: {es-issue}98834[#98834])
* Fix painless execute api and tsdb issue {es-pull}101212[#101212] (issue: {es-issue}101072[#101072])
* Make tsdb settings public in Serverless {es-pull}99567[#99567] (issue: {es-issue}99563[#99563])

Transform::
* Fix possible NPE when getting transform stats for failed transforms {es-pull}98061[#98061] (issue: {es-issue}98052[#98052])
* Ignore `IndexNotFound` error when refreshing destination index {es-pull}101627[#101627]
* Make Transform Feature Reset really wait for all the tasks {es-pull}100624[#100624]
* Make tasks that calculate checkpoints cancellable {es-pull}100808[#100808]

Watcher::
* Treating watcher webhook response header names as case-insensitive {es-pull}99717[#99717]

[[deprecation-8.11.0]]
[float]
=== Deprecations

Rollup::
* Rollup functionality is now deprecated {es-pull}101265[#101265]

[[enhancement-8.11.0]]
[float]
=== Enhancements

Aggregations::
* Disable `FilterByFilterAggregator` through `ClusterSettings` {es-pull}99417[#99417] (issue: {es-issue}99335[#99335])
* Represent histogram value count as long {es-pull}99912[#99912] (issue: {es-issue}99820[#99820])
* Skip `DisiPriorityQueue` on single filter agg {es-pull}99215[#99215] (issue: {es-issue}99202[#99202])
* Use a competitive iterator in `FiltersAggregator` {es-pull}98360[#98360] (issue: {es-issue}97544[#97544])

Allocation::
* Report a node's "roles" setting in the /_cluster/allocation/explain response {es-pull}98550[#98550] (issue: {es-issue}97859[#97859])

Application::
* Add flamegraph API {es-pull}99091[#99091]
* [Profiling] Allow to customize the ILM policy {es-pull}99909[#99909]
* [Profiling] Allow to wait until resources created {es-pull}99655[#99655]

Audit::
* Reduce verbosity of the bulk indexing audit log {es-pull}98470[#98470]

Authentication::
* Allow native users/roles to be disabled via setting {es-pull}98654[#98654]

CAT APIs::
* Add 'dataset' size to cat indices and cat shards {es-pull}98622[#98622] (issue: {es-issue}95092[#95092])

Data streams::
* Allow explain data stream lifecycle to accept a data stream {es-pull}98811[#98811]

ES|QL::
* Add `CEIL` function {es-pull}98847[#98847]
* Add ability to perform date math {es-pull}98870[#98870] (issue: {es-issue}98402[#98402])
* Add support for TEXT fields in comparison operators and SORT {es-pull}98528[#98528] (issue: {es-issue}98642[#98642])
* Compact topn {es-pull}99316[#99316]
* Date math for negatives {es-pull}99711[#99711]
* Enable arithmetics for durations and periods {es-pull}99432[#99432] (issue: {es-issue}99293[#99293])
* Enhance SHOW FUNCTIONS command {es-pull}99736[#99736] (issue: {es-issue}99507[#99507])
* Improve log messages {es-pull}99470[#99470]
* Log execution time consistently {es-pull}99286[#99286]
* Log query and execution time {es-pull}99058[#99058]
* Log start and end of queries {es-pull}99746[#99746]
* Lower the implicit limit, if none is user-provided {es-pull}99816[#99816] (issue: {es-issue}99458[#99458])
* Make settings dynamic {es-pull}101516[#101516]
* Mark counter fields as unsupported {es-pull}99054[#99054]
* Remove the swapped-args check for date_xxx() {es-pull}101362[#101362] (issue: {es-issue}99562[#99562])
* Serialize the source in expressions {es-pull}99956[#99956]
* Simple check if all blocks get released {es-pull}100199[#100199]
* Support unsigned long in sqrt and log10 {es-pull}98711[#98711]
* Use DEBUG log level to report execution steps {es-pull}99303[#99303]

Engine::
* Use `IndexWriter.flushNextBuffer()` to reclaim memory from indexing buffers {es-pull}94607[#94607]

Health::
* Avoiding the use of nodes that are no longer in the cluster when computing master stability {es-pull}98809[#98809] (issue: {es-issue}98636[#98636])
* When a primary is inactive but this is considered expected, the same applies for the replica of this shard. {es-pull}99995[#99995] (issue: {es-issue}99951[#99951])

Infra/Core::
* APM Metering API {es-pull}99832[#99832]
* Update the elastic-apm-agent version {es-pull}100064[#100064]
* Use mappings version to retrieve system index mappings at creation time {es-pull}99555[#99555]

Infra/Node Lifecycle::
* Add links to docs from failing bootstrap checks {es-pull}99644[#99644] (issue: {es-issue}99614[#99614])
* Chunk `SingleNodeShutdownStatus` and `ShutdownShardMigrationStatus` (and related action) response {es-pull}99798[#99798] (issue: {es-issue}99678[#99678])

Infra/REST API::
* Add `IndexVersion` to node info {es-pull}99515[#99515]
* Add component info versions to node info in a pluggable way {es-pull}99631[#99631]
* Return a 410 (Gone) status code for unavailable API endpoints {es-pull}97397[#97397]

Machine Learning::
* Add new _inference API {es-pull}99224[#99224]
* Adding an option for trained models to be platform specific {es-pull}99584[#99584]
* Log warnings for jobs unassigned for a long time {es-pull}100154[#100154]
* Simplify the Inference Ingest Processor configuration {es-pull}100205[#100205]

Mapping::
* Automatically flatten objects when subobjects:false {es-pull}97972[#97972] (issue: {es-issue}88934[#88934])
* Explicit parsing object capabilities of `FieldMappers` {es-pull}98684[#98684] (issue: {es-issue}98537[#98537])
* Reintroduce `sparse_vector` mapping {es-pull}98996[#98996]

Network::
* Chunk the cluster allocation explain response {es-pull}99641[#99641] (issue: {es-issue}97803[#97803])

Recovery::
* Wait for cluster state in recovery {es-pull}99193[#99193]

Search::
* Add additional counters to `_clusters` response for all Cluster search states {es-pull}99566[#99566] (issue: {es-issue}98927[#98927])
* Adding support for exist queries to `sparse_vector` fields {es-pull}99775[#99775] (issue: {es-issue}99319[#99319])
* Make `_index` optional for pinned query docs {es-pull}97450[#97450]
* Reduce copying when creating scroll/PIT ids {es-pull}99219[#99219]
* Refactor `SearchResponseClusters` to use CHM {es-pull}100129[#100129] (issue: {es-issue}99101[#99101])
* Support cluster/details for CCS minimize_roundtrips=false {es-pull}98457[#98457]

Security::
* Support rotatating the JWT shared secret {es-pull}99278[#99278]

Snapshot/Restore::
* Remove shard data files when they fail to write for snapshot {es-pull}99694[#99694]

Stats::
* Prune unnecessary information from TransportNodesInfoAction.NodeInfoRequest {es-pull}99938[#99938] (issue: {es-issue}99744[#99744])

TSDB::
* Add `index.look_back_time` setting for tsdb data streams {es-pull}98518[#98518] (issue: {es-issue}98463[#98463])
* Improve time-series error and documentation {es-pull}100018[#100018]
* Trim stored fields for `_id` field in tsdb {es-pull}97409[#97409]

Transform::
* Add accessors required to recreate `TransformStats` object from the fields {es-pull}98844[#98844]

Vector Search::
* Add new max_inner_product vector similarity function {es-pull}99527[#99527]
* Adds `nested` support for indexed `dense_vector` fields {es-pull}99763[#99763]
* Dense vector field types are indexed by default {es-pull}98268[#98268]
* Increase the max vector dims to 4096 {es-pull}99682[#99682]

[[feature-8.11.0]]
[float]
=== New features

Analysis::
* Add support for Persian language stemmer {es-pull}99106[#99106] (issue: {es-issue}98911[#98911])

Application::
* Automatically map float arrays of lengths 128 - 2048 as dense_vector {es-pull}98512[#98512] (issue: {es-issue}97532[#97532])

Data streams::
* GA the data stream lifecycle {es-pull}100187[#100187]
* GET `_data_stream` displays both ILM and DSL information {es-pull}99947[#99947]

ES|QL::
* Integrate Elasticsearch Query Language, ES|QL {es-pull}98309[#98309]
* LEAST and GREATEST functions {es-pull}98630[#98630]
* LEFT function {es-pull}98942[#98942]
* LTRIM, RTRIM and fix unicode whitespace {es-pull}98590[#98590]
* RIGHT function {es-pull}98974[#98974]
* TopN sorting with min and max for multi-value fields {es-pull}98337[#98337]

[[upgrade-8.11.0]]
[float]
=== Upgrades

Packaging::
* Update bundled JDK to 21.0.1 {es-pull}101133[#101133]

Search::
* Upgrade main to Lucene 9.8.0 {es-pull}100138[#100138]