[[release-notes-8.11.0]] == {es} version 8.11.0 Also see <>. [[breaking-8.11.0]] [float] === Breaking changes Infra/Core:: * Remove `transport_versions` from cluster state API {es-pull}99223[#99223] [[known-issues-8.11.0]] [float] === Known issues include::8.10.3.asciidoc[tag=no-preventive-gc-issue] [[bug-8.11.0]] [float] === Bug fixes Aggregations:: * Adjust `DateHistogram's` bucket accounting to be iteratively {es-pull}101012[#101012] * Allow parsing on non-string routing fields {es-pull}97729[#97729] * Support runtime fields in synthetic source {es-pull}99796[#99796] (issue: {es-issue}98287[#98287]) Allocation:: * Consider node shutdown in `DataTierAllocationDecider` {es-pull}98824[#98824] (issue: {es-issue}97207[#97207]) Application:: * Align look-back with client-side cache {es-pull}101264[#101264] * Increase K/V look-back time interval {es-pull}101205[#101205] * Provide stable resampling {es-pull}101255[#101255] * [Profiling] Tighten resource creation check {es-pull}99873[#99873] Authorization:: * Allow `enrich_user` to read/view enrich indices {es-pull}100707[#100707] * Grant editor and viewer access to profiling {es-pull}100594[#100594] CCR:: * CCR: Use local cluster state request {es-pull}100323[#100323] CRUD:: * Change `GetFromTranslog` to indices action {es-pull}99300[#99300] * Wait for cluster to recover before resolving index template {es-pull}99797[#99797] Cluster Coordination:: * Reset `GatewayService` flags before reroute {es-pull}98653[#98653] (issue: {es-issue}98606[#98606]) Data streams:: * DSL waits for the tsdb time boundaries to lapse {es-pull}100470[#100470] (issue: {es-issue}99696[#99696]) * Propagate cancellation in `DataTiersUsageTransportAction` {es-pull}100253[#100253] * [DSL] skip deleting indices that have in-progress downsampling operations {es-pull}101495[#101495] Downsampling:: * Make downsample target index replicas configurable {es-pull}99712[#99712] ES|QL:: * "params" correctly parses the values including an optional "type" {es-pull}99310[#99310] (issue: {es-issue}99294[#99294]) * Account for an exception being thrown when building a `BytesRefArrayBlock` {es-pull}99726[#99726] (issue: {es-issue}99472[#99472]) * Add arithmetic operators {es-pull}98628[#98628] * Add identity check in Block equality {es-pull}100377[#100377] (issue: {es-issue}100374[#100374]) * Adds Enrich implicit `match_fields` to `field_caps` call {es-pull}101456[#101456] (issue: {es-issue}101328[#101328]) * Better management of not stored TEXT fiels with synthetic source {es-pull}99695[#99695] * Continue resolving attributes for Eval {es-pull}99601[#99601] (issue: {es-issue}99576[#99576]) * Create a Vector when needed for IN {es-pull}99382[#99382] (issue: {es-issue}99347[#99347]) * ESQL: Fix unreleased block in topn {es-pull}101648[#101648] (issue: {es-issue}101588[#101588]) * ESQL: check type before casting {es-pull}101492[#101492] (issue: {es-issue}101489[#101489]) * Fix NPE when aggregating literals {es-pull}99827[#99827] * Fix escaping of backslash in LIKE operator {es-pull}101120[#101120] (issue: {es-issue}101106[#101106]) * Fix eval of functions on foldable literals {es-pull}101438[#101438] (issue: {es-issue}101425[#101425]) * Fix non-null value being returned for unsupported data types in `ValueSources` {es-pull}100656[#100656] (issue: {es-issue}100048[#100048]) * Graceful handling of non-bool condition in the filter {es-pull}100645[#100645] (issues: {es-issue}100049[#100049], {es-issue}100409[#100409]) * Handle queries with non-existing enrich policies and no field {es-pull}100647[#100647] (issue: {es-issue}100593[#100593]) * Implement serialization of `InvalidMappedField` {es-pull}98972[#98972] (issue: {es-issue}98851[#98851]) * Improve verifier error for incorrect agg declaration {es-pull}100650[#100650] (issue: {es-issue}100641[#100641]) * Limit how many bytes `concat()` can process {es-pull}100360[#100360] * Make DISSECT parameter `append_separator` case insensitive {es-pull}101358[#101358] (issue: {es-issue}101138[#101138]) * Page shouldn't close a block twice {es-pull}100370[#100370] (issues: {es-issue}100356[#100356], {es-issue}100365[#100365]) * Preserve intermediate aggregation output in local relation {es-pull}100866[#100866] (issue: {es-issue}100807[#100807]) * Properly handle multi-values in fold() and date math {es-pull}100766[#100766] (issue: {es-issue}100497[#100497]) * Remove aliasing inside Eval {es-pull}100238[#100238] (issue: {es-issue}100174[#100174]) * Resilience to non-indexed fields {es-pull}99588[#99588] (issue: {es-issue}99506[#99506]) * Skip synthetic attributes when planning the physical fragment {es-pull}99188[#99188] (issue: {es-issue}99170[#99170]) * Support date and time intervals as input params {es-pull}101001[#101001] (issue: {es-issue}99570[#99570]) * Support queries that don't return underlying fields {es-pull}98759[#98759] (issue: {es-issue}98404[#98404]) * Use exact attributes for data source extraction {es-pull}99874[#99874] (issue: {es-issue}99183[#99183]) * `mv_expand` pushes down limit and project and keep the limit after it untouched {es-pull}100782[#100782] (issues: {es-issue}99971[#99971], {es-issue}100774[#100774]) * support metric tsdb fields while querying index patterns {es-pull}100351[#100351] (issue: {es-issue}100144[#100144]) Geo:: * Use `NamedWritable` to enable `GeoBoundingBox` serialisation {es-pull}99163[#99163] (issue: {es-issue}99089[#99089]) Health:: * Fix NPE in `StableMasterHealthIndicatorService` {es-pull}98635[#98635] * Health report infrastructure doesn't trip the circuit breakers {es-pull}101629[#101629] * Propagate cancellation in `GetHealthAction` {es-pull}100273[#100273] Highlighting:: * Correctly handle `ScriptScoreQuery` in plain highlighter {es-pull}99804[#99804] (issue: {es-issue}99700[#99700]) * Disable `weight_matches` when kNN query is present {es-pull}101713[#101713] ILM+SLM:: * Compute SLM retention from `RepositoryData` {es-pull}100092[#100092] (issue: {es-issue}99953[#99953]) * `WaitForSnapshotStep` verifies if the index belongs to the latest snapshot of that SLM policy {es-pull}100911[#100911] Infra/Core:: * Add `java.net.NetPermission` to APM module's permissions {es-pull}99474[#99474] * Don't update system index mappings in mixed clusters {es-pull}101778[#101778] (issues: {es-issue}101331[#101331], {es-issue}99778[#99778]) * Revert "Kibana system index does not allow user templates to affect it" {es-pull}98888[#98888] * Specify correct current `IndexVersion` after 8.10 release {es-pull}98574[#98574] (issue: {es-issue}98555[#98555]) * Tracing: Use `doPriv` when working with spans, use `SpanId` {es-pull}100232[#100232] Infra/Scripting:: * Improve painless error wrapping {es-pull}100872[#100872] Ingest Node:: * Improving tika handling {es-pull}101486[#101486] * Update enrich execution to only set index false on fields that support it {es-pull}98038[#98038] (issue: {es-issue}98019[#98019]) Machine Learning:: * Avoid risk of OOM in datafeeds when memory is constrained {es-pull}98915[#98915] (issue: {es-issue}89769[#89769]) * Fix for inference requests being sent to every node with a model allocation. If there are more nodes than items in the original request then empty requests were sent. {es-pull}100388[#100388] (issue: {es-issue}100180[#100180]) * Preserve order of inference results when calling the _infer API with multiple inputs on a model deployment with more than one allocation the output results order was not guaranteed to match the input order. The fix ensures the output order matches the input order. {es-pull}100143[#100143] * Remove noisy 'Could not find trained model' message {es-pull}100760[#100760] * Safely drain deployment request queues before allowing node to shutdown {es-pull}98406[#98406] * Use the correct writable name for model assignment metadata in mixed version clusters. Prevents a node failure due to IllegalArgumentException Unknown NamedWriteable [trained_model_assignment] {es-pull}100886[#100886] * Wait to gracefully stop deployments until alternative allocation exists {es-pull}99107[#99107] Mapping:: * Automatically disable `ignore_malformed` on datastream `@timestamp` fields {es-pull}99346[#99346] * Correct behaviour of `ContentPath::remove()` {es-pull}98332[#98332] (issue: {es-issue}98327[#98327]) * Fix merges of mappings with `subobjects: false` for composable index templates {es-pull}97317[#97317] (issue: {es-issue}96768[#96768]) * Percolator to support parsing script score query with params {es-pull}101051[#101051] (issue: {es-issue}97377[#97377]) Network:: * Do not report failure after connections are made {es-pull}99117[#99117] Percolator:: * Fix percolator query for stored queries that expand on wildcard field names {es-pull}98878[#98878] Query Languages:: * Preserve subfields for unsupported types {es-pull}100875[#100875] (issue: {es-issue}100869[#100869]) Recovery:: * Fix interruption of `markAllocationIdAsInSync` {es-pull}100610[#100610] (issues: {es-issue}96578[#96578], {es-issue}100589[#100589]) Search:: * Consistent scores for multi-term `SourceConfirmedTestQuery` {es-pull}100846[#100846] (issue: {es-issue}98712[#98712]) * Fix UnsignedLong field range query gt "0" can get the result equal to 0 {es-pull}98843[#98843] * Fix `advanceExact` for doc values from sources {es-pull}99685[#99685] * Fork response-sending in `OpenPointInTimeAction` {es-pull}99222[#99222] * [CI] `SearchResponseTests#testSerialization` failing resolved {es-pull}100020[#100020] (issue: {es-issue}100005[#100005]) * fix fuzzy query rewrite parameter not work {es-pull}97642[#97642] Security:: * Fix NullPointerException in RotableSecret {es-pull}100779[#100779] (issue: {es-issue}99759[#99759]) Snapshot/Restore:: * Fix race condition in `SnapshotsService` {es-pull}101652[#101652] * Fix snapshot double finalization {es-pull}101497[#101497] * Fix thread context in `getRepositoryData` {es-pull}99627[#99627] * Frozen index input clone copy cache file {es-pull}98930[#98930] * Make S3 anti-contention delay configurable {es-pull}101245[#101245] * More robust timeout for repo analysis {es-pull}101184[#101184] (issue: {es-issue}101182[#101182]) * Register `repository_s3` settings {es-pull}101344[#101344] * Reinstate `RepositoryData` BwC {es-pull}100447[#100447] TSDB:: * Don't ignore empty index template that have no template definition {es-pull}98840[#98840] (issue: {es-issue}98834[#98834]) * Fix painless execute api and tsdb issue {es-pull}101212[#101212] (issue: {es-issue}101072[#101072]) * Make tsdb settings public in Serverless {es-pull}99567[#99567] (issue: {es-issue}99563[#99563]) Transform:: * Fix possible NPE when getting transform stats for failed transforms {es-pull}98061[#98061] (issue: {es-issue}98052[#98052]) * Ignore `IndexNotFound` error when refreshing destination index {es-pull}101627[#101627] * Make Transform Feature Reset really wait for all the tasks {es-pull}100624[#100624] * Make tasks that calculate checkpoints cancellable {es-pull}100808[#100808] Watcher:: * Treating watcher webhook response header names as case-insensitive {es-pull}99717[#99717] [[deprecation-8.11.0]] [float] === Deprecations Rollup:: * Rollup functionality is now deprecated {es-pull}101265[#101265] [[enhancement-8.11.0]] [float] === Enhancements Aggregations:: * Disable `FilterByFilterAggregator` through `ClusterSettings` {es-pull}99417[#99417] (issue: {es-issue}99335[#99335]) * Represent histogram value count as long {es-pull}99912[#99912] (issue: {es-issue}99820[#99820]) * Skip `DisiPriorityQueue` on single filter agg {es-pull}99215[#99215] (issue: {es-issue}99202[#99202]) * Use a competitive iterator in `FiltersAggregator` {es-pull}98360[#98360] (issue: {es-issue}97544[#97544]) Allocation:: * Report a node's "roles" setting in the /_cluster/allocation/explain response {es-pull}98550[#98550] (issue: {es-issue}97859[#97859]) Application:: * Add flamegraph API {es-pull}99091[#99091] * [Profiling] Allow to customize the ILM policy {es-pull}99909[#99909] * [Profiling] Allow to wait until resources created {es-pull}99655[#99655] Audit:: * Reduce verbosity of the bulk indexing audit log {es-pull}98470[#98470] Authentication:: * Allow native users/roles to be disabled via setting {es-pull}98654[#98654] CAT APIs:: * Add 'dataset' size to cat indices and cat shards {es-pull}98622[#98622] (issue: {es-issue}95092[#95092]) Data streams:: * Allow explain data stream lifecycle to accept a data stream {es-pull}98811[#98811] ES|QL:: * Add `CEIL` function {es-pull}98847[#98847] * Add ability to perform date math {es-pull}98870[#98870] (issue: {es-issue}98402[#98402]) * Add support for TEXT fields in comparison operators and SORT {es-pull}98528[#98528] (issue: {es-issue}98642[#98642]) * Compact topn {es-pull}99316[#99316] * Date math for negatives {es-pull}99711[#99711] * Enable arithmetics for durations and periods {es-pull}99432[#99432] (issue: {es-issue}99293[#99293]) * Enhance SHOW FUNCTIONS command {es-pull}99736[#99736] (issue: {es-issue}99507[#99507]) * Improve log messages {es-pull}99470[#99470] * Log execution time consistently {es-pull}99286[#99286] * Log query and execution time {es-pull}99058[#99058] * Log start and end of queries {es-pull}99746[#99746] * Lower the implicit limit, if none is user-provided {es-pull}99816[#99816] (issue: {es-issue}99458[#99458]) * Make settings dynamic {es-pull}101516[#101516] * Mark counter fields as unsupported {es-pull}99054[#99054] * Remove the swapped-args check for date_xxx() {es-pull}101362[#101362] (issue: {es-issue}99562[#99562]) * Serialize the source in expressions {es-pull}99956[#99956] * Simple check if all blocks get released {es-pull}100199[#100199] * Support unsigned long in sqrt and log10 {es-pull}98711[#98711] * Use DEBUG log level to report execution steps {es-pull}99303[#99303] Engine:: * Use `IndexWriter.flushNextBuffer()` to reclaim memory from indexing buffers {es-pull}94607[#94607] Health:: * Avoiding the use of nodes that are no longer in the cluster when computing master stability {es-pull}98809[#98809] (issue: {es-issue}98636[#98636]) * When a primary is inactive but this is considered expected, the same applies for the replica of this shard. {es-pull}99995[#99995] (issue: {es-issue}99951[#99951]) Infra/Core:: * APM Metering API {es-pull}99832[#99832] * Update the elastic-apm-agent version {es-pull}100064[#100064] * Use mappings version to retrieve system index mappings at creation time {es-pull}99555[#99555] Infra/Node Lifecycle:: * Add links to docs from failing bootstrap checks {es-pull}99644[#99644] (issue: {es-issue}99614[#99614]) * Chunk `SingleNodeShutdownStatus` and `ShutdownShardMigrationStatus` (and related action) response {es-pull}99798[#99798] (issue: {es-issue}99678[#99678]) Infra/REST API:: * Add `IndexVersion` to node info {es-pull}99515[#99515] * Add component info versions to node info in a pluggable way {es-pull}99631[#99631] * Return a 410 (Gone) status code for unavailable API endpoints {es-pull}97397[#97397] Machine Learning:: * Add new _inference API {es-pull}99224[#99224] * Adding an option for trained models to be platform specific {es-pull}99584[#99584] * Log warnings for jobs unassigned for a long time {es-pull}100154[#100154] * Simplify the Inference Ingest Processor configuration {es-pull}100205[#100205] Mapping:: * Automatically flatten objects when subobjects:false {es-pull}97972[#97972] (issue: {es-issue}88934[#88934]) * Explicit parsing object capabilities of `FieldMappers` {es-pull}98684[#98684] (issue: {es-issue}98537[#98537]) * Reintroduce `sparse_vector` mapping {es-pull}98996[#98996] Network:: * Chunk the cluster allocation explain response {es-pull}99641[#99641] (issue: {es-issue}97803[#97803]) Recovery:: * Wait for cluster state in recovery {es-pull}99193[#99193] Search:: * Add additional counters to `_clusters` response for all Cluster search states {es-pull}99566[#99566] (issue: {es-issue}98927[#98927]) * Adding support for exist queries to `sparse_vector` fields {es-pull}99775[#99775] (issue: {es-issue}99319[#99319]) * Make `_index` optional for pinned query docs {es-pull}97450[#97450] * Reduce copying when creating scroll/PIT ids {es-pull}99219[#99219] * Refactor `SearchResponseClusters` to use CHM {es-pull}100129[#100129] (issue: {es-issue}99101[#99101]) * Support cluster/details for CCS minimize_roundtrips=false {es-pull}98457[#98457] Security:: * Support rotatating the JWT shared secret {es-pull}99278[#99278] Snapshot/Restore:: * Remove shard data files when they fail to write for snapshot {es-pull}99694[#99694] Stats:: * Prune unnecessary information from TransportNodesInfoAction.NodeInfoRequest {es-pull}99938[#99938] (issue: {es-issue}99744[#99744]) TSDB:: * Add `index.look_back_time` setting for tsdb data streams {es-pull}98518[#98518] (issue: {es-issue}98463[#98463]) * Improve time-series error and documentation {es-pull}100018[#100018] * Trim stored fields for `_id` field in tsdb {es-pull}97409[#97409] Transform:: * Add accessors required to recreate `TransformStats` object from the fields {es-pull}98844[#98844] Vector Search:: * Add new max_inner_product vector similarity function {es-pull}99527[#99527] * Adds `nested` support for indexed `dense_vector` fields {es-pull}99763[#99763] * Dense vector field types are indexed by default {es-pull}98268[#98268] * Increase the max vector dims to 4096 {es-pull}99682[#99682] [[feature-8.11.0]] [float] === New features Analysis:: * Add support for Persian language stemmer {es-pull}99106[#99106] (issue: {es-issue}98911[#98911]) Application:: * Automatically map float arrays of lengths 128 - 2048 as dense_vector {es-pull}98512[#98512] (issue: {es-issue}97532[#97532]) Data streams:: * GA the data stream lifecycle {es-pull}100187[#100187] * GET `_data_stream` displays both ILM and DSL information {es-pull}99947[#99947] ES|QL:: * Integrate Elasticsearch Query Language, ES|QL {es-pull}98309[#98309] * LEAST and GREATEST functions {es-pull}98630[#98630] * LEFT function {es-pull}98942[#98942] * LTRIM, RTRIM and fix unicode whitespace {es-pull}98590[#98590] * RIGHT function {es-pull}98974[#98974] * TopN sorting with min and max for multi-value fields {es-pull}98337[#98337] [[upgrade-8.11.0]] [float] === Upgrades Packaging:: * Update bundled JDK to 21.0.1 {es-pull}101133[#101133] Search:: * Upgrade main to Lucene 9.8.0 {es-pull}100138[#100138]