elasticsearch/docs/reference/release-notes/8.12.0.asciidoc

[[release-notes-8.12.0]]
== {es} version 8.12.0

Also see <<breaking-changes-8.12,Breaking changes in 8.12>>.

[[breaking-8.12.0]]
[float]
=== Breaking changes
There are no breaking changes in 8.12

[[notable-8.12.0]]
[float]  
=== Notable changes
There are notable changes in 8.12 that you need to be aware of but that we do not consider breaking, items that we may consider as notable changes are

* Changes to features that are in Technical Preview.
* Changes to log formats.
* Changes to non-public APIs.
* Behaviour changes that repair critical bugs.

Authorization::
* Fixed JWT principal from claims {es-pull}101333[#101333]

ES|QL::
* [ES|QL] pow function always returns double {es-pull}102183[#102183] (issue: {es-issue}99055[#99055])

Infra/Plugins::
* Remove Plugin.createComponents method in favour of overload with a PluginServices object {es-pull}101457[#101457]

[[bug-8.12.0]]
[float]
=== Bug fixes

Aggregations::
* Adjust Histogram's bucket accounting to be iteratively {es-pull}102172[#102172]
* Aggs error codes part 1 {es-pull}99963[#99963]
* Skip global ordinals loading if query does not match after rewrite {es-pull}102844[#102844]
* Trigger parent circuit breaker when building scorers in filters aggregation {es-pull}102511[#102511]
* Unwrap `ExecutionException` when loading from cache in `AbstractIndexOrdinalsFieldData` {es-pull}102476[#102476]

Application::
* [Connector API] Fix bug with nullable tooltip field in parser {es-pull}103427[#103427]
* [Connectors API] Fix `ClassCastException` when creating a new sync job {es-pull}103508[#103508]
* [Connectors API] Fix bug with missing TEXT `DisplayType` enum {es-pull}103430[#103430]
* [Connectors API] Handle nullable fields correctly in the `ConnectorSyncJob` parser {es-pull}103183[#103183]
* [Profiling] Query in parallel only if beneficial {es-pull}103061[#103061]
* [Search Applications] Return 400 response when template rendering produces invalid JSON {es-pull}101474[#101474]

Authentication::
* Fall through malformed JWTs to subsequent realms in the chain {es-pull}101660[#101660] (issue: {es-issue}101367[#101367])

Authorization::
* Fix cache invalidation on privilege modification {es-pull}102193[#102193]

Data streams::
* Use dataset size instead of on-disk size for data stream stats {es-pull}103342[#103342]

Distributed::
* Active shards message corrected for search shards {es-pull}102808[#102808] (issue: {es-issue}101896[#101896])
* Dispatch `ClusterStateAction#buildResponse` to executor {es-pull}103435[#103435]
* Fix listeners in `SharedBlobCacheService.readMultiRegions` {es-pull}101727[#101727]

Downsampling::
* Copy counter field properties to downsampled index {es-pull}103580[#103580] (issue: {es-issue}103569[#103569])
* Fix downsample api by returning a failure in case one or more downsample persistent tasks failed {es-pull}103615[#103615]

EQL::
* Cover head/tail commands edge cases and data types coverage {es-pull}101859[#101859] (issue: {es-issue}101724[#101724])
* Fix NPE on missing event queries {es-pull}103611[#103611] (issue: {es-issue}103608[#103608])
* Samples should check if the aggregations result is empty or null {es-pull}103574[#103574]

ES|QL::
* ESQL: Fix `to_degrees()` returning infinity {es-pull}103209[#103209] (issue: {es-issue}102987[#102987])
* ESQL: Fix planning of MV_EXPAND with foldable expressions {es-pull}101385[#101385] (issue: {es-issue}101118[#101118])
* ESQL: Fix rare bug with empty string {es-pull}102350[#102350] (issue: {es-issue}101969[#101969])
* ESQL: Fix resolution of MV_EXPAND after KEEP * {es-pull}103339[#103339] (issue: {es-issue}103331[#103331])
* ESQL: Fix single value query {es-pull}102317[#102317] (issue: {es-issue}102298[#102298])
* ESQL: Improve local folding of aggregates {es-pull}103670[#103670]
* ESQL: Improve pushdown of certain filters {es-pull}103671[#103671]
* ESQL: Narrow catch in convert functions {es-pull}101788[#101788] (issue: {es-issue}100820[#100820])
* ESQL: Update the use of some user-caused exceptions {es-pull}104046[#104046]
* ESQL: remove `time_zone` request parameter {es-pull}102767[#102767] (issue: {es-issue}102159[#102159])
* ES|QL: Fix NPE on single value detection {es-pull}103150[#103150] (issue: {es-issue}103141[#103141])
* ES|QL: Improve resolution error management in `mv_expand` {es-pull}102967[#102967] (issue: {es-issue}102964[#102964])
* Fix layout for MV_EXPAND {es-pull}102916[#102916] (issue: {es-issue}102912[#102912])
* Fix now in millis for ESQL search contexts {es-pull}103474[#103474] (issue: {es-issue}103455[#103455])
* Fix planning of duplicate aggs {es-pull}102165[#102165] (issue: {es-issue}102083[#102083])
* Fix the transport version of `PlanStreamOutput` {es-pull}103758[#103758]
* `AsyncOperator#isFinished` must never return true on failure {es-pull}104029[#104029]

Engine::
* Fix `lastUnsafeSegmentGenerationForGets` for realtime get {es-pull}101700[#101700]

Geo::
* Fix geo tile bounding boxes to be consistent with arithmetic method {es-pull}100826[#100826] (issues: {es-issue}92611[#92611], {es-issue}95574[#95574])

ILM+SLM::
* Collect data tiers usage stats more efficiently {es-pull}102140[#102140] (issue: {es-issue}100230[#100230])

Indices APIs::
* Fix template simulate setting application ordering {es-pull}103024[#103024] (issue: {es-issue}103008[#103008])

Infra/Core::
* Cache component versions {es-pull}103408[#103408] (issue: {es-issue}102103[#102103])
* Fix metric gauge creation model {es-pull}100609[#100609]

Infra/Node Lifecycle::
* Wait for reroute before acking put-shutdown {es-pull}103251[#103251]

Infra/Plugins::
* Making classname optional in Transport protocol {es-pull}99702[#99702] (issue: {es-issue}98584[#98584])

Infra/Scripting::
* Make IPAddress writeable {es-pull}101093[#101093] (issue: {es-issue}101082[#101082])
* Wrap painless explain error {es-pull}103151[#103151] (issue: {es-issue}103018[#103018])

Infra/Settings::
* Report full stack trace for non-state file settings transforms {es-pull}101346[#101346]

Ingest Node::
* Sending an index name to `DocumentParsingObserver` that is not ever null {es-pull}100862[#100862]

License::
* Error log when license verification fails locally {es-pull}102919[#102919]

Machine Learning::
* Catch exceptions during `pytorch_inference` startup {es-pull}103873[#103873]
* Exclude quantiles when fetching model snapshots where possible {es-pull}103530[#103530]
* Fix `frequent_item_sets` aggregation on empty index {es-pull}103116[#103116] (issue: {es-issue}103067[#103067])
* If trained model download task is in progress, wait for it to finish before executing start trained model deployment {es-pull}102944[#102944]
* Persist data counts on job close before results index refresh {es-pull}101147[#101147]
* Preserve response headers in Datafeed preview {es-pull}103923[#103923]
* Prevent attempts to access non-existent node information during rebalancing {es-pull}103361[#103361]
* Prevent resource over-subscription in model allocation planner {es-pull}100392[#100392]
* Start a new trace context before loading a trained model {es-pull}103124[#103124]
* Wait for the model results on graceful shutdown {es-pull}103591[#103591] (issue: {es-issue}103414[#103414])

Mapping::
* Revert change {es-pull}103865[#103865]

Monitoring::
* [Monitoring] Dont get cluster state until recovery {es-pull}100565[#100565]

Network::
* Ensure the correct `threadContext` for `RemoteClusterNodesAction` {es-pull}101050[#101050]

Ranking::
* Add an additional tiebreaker to RRF {es-pull}101847[#101847] (issue: {es-issue}101232[#101232])

Reindex::
* Allow prefix index naming while reindexing from remote {es-pull}96968[#96968] (issue: {es-issue}89120[#89120])

Search::
* Add JIT compiler excludes for `computeCommonPrefixLengthAndBuildHistogram` {es-pull}103112[#103112]
* Check that scripts produce correct json in render template action {es-pull}101518[#101518] (issue: {es-issue}101477[#101477])
* Fix NPE & empty result handling in `CountOnlyQueryPhaseResultConsumer` {es-pull}103203[#103203]
* Fix format string in `OldLuceneVersions` {es-pull}103185[#103185]
* Handle timeout on standalone rewrite calls {es-pull}103546[#103546]
* Introduce Elasticsearch `PostingFormat` based on Lucene 90 positing format using PFOR {es-pull}103601[#103601] (issue: {es-issue}103002[#103002])
* Restore inter-segment search concurrency with synthetic source is enabled {es-pull}103690[#103690]
* Support complex datemath expressions in index and index alias names {es-pull}100646[#100646]

Snapshot/Restore::
* Decref `SharedBytes.IO` after read is done not before {es-pull}102848[#102848]
* More consistent logging messages for snapshot deletion {es-pull}101024[#101024]
* Reroute on shard snapshot completion {es-pull}101585[#101585] (issue: {es-issue}101514[#101514])
* Restore `SharedBytes.IO` refcounting on reads & writes {es-pull}102843[#102843]

TSDB::
* Throw when wrapping rate agg in `DeferableBucketAggregator` {es-pull}101032[#101032]

Transform::
* Add an assertion to the testTransformFeatureReset test case {es-pull}100287[#100287]
* Consider search context missing exceptions as recoverable {es-pull}102602[#102602]
* Consider task cancelled exceptions as recoverable {es-pull}100828[#100828]
* Fix NPE that is thrown by `_update` API {es-pull}104051[#104051] (issue: {es-issue}104048[#104048])
* Log stacktrace together with log message in order to help debugging {es-pull}101607[#101607]
* Split comma-separated source index strings into separate indices {es-pull}102811[#102811] (issue: {es-issue}99564[#99564])

Vector Search::
* Disallow vectors whose magnitudes will not fit in a float {es-pull}100519[#100519]

Watcher::
* Correctly logging watcher history write failures {es-pull}101802[#101802]
* Fix: Watcher REST API `GET /_watcher/settings` now includes product header {es-pull}103003[#103003] (issue: {es-issue}102928[#102928])

[[enhancement-8.12.0]]
[float]
=== Enhancements

Aggregations::
* Check the real memory circuit breaker when building global ordinals {es-pull}102462[#102462]
* Disable concurrency for sampler and diversified sampler {es-pull}102832[#102832]
* Disable parallelism for composite agg against high cardinality fields {es-pull}102644[#102644]
* Enable concurrency for multi terms agg {es-pull}102710[#102710]
* Enable concurrency for scripted metric agg {es-pull}102461[#102461]
* Enable inter-segment concurrency for terms aggs {es-pull}101390[#101390]
* Export circuit breaker trip count as a counter metric {es-pull}101423[#101423]
* Introduce fielddata cache ttl {es-pull}102682[#102682]
* Status codes for Aggregation errors, part 2 {es-pull}100368[#100368]
* Support keyed histograms {es-pull}101826[#101826] (issue: {es-issue}100242[#100242])

Allocation::
* Add more desired balance stats {es-pull}102065[#102065]
* Add undesired shard count {es-pull}101426[#101426]
* Expose reconciliation metrics via APM {es-pull}102244[#102244]

Application::
* Calculate CO2 and emmission and costs {es-pull}101979[#101979]
* Consider duplicate stacktraces in custom index {es-pull}102292[#102292]
* Enable Universal Profiling as Enterprise feature {es-pull}100333[#100333]
* Include totals in flamegraph response {es-pull}101126[#101126]
* Retrieve stacktrace events from a custom index {es-pull}102020[#102020]
* [Profiling] Notify early about task cancellation {es-pull}102740[#102740]
* [Profiling] Report in status API if docs exist {es-pull}102735[#102735]

Authentication::
* Add ldap user metadata mappings for full name and email {es-pull}102925[#102925]
* Add manage_enrich cluster privilege to kibana_system role {es-pull}101682[#101682]

Authorization::
* Remove `auto_configure` privilege for profiling {es-pull}101026[#101026]
* Use `BulkRequest` to store Application Privileges {es-pull}102056[#102056]
* Use non-deprecated SAML callback URL in SAML smoketests {es-pull}99983[#99983] (issue: {es-issue}99986[#99986])
* Use non-deprecated SAML callback URL in tests {es-pull}99983[#99983] (issue: {es-issue}99985[#99985])

CAT APIs::
* Expose roles by default in cat allocation API {es-pull}101753[#101753]

CRUD::
* Cache resolved index for mgets {es-pull}101311[#101311]

Data streams::
* Introduce new endpoint to expose data stream lifecycle stats {es-pull}101845[#101845]
* Switch logs data streams to search all fields by default {es-pull}102456[#102456] (issue: {es-issue}99872[#99872])

Distributed::
* Add support for configuring proxy scheme in S3 client settings and EC2 discovery plugin {es-pull}102495[#102495] (issue: {es-issue}101873[#101873])
* Introduce a `StreamOutput` that counts how many bytes are written to the stream {es-pull}102906[#102906]
* Push s3 requests count via metrics API {es-pull}100383[#100383]
* Record operation purpose for s3 stats collection {es-pull}100236[#100236]

EQL::
* Add error logging for *QL {es-pull}101057[#101057]
* Use the eql query filter for the open-pit request {es-pull}103212[#103212]

ES|QL::
* ESQL: Add `profile` option {es-pull}102713[#102713]
* ESQL: Alias duplicated aggregations in a stats {es-pull}100642[#100642] (issue: {es-issue}100544[#100544])
* ESQL: Load more than one field at once {es-pull}102192[#102192]
* ESQL: Load stored fields sequentially {es-pull}102727[#102727]
* ESQL: Load text field from parent keyword field {es-pull}102490[#102490] (issue: {es-issue}102473[#102473])
* ESQL: Make blocks ref counted {es-pull}100408[#100408]
* ESQL: Make fieldcaps calls lighter {es-pull}102510[#102510] (issues: {es-issue}101763[#101763], {es-issue}102393[#102393])
* ESQL: More tracking in `BlockHash` impls {es-pull}101488[#101488]
* ESQL: New telemetry commands {es-pull}102937[#102937]
* ESQL: Share constant null Blocks {es-pull}102673[#102673]
* ESQL: Short circuit loading empty doc values {es-pull}102434[#102434]
* ESQL: Support the `_source` metadata field {es-pull}102391[#102391]
* ESQL: Track blocks emitted from lucene {es-pull}101396[#101396]
* ESQL: Track memory from values loaded from lucene {es-pull}101383[#101383]
* Fast path for reading single doc with ordinals {es-pull}102902[#102902]
* Introduce local block factory {es-pull}102901[#102901]
* Load different way {es-pull}101235[#101235]
* Track ESQL enrich memory {es-pull}102184[#102184]
* Track blocks in `AsyncOperator` {es-pull}102188[#102188]
* Track blocks of intermediate state of aggs {es-pull}102562[#102562]
* Track blocks when hashing single multi-valued field {es-pull}102612[#102612]
* Track pages in ESQL enrich request/response {es-pull}102190[#102190]

Engine::
* Add static node settings to set default values for max merged segment sizes {es-pull}102208[#102208]

Geo::
* Add runtime field of type `geo_shape` {es-pull}100492[#100492] (issue: {es-issue}61299[#61299])

Health::
* Add message field to `HealthPeriodicLogger` and `S3RequestRetryStats` {es-pull}101989[#101989]
* Add non-green indicator names to `HealthPeriodicLogger` message {es-pull}102245[#102245]

ILM+SLM::
* Health Report API should not return RED for unassigned cold/frozen shards when data is available {es-pull}100776[#100776]
* Switch fleet's built-in ILM policies to use .actions.rollover.max_primary_shard_size {es-pull}99984[#99984] (issue: {es-issue}99983[#99983])

Indices APIs::
* Add executed pipelines to bulk api response {es-pull}100031[#100031]
* Add support for marking component templates as deprecated {es-pull}101148[#101148] (issue: {es-issue}100992[#100992])
* Allowing non-dynamic index settings to be updated by automatically unassigning shards {es-pull}101723[#101723]
* Rename component templates and pipelines according to the new naming conventions {es-pull}99975[#99975]
* Run `TransportGetAliasesAction` on local node {es-pull}101815[#101815]

Infra/CLI::
* Set `ActiveProcessorCount` when `node.processors` is set {es-pull}101846[#101846]

Infra/Core::
* Add apm api for asynchronous counters (always increasing) {es-pull}102598[#102598]
* Log errors in `RestResponse` regardless of `error_trace` parameter {es-pull}101066[#101066] (issue: {es-issue}100884[#100884])

Infra/Logging::
* Add status code to `rest.suppressed` log output {es-pull}100990[#100990]

Ingest Node::
* Deprecate the unused `elasticsearch_version` field of enrich policy json {es-pull}103013[#103013]
* Optimize `MurmurHash3` {es-pull}101202[#101202]

Machine Learning::
* Accept a single or multiple inputs to `_inference` {es-pull}102075[#102075]
* Add basic telelemetry for the inference feature {es-pull}102877[#102877]
* Add internal inference action for ml models an services {es-pull}102731[#102731]
* Add prefix strings option to trained models {es-pull}102089[#102089]
* Estimate the memory required to deploy trained models more accurately {es-pull}98874[#98874]
* Improve stability of spike and dip detection for the change point aggregation {es-pull}102637[#102637]
* Include ML processor limits in `_ml/info` response {es-pull}101392[#101392]
* Read scores from downloaded vocabulary for XLM Roberta tokenizers {es-pull}101868[#101868]
* Support for GET all models and by task type in the `_inference` API {es-pull}102806[#102806]

Mapping::
* Improve analyzer reload log message {es-pull}102273[#102273]

Monitoring::
* Add memory utilization Kibana metric to the monitoring index templates {es-pull}102810[#102810]
* Added `beat.stats.libbeat.pipeline.queue.max_events` {es-pull}102570[#102570]

Network::
* Record more detailed HTTP stats {es-pull}99852[#99852]

Search::
* Add metrics to the shared blob cache {es-pull}101577[#101577]
* Add support for Serbian Language Analyzer {es-pull}100921[#100921]
* Add support for `index_filter` to open pit {es-pull}102388[#102388] (issue: {es-issue}99740[#99740])
* Added metric for cache eviction of entries with non zero frequency {es-pull}100570[#100570]
* Disable inter-segment concurrency when sorting by field {es-pull}101535[#101535]
* Enable query phase parallelism within a single shard {es-pull}101230[#101230] (issue: {es-issue}80693[#80693])
* Node stats as metrics {es-pull}102248[#102248]
* Optimize `_count` type API requests {es-pull}102888[#102888]

Security::
* Expose the `invalidation` field in Get/Query `ApiKey` APIs {es-pull}102472[#102472]
* Make `api_key.delete.interval` a dynamic setting {es-pull}102680[#102680]

Snapshot/Restore::
* Fail S3 repository analysis on partial reads {es-pull}102840[#102840]
* Parallelize stale index deletion {es-pull}100316[#100316] (issue: {es-issue}61513[#61513])
* Repo analysis of uncontended register behaviour {es-pull}101185[#101185]
* Repo analysis: allow configuration of register ops {es-pull}102051[#102051]
* Repo analysis: verify empty register {es-pull}102048[#102048]

Stats::
* Introduce includeShardsStats in the stats request to indicate that we only fetch a summary {es-pull}100466[#100466] (issue: {es-issue}99744[#99744])
* Set includeShardsStats = false in NodesStatsRequest where the caller does not use shards-level statistics {es-pull}100938[#100938]

Store::
* Add methods for adding generation listeners with primary term {es-pull}100899[#100899]
* Allow executing multiple periodic flushes while they are being made durable {es-pull}102571[#102571]
* Pass shard's primary term to Engine#addSegmentGenerationListener {es-pull}99752[#99752]

Transform::
* Implement exponential backoff for transform state persistence retrying {es-pull}102512[#102512] (issue: {es-issue}102528[#102528])
* Make tasks that calculate checkpoints time out {es-pull}101055[#101055]
* Pass source query to `_field_caps` (as `index_filter`) when deducing destination index mappings for better performance {es-pull}102379[#102379]
* Pass transform source query as `index_filter` to `open_point_in_time` request {es-pull}102447[#102447] (issue: {es-issue}101049[#101049])
* Skip shards that don't match the source query during checkpointing {es-pull}102138[#102138]

Vector Search::
* Add vector_operation_count in profile output for knn searches {es-pull}102032[#102032]
* Make cosine similarity faster by storing magnitude and normalizing vectors {es-pull}99445[#99445]

[[feature-8.12.0]]
[float]
=== New features

Application::
* Enable Connectors API as technical preview {es-pull}102994[#102994]
* [Behavioral Analytics] Analytics collections use Data Stream Lifecycle (DSL) instead of Index Lifecycle Management (ILM) for data retention management. Behavioral analytics has traditionally used ILM to manage data retention. Starting with 8.12.0, this will change. Analytics collections created prior to 8.12.0 will continue to use their existing ILM policies, but new analytics collections will be managed using DSL. {es-pull}100033[#100033]

Authentication::
* Patterns support for allowed subjects by the JWT realm {es-pull}102426[#102426]

Cluster Coordination::
* Add a node feature join barrier. This prevents nodes from joining clusters that do not have all the features already present in the cluster. This ensures that once a features is supported by all the nodes in a cluster, that feature will never then not be supported in the future. This is the corresponding functionality for the version join barrier, but for features
 {es-pull}101609[#101609]

Data streams::
* Add ability to create a data stream failure store {es-pull}99134[#99134]

ES|QL::
* ESQL: emit warnings from single-value functions processing multi-values {es-pull}102417[#102417] (issue: {es-issue}98743[#98743])
* GEO_POINT and CARTESIAN_POINT type support {es-pull}102177[#102177]

Infra/Core::
* Create new cluster state API for querying features present on a cluster {es-pull}100974[#100974]

Ingest Node::
* Adding a simulate ingest api {es-pull}101409[#101409]

Security::
* Allow granting API keys with JWT as the access_token {es-pull}101904[#101904]

Vector Search::
* Add byte quantization for float vectors in HNSW {es-pull}102093[#102093]
* Make knn search a query {es-pull}98916[#98916]

[[regression-8.12.0]]
[float]
=== Regressions

Infra/Core::
* Revert non-semantic `NodeInfo` {es-pull}102636[#102636]

[[upgrade-8.12.0]]
[float]
=== Upgrades

Search::
* Upgrade to Lucene 9.9.1 {es-pull}103549[#103549]