elasticsearch/docs/reference/release-notes/9.0.0.asciidoc

[[release-notes-9.0.0]]
== {es} version 9.0.0

coming[9.0.0]

Also see <<breaking-changes-9.0,Breaking changes in 9.0>>.

[[breaking-9.0.0]]
[float]
=== Breaking changes

Allocation::
* Increase minimum threshold in shard balancer {es-pull}115831[#115831]
* Remove `cluster.routing.allocation.disk.watermark.enable_for_single_data_node` setting {es-pull}114207[#114207]
* Remove cluster state from `/_cluster/reroute` response {es-pull}114231[#114231] (issue: {es-issue}88978[#88978])

Analysis::
* Snowball stemmers have been upgraded {es-pull}114146[#114146]
* The 'german2' stemmer is now an alias for the 'german' snowball stemmer {es-pull}113614[#113614]
* The 'persian' analyzer has stemmer by default {es-pull}113482[#113482] (issue: {es-issue}113050[#113050])
* The Korean dictionary for Nori has been updated {es-pull}114124[#114124]

Cluster Coordination::
* Remove unsupported legacy value for `discovery.type` {es-pull}112903[#112903]

Highlighting::
* Remove support for deprecated `force_source` highlighting parameter {es-pull}116943[#116943]

Indices APIs::
* Apply more strict parsing of actions in bulk API {es-pull}115923[#115923]
* Remove deprecated local attribute from alias APIs {es-pull}115393[#115393]

Infra/REST API::
* Output a consistent format when generating error json {es-pull}90529[#90529] (issue: {es-issue}89387[#89387])

Ingest Node::
* Remove `ecs` option on `user_agent` processor {es-pull}116077[#116077]
* Remove ignored fallback option on GeoIP processor {es-pull}116112[#116112]

Mapping::
* Remove support for type, fields, `copy_to` and boost in metadata field definition {es-pull}116944[#116944]

Search::
* Remove legacy params from range query {es-pull}116970[#116970]

Snapshot/Restore::
* Remove deprecated `xpack.searchable.snapshot.allocate_on_rolling_restart` setting {es-pull}114202[#114202]

[[bug-9.0.0]]
[float]
=== Bug fixes

Aggregations::
* Handle with `illegalArgumentExceptions` negative values in HDR percentile aggregations {es-pull}116174[#116174] (issue: {es-issue}115777[#115777])

Analysis::
* Adjust analyze limit exception to be a `bad_request` {es-pull}116325[#116325]

CCS::
* Fix long metric deserialize & add - auto-resize needs to be set manually {es-pull}117105[#117105] (issue: {es-issue}116914[#116914])

CRUD::
* Preserve thread context when waiting for segment generation in RTG {es-pull}114623[#114623]
* Standardize error code when bulk body is invalid {es-pull}114869[#114869]

Data streams::
* Add missing header in `put_data_lifecycle` rest-api-spec {es-pull}116292[#116292]

EQL::
* Don't use a `BytesStreamOutput` to copy keys in `BytesRefBlockHash` {es-pull}114819[#114819] (issue: {es-issue}114599[#114599])

ES|QL::
* Added stricter range type checks and runtime warnings for ENRICH {es-pull}115091[#115091] (issues: {es-issue}107357[#107357], {es-issue}116799[#116799])
* Don't return TEXT type for functions that take TEXT {es-pull}114334[#114334] (issues: {es-issue}111537[#111537], {es-issue}114333[#114333])
* ESQL: Fix sorts containing `_source` {es-pull}116980[#116980] (issue: {es-issue}116659[#116659])
* ESQL: fix the column position in errors {es-pull}117153[#117153]
* ES|QL: Fix stats by constant expression {es-pull}114899[#114899]
* Fix NPE in `EnrichLookupService` on mixed clusters with <8.14 versions {es-pull}116583[#116583] (issues: {es-issue}116529[#116529], {es-issue}116544[#116544])
* Fix TDigestState.read CB leaks {es-pull}114303[#114303] (issue: {es-issue}114194[#114194])
* Fixing remote ENRICH by pushing the Enrich inside `FragmentExec` {es-pull}114665[#114665] (issue: {es-issue}105095[#105095])
* Use `SearchStats` instead of field.isAggregatable in data node planning {es-pull}115744[#115744] (issue: {es-issue}115737[#115737])
* [ESQL] Fix Binary Comparisons on Date Nanos {es-pull}116346[#116346]
* [ES|QL] To_DatePeriod and To_TimeDuration return better error messages on `union_type` fields {es-pull}114934[#114934]

Infra/CLI::
* Fix NPE on plugin sync {es-pull}115640[#115640] (issue: {es-issue}114818[#114818])

Infra/Metrics::
* Make `randomInstantBetween` always return value in range [minInstant, `maxInstant]` {es-pull}114177[#114177]

Infra/REST API::
* Fixed a `NullPointerException` in `_capabilities` API when the `path` parameter is null. {es-pull}113413[#113413] (issue: {es-issue}113413[#113413])

Infra/Settings::
* Don't allow secure settings in YML config (109115) {es-pull}115779[#115779] (issue: {es-issue}109115[#109115])

Ingest Node::
* Add warning headers for ingest pipelines containing special characters {es-pull}114837[#114837] (issue: {es-issue}104411[#104411])
* Reducing error-level stack trace logging for normal events in `GeoIpDownloader` {es-pull}114924[#114924]

Logs::
* Always check if index mode is logsdb {es-pull}116922[#116922]
* Prohibit changes to index mode, source, and sort settings during resize {es-pull}115812[#115812]

Machine Learning::
* Fix bug in ML autoscaling when some node info is unavailable {es-pull}116650[#116650]
* Fix deberta tokenizer bug caused by bug in normalizer {es-pull}117189[#117189]
* Hides `hugging_face_elser` service from the `GET _inference/_services API` {es-pull}116664[#116664] (issue: {es-issue}116644[#116644])
* Mitigate IOSession timeouts {es-pull}115414[#115414] (issues: {es-issue}114385[#114385], {es-issue}114327[#114327], {es-issue}114105[#114105], {es-issue}114232[#114232])
* Propagate scoring function through random sampler {es-pull}116957[#116957] (issue: {es-issue}110134[#110134])
* Update Deberta tokenizer {es-pull}116358[#116358]
* Wait for up to 2 seconds for yellow status before starting search {es-pull}115938[#115938] (issues: {es-issue}107777[#107777], {es-issue}105955[#105955], {es-issue}107815[#107815], {es-issue}112191[#112191])

Mapping::
* Change synthetic source logic for `constant_keyword` {es-pull}117182[#117182] (issue: {es-issue}117083[#117083])
* Ignore conflicting fields during dynamic mapping update {es-pull}114227[#114227] (issue: {es-issue}114228[#114228])

Network::
* Use underlying `ByteBuf` `refCount` for `ReleasableBytesReference` {es-pull}116211[#116211]

Ranking::
* Propagating nested `inner_hits` to the parent compound retriever {es-pull}116408[#116408] (issue: {es-issue}116397[#116397])

Relevance::
* Fix handling of bulk requests with semantic text fields and delete ops {es-pull}116942[#116942]

Search::
* Catch and handle disconnect exceptions in search {es-pull}115836[#115836]
* Fields caps does not honour ignore_unavailable {es-pull}116021[#116021] (issue: {es-issue}107767[#107767])
* Fix handling of time exceeded exception in fetch phase {es-pull}116676[#116676]
* Fix leak in `DfsQueryPhase` and introduce search disconnect stress test {es-pull}116060[#116060] (issue: {es-issue}115056[#115056])
* Inconsistency in the _analyzer api when the index is not included {es-pull}115930[#115930]
* Semantic text simple partial update {es-pull}116478[#116478]
* Updated Date Range to Follow Documentation When Assuming Missing Values {es-pull}112258[#112258] (issue: {es-issue}111484[#111484])
* Validate missing shards after the coordinator rewrite {es-pull}116382[#116382]
* _validate does not honour ignore_unavailable {es-pull}116656[#116656] (issue: {es-issue}116594[#116594])

Snapshot/Restore::
* Retry throttled snapshot deletions {es-pull}113237[#113237]

Vector Search::
* Update Semantic Query To Handle Zero Size Responses {es-pull}116277[#116277] (issue: {es-issue}116083[#116083])

Watcher::
* Watch Next Run Interval Resets On Shard Move or Node Restart {es-pull}115102[#115102] (issue: {es-issue}111433[#111433])

[[deprecation-9.0.0]]
[float]
=== Deprecations

Ingest Node::
* Fix `_type` deprecation on simulate pipeline API {es-pull}116259[#116259]

Machine Learning::
* [Inference API] Deprecate elser service {es-pull}113216[#113216]

Mapping::
* Deprecate `_source.mode` in mappings {es-pull}116689[#116689]

[[enhancement-9.0.0]]
[float]
=== Enhancements

Allocation::
* Only publish desired balance gauges on master {es-pull}115383[#115383]

Authorization::
* Add a `monitor_stats` privilege and allow that privilege for remote cluster privileges {es-pull}114964[#114964]
* [Security Solution] Add `create_index` to `kibana_system` role for index/DS `.logs-endpoint.action.responses-*` {es-pull}115241[#115241]

CRUD::
* Suppress merge-on-recovery for older indices {es-pull}113462[#113462]

Data streams::
* Adding a deprecation info API warning for data streams with old indices {es-pull}116447[#116447]
* Apm-data: disable date_detection for all apm data streams {es-pull}116995[#116995]

Distributed::
* Metrics for incremental bulk splits {es-pull}116765[#116765]
* Use Azure blob batch API to delete blobs in batches {es-pull}114566[#114566]

ES|QL::
* Add ES|QL `bit_length` function {es-pull}115792[#115792]
* ESQL: Honor skip_unavailable setting for nonmatching indices errors at planning time {es-pull}116348[#116348] (issue: {es-issue}114531[#114531])
* ESQL: Remove parent from `FieldAttribute` {es-pull}112881[#112881]
* ESQL: extract common filter from aggs {es-pull}115678[#115678]
* ESQL: optimise aggregations filtered by false/null into evals {es-pull}115858[#115858]
* ES|QL CCS uses `skip_unavailable` setting for handling disconnected remote clusters {es-pull}115266[#115266] (issue: {es-issue}114531[#114531])
* ES|QL: add metrics for functions {es-pull}114620[#114620]
* Esql Enable Date Nanos (tech preview) {es-pull}117080[#117080]
* Support partial sort fields in TopN pushdown {es-pull}116043[#116043] (issue: {es-issue}114515[#114515])
* [ES|QL] Implicit casting string literal to intervals {es-pull}115814[#115814] (issue: {es-issue}115352[#115352])

Health::
* Increase `replica_unassigned_buffer_time` default from 3s to 5s {es-pull}112834[#112834]

Indices APIs::
* Ensure class resource stream is closed in `ResourceUtils` {es-pull}116437[#116437]

Inference::
* Add version prefix to Inference Service API path {es-pull}117095[#117095]

Infra/Circuit Breakers::
* Add link to Circuit Breaker "Data too large" exception message {es-pull}113561[#113561]

Infra/Core::
* Support for unsigned 64 bit numbers in Cpu stats {es-pull}114681[#114681] (issue: {es-issue}112274[#112274])

Infra/Metrics::
* Add `ensureGreen` test method for use with `adminClient` {es-pull}113425[#113425]

Infra/Scripting::
* Add a `mustache.max_output_size_bytes` setting to limit the length of results from mustache scripts {es-pull}114002[#114002]

Ingest Node::
* Add postal_code support to the City and Enterprise databases {es-pull}114193[#114193]
* Add support for registered country fields for maxmind geoip databases {es-pull}114521[#114521]
* Adding support for additional mapping to simulate ingest API {es-pull}114742[#114742]
* Adding support for simulate ingest mapping adddition for indices with mappings that do not come from templates {es-pull}115359[#115359]
* Support IPinfo database configurations {es-pull}114548[#114548]
* Support more maxmind fields in the geoip processor {es-pull}114268[#114268]

Logs::
* Add logsdb telemetry {es-pull}115994[#115994]
* Add num docs and size to logsdb telemetry {es-pull}116128[#116128]
* Feature: re-structure document ID generation favoring _id inverted index compression {es-pull}104683[#104683]

Machine Learning::
* Add DeBERTa-V2/V3 tokenizer {es-pull}111852[#111852]
* Add special case for elastic reranker in inference API {es-pull}116962[#116962]
* Adding inference endpoint validation for `AzureAiStudioService` {es-pull}113713[#113713]
* Adds support for `input_type` field to Vertex inference service {es-pull}116431[#116431]
* Enable built-in Inference Endpoints and default for Semantic Text {es-pull}116931[#116931]
* Increase default `queue_capacity` to 10_000 and decrease max `queue_capacity` to 100_000 {es-pull}115041[#115041]
* Inference duration and error metrics {es-pull}115876[#115876]
* Remove all mentions of eis and gateway and deprecate flags that do {es-pull}116692[#116692]
* [Inference API] Add API to get configuration of inference services {es-pull}114862[#114862]
* [Inference API] Improve chunked results error message {es-pull}115807[#115807]

Network::
* Allow http unsafe buffers by default {es-pull}116115[#116115]

Recovery::
* Attempt to clean up index before remote transfer {es-pull}115142[#115142] (issue: {es-issue}104473[#104473])
* Trigger merges after recovery {es-pull}113102[#113102]

Reindex::
* Change Reindexing metrics unit from millis to seconds {es-pull}115721[#115721]

Relevance::
* Add query rules retriever {es-pull}114855[#114855]
* Add tracking for query rule types {es-pull}116357[#116357]

Search::
* Add Search Phase APM metrics {es-pull}113194[#113194]
* Add `docvalue_fields` Support for `dense_vector` Fields {es-pull}114484[#114484] (issue: {es-issue}108470[#108470])
* Add initial support for `semantic_text` field type {es-pull}113920[#113920]
* Adds access to flags no_sub_matches and no_overlapping_matches to hyphenation-decompounder-tokenfilter {es-pull}115459[#115459] (issue: {es-issue}97849[#97849])
* Better sizing `BytesRef` for Strings in Queries {es-pull}115655[#115655]
* Enable `_tier` based coordinator rewrites for all indices (not just mounted indices) {es-pull}115797[#115797]
* Only aggregations require at least one shard request {es-pull}115314[#115314]

Security::
* Add refresh `.security` index call between security migrations {es-pull}114879[#114879]

Snapshot/Restore::
* Improve message about insecure S3 settings {es-pull}116915[#116915]
* Retry `S3BlobContainer#getRegister` on all exceptions {es-pull}114813[#114813]
* Split searchable snapshot into multiple repo operations {es-pull}116918[#116918]
* Track shard snapshot progress during node shutdown {es-pull}112567[#112567]

Vector Search::
* Add support for bitwise inner-product in painless {es-pull}116082[#116082]

[[feature-9.0.0]]
[float]
=== New features

Data streams::
* Add default ILM policies and switch to ILM for apm-data plugin {es-pull}115687[#115687]

ES|QL::
* Add support for `BYTE_LENGTH` scalar function {es-pull}116591[#116591]
* Esql/lookup join grammar {es-pull}116515[#116515]
* Remove snapshot build restriction for match and qstr functions {es-pull}114482[#114482]

Search::
* ESQL - Add match operator (:) {es-pull}116819[#116819]

[[upgrade-9.0.0]]
[float]
=== Upgrades

Search::
* Upgrade to Lucene 10 {es-pull}114741[#114741]