[[release-notes-9.0.0]] == {es} version 9.0.0 coming[9.0.0] Also see <>. [[breaking-9.0.0]] [float] === Breaking changes Allocation:: * Increase minimum threshold in shard balancer {es-pull}115831[#115831] * Remove `cluster.routing.allocation.disk.watermark.enable_for_single_data_node` setting {es-pull}114207[#114207] * Remove cluster state from `/_cluster/reroute` response {es-pull}114231[#114231] (issue: {es-issue}88978[#88978]) Analysis:: * Snowball stemmers have been upgraded {es-pull}114146[#114146] * The 'german2' stemmer is now an alias for the 'german' snowball stemmer {es-pull}113614[#113614] * The 'persian' analyzer has stemmer by default {es-pull}113482[#113482] (issue: {es-issue}113050[#113050]) * The Korean dictionary for Nori has been updated {es-pull}114124[#114124] Cluster Coordination:: * Remove unsupported legacy value for `discovery.type` {es-pull}112903[#112903] Highlighting:: * Remove support for deprecated `force_source` highlighting parameter {es-pull}116943[#116943] Indices APIs:: * Apply more strict parsing of actions in bulk API {es-pull}115923[#115923] * Remove deprecated local attribute from alias APIs {es-pull}115393[#115393] Infra/REST API:: * Output a consistent format when generating error json {es-pull}90529[#90529] (issue: {es-issue}89387[#89387]) Ingest Node:: * Remove `ecs` option on `user_agent` processor {es-pull}116077[#116077] * Remove ignored fallback option on GeoIP processor {es-pull}116112[#116112] Mapping:: * Remove support for type, fields, `copy_to` and boost in metadata field definition {es-pull}116944[#116944] Search:: * Remove legacy params from range query {es-pull}116970[#116970] Snapshot/Restore:: * Remove deprecated `xpack.searchable.snapshot.allocate_on_rolling_restart` setting {es-pull}114202[#114202] [[bug-9.0.0]] [float] === Bug fixes Aggregations:: * Handle with `illegalArgumentExceptions` negative values in HDR percentile aggregations {es-pull}116174[#116174] (issue: {es-issue}115777[#115777]) Analysis:: * Adjust analyze limit exception to be a `bad_request` {es-pull}116325[#116325] CCS:: * Fix long metric deserialize & add - auto-resize needs to be set manually {es-pull}117105[#117105] (issue: {es-issue}116914[#116914]) CRUD:: * Preserve thread context when waiting for segment generation in RTG {es-pull}114623[#114623] * Standardize error code when bulk body is invalid {es-pull}114869[#114869] Data streams:: * Add missing header in `put_data_lifecycle` rest-api-spec {es-pull}116292[#116292] EQL:: * Don't use a `BytesStreamOutput` to copy keys in `BytesRefBlockHash` {es-pull}114819[#114819] (issue: {es-issue}114599[#114599]) ES|QL:: * Added stricter range type checks and runtime warnings for ENRICH {es-pull}115091[#115091] (issues: {es-issue}107357[#107357], {es-issue}116799[#116799]) * Don't return TEXT type for functions that take TEXT {es-pull}114334[#114334] (issues: {es-issue}111537[#111537], {es-issue}114333[#114333]) * ESQL: Fix sorts containing `_source` {es-pull}116980[#116980] (issue: {es-issue}116659[#116659]) * ESQL: fix the column position in errors {es-pull}117153[#117153] * ES|QL: Fix stats by constant expression {es-pull}114899[#114899] * Fix NPE in `EnrichLookupService` on mixed clusters with <8.14 versions {es-pull}116583[#116583] (issues: {es-issue}116529[#116529], {es-issue}116544[#116544]) * Fix TDigestState.read CB leaks {es-pull}114303[#114303] (issue: {es-issue}114194[#114194]) * Fixing remote ENRICH by pushing the Enrich inside `FragmentExec` {es-pull}114665[#114665] (issue: {es-issue}105095[#105095]) * Use `SearchStats` instead of field.isAggregatable in data node planning {es-pull}115744[#115744] (issue: {es-issue}115737[#115737]) * [ESQL] Fix Binary Comparisons on Date Nanos {es-pull}116346[#116346] * [ES|QL] To_DatePeriod and To_TimeDuration return better error messages on `union_type` fields {es-pull}114934[#114934] Infra/CLI:: * Fix NPE on plugin sync {es-pull}115640[#115640] (issue: {es-issue}114818[#114818]) Infra/Metrics:: * Make `randomInstantBetween` always return value in range [minInstant, `maxInstant]` {es-pull}114177[#114177] Infra/REST API:: * Fixed a `NullPointerException` in `_capabilities` API when the `path` parameter is null. {es-pull}113413[#113413] (issue: {es-issue}113413[#113413]) Infra/Settings:: * Don't allow secure settings in YML config (109115) {es-pull}115779[#115779] (issue: {es-issue}109115[#109115]) Ingest Node:: * Add warning headers for ingest pipelines containing special characters {es-pull}114837[#114837] (issue: {es-issue}104411[#104411]) * Reducing error-level stack trace logging for normal events in `GeoIpDownloader` {es-pull}114924[#114924] Logs:: * Always check if index mode is logsdb {es-pull}116922[#116922] * Prohibit changes to index mode, source, and sort settings during resize {es-pull}115812[#115812] Machine Learning:: * Fix bug in ML autoscaling when some node info is unavailable {es-pull}116650[#116650] * Fix deberta tokenizer bug caused by bug in normalizer {es-pull}117189[#117189] * Hides `hugging_face_elser` service from the `GET _inference/_services API` {es-pull}116664[#116664] (issue: {es-issue}116644[#116644]) * Mitigate IOSession timeouts {es-pull}115414[#115414] (issues: {es-issue}114385[#114385], {es-issue}114327[#114327], {es-issue}114105[#114105], {es-issue}114232[#114232]) * Propagate scoring function through random sampler {es-pull}116957[#116957] (issue: {es-issue}110134[#110134]) * Update Deberta tokenizer {es-pull}116358[#116358] * Wait for up to 2 seconds for yellow status before starting search {es-pull}115938[#115938] (issues: {es-issue}107777[#107777], {es-issue}105955[#105955], {es-issue}107815[#107815], {es-issue}112191[#112191]) Mapping:: * Change synthetic source logic for `constant_keyword` {es-pull}117182[#117182] (issue: {es-issue}117083[#117083]) * Ignore conflicting fields during dynamic mapping update {es-pull}114227[#114227] (issue: {es-issue}114228[#114228]) Network:: * Use underlying `ByteBuf` `refCount` for `ReleasableBytesReference` {es-pull}116211[#116211] Ranking:: * Propagating nested `inner_hits` to the parent compound retriever {es-pull}116408[#116408] (issue: {es-issue}116397[#116397]) Relevance:: * Fix handling of bulk requests with semantic text fields and delete ops {es-pull}116942[#116942] Search:: * Catch and handle disconnect exceptions in search {es-pull}115836[#115836] * Fields caps does not honour ignore_unavailable {es-pull}116021[#116021] (issue: {es-issue}107767[#107767]) * Fix handling of time exceeded exception in fetch phase {es-pull}116676[#116676] * Fix leak in `DfsQueryPhase` and introduce search disconnect stress test {es-pull}116060[#116060] (issue: {es-issue}115056[#115056]) * Inconsistency in the _analyzer api when the index is not included {es-pull}115930[#115930] * Semantic text simple partial update {es-pull}116478[#116478] * Updated Date Range to Follow Documentation When Assuming Missing Values {es-pull}112258[#112258] (issue: {es-issue}111484[#111484]) * Validate missing shards after the coordinator rewrite {es-pull}116382[#116382] * _validate does not honour ignore_unavailable {es-pull}116656[#116656] (issue: {es-issue}116594[#116594]) Snapshot/Restore:: * Retry throttled snapshot deletions {es-pull}113237[#113237] Vector Search:: * Update Semantic Query To Handle Zero Size Responses {es-pull}116277[#116277] (issue: {es-issue}116083[#116083]) Watcher:: * Watch Next Run Interval Resets On Shard Move or Node Restart {es-pull}115102[#115102] (issue: {es-issue}111433[#111433]) [[deprecation-9.0.0]] [float] === Deprecations Ingest Node:: * Fix `_type` deprecation on simulate pipeline API {es-pull}116259[#116259] Machine Learning:: * [Inference API] Deprecate elser service {es-pull}113216[#113216] Mapping:: * Deprecate `_source.mode` in mappings {es-pull}116689[#116689] [[enhancement-9.0.0]] [float] === Enhancements Allocation:: * Only publish desired balance gauges on master {es-pull}115383[#115383] Authorization:: * Add a `monitor_stats` privilege and allow that privilege for remote cluster privileges {es-pull}114964[#114964] * [Security Solution] Add `create_index` to `kibana_system` role for index/DS `.logs-endpoint.action.responses-*` {es-pull}115241[#115241] CRUD:: * Suppress merge-on-recovery for older indices {es-pull}113462[#113462] Data streams:: * Adding a deprecation info API warning for data streams with old indices {es-pull}116447[#116447] * Apm-data: disable date_detection for all apm data streams {es-pull}116995[#116995] Distributed:: * Metrics for incremental bulk splits {es-pull}116765[#116765] * Use Azure blob batch API to delete blobs in batches {es-pull}114566[#114566] ES|QL:: * Add ES|QL `bit_length` function {es-pull}115792[#115792] * ESQL: Honor skip_unavailable setting for nonmatching indices errors at planning time {es-pull}116348[#116348] (issue: {es-issue}114531[#114531]) * ESQL: Remove parent from `FieldAttribute` {es-pull}112881[#112881] * ESQL: extract common filter from aggs {es-pull}115678[#115678] * ESQL: optimise aggregations filtered by false/null into evals {es-pull}115858[#115858] * ES|QL CCS uses `skip_unavailable` setting for handling disconnected remote clusters {es-pull}115266[#115266] (issue: {es-issue}114531[#114531]) * ES|QL: add metrics for functions {es-pull}114620[#114620] * Esql Enable Date Nanos (tech preview) {es-pull}117080[#117080] * Support partial sort fields in TopN pushdown {es-pull}116043[#116043] (issue: {es-issue}114515[#114515]) * [ES|QL] Implicit casting string literal to intervals {es-pull}115814[#115814] (issue: {es-issue}115352[#115352]) Health:: * Increase `replica_unassigned_buffer_time` default from 3s to 5s {es-pull}112834[#112834] Indices APIs:: * Ensure class resource stream is closed in `ResourceUtils` {es-pull}116437[#116437] Inference:: * Add version prefix to Inference Service API path {es-pull}117095[#117095] Infra/Circuit Breakers:: * Add link to Circuit Breaker "Data too large" exception message {es-pull}113561[#113561] Infra/Core:: * Support for unsigned 64 bit numbers in Cpu stats {es-pull}114681[#114681] (issue: {es-issue}112274[#112274]) Infra/Metrics:: * Add `ensureGreen` test method for use with `adminClient` {es-pull}113425[#113425] Infra/Scripting:: * Add a `mustache.max_output_size_bytes` setting to limit the length of results from mustache scripts {es-pull}114002[#114002] Ingest Node:: * Add postal_code support to the City and Enterprise databases {es-pull}114193[#114193] * Add support for registered country fields for maxmind geoip databases {es-pull}114521[#114521] * Adding support for additional mapping to simulate ingest API {es-pull}114742[#114742] * Adding support for simulate ingest mapping adddition for indices with mappings that do not come from templates {es-pull}115359[#115359] * Support IPinfo database configurations {es-pull}114548[#114548] * Support more maxmind fields in the geoip processor {es-pull}114268[#114268] Logs:: * Add logsdb telemetry {es-pull}115994[#115994] * Add num docs and size to logsdb telemetry {es-pull}116128[#116128] * Feature: re-structure document ID generation favoring _id inverted index compression {es-pull}104683[#104683] Machine Learning:: * Add DeBERTa-V2/V3 tokenizer {es-pull}111852[#111852] * Add special case for elastic reranker in inference API {es-pull}116962[#116962] * Adding inference endpoint validation for `AzureAiStudioService` {es-pull}113713[#113713] * Adds support for `input_type` field to Vertex inference service {es-pull}116431[#116431] * Enable built-in Inference Endpoints and default for Semantic Text {es-pull}116931[#116931] * Increase default `queue_capacity` to 10_000 and decrease max `queue_capacity` to 100_000 {es-pull}115041[#115041] * Inference duration and error metrics {es-pull}115876[#115876] * Remove all mentions of eis and gateway and deprecate flags that do {es-pull}116692[#116692] * [Inference API] Add API to get configuration of inference services {es-pull}114862[#114862] * [Inference API] Improve chunked results error message {es-pull}115807[#115807] Network:: * Allow http unsafe buffers by default {es-pull}116115[#116115] Recovery:: * Attempt to clean up index before remote transfer {es-pull}115142[#115142] (issue: {es-issue}104473[#104473]) * Trigger merges after recovery {es-pull}113102[#113102] Reindex:: * Change Reindexing metrics unit from millis to seconds {es-pull}115721[#115721] Relevance:: * Add query rules retriever {es-pull}114855[#114855] * Add tracking for query rule types {es-pull}116357[#116357] Search:: * Add Search Phase APM metrics {es-pull}113194[#113194] * Add `docvalue_fields` Support for `dense_vector` Fields {es-pull}114484[#114484] (issue: {es-issue}108470[#108470]) * Add initial support for `semantic_text` field type {es-pull}113920[#113920] * Adds access to flags no_sub_matches and no_overlapping_matches to hyphenation-decompounder-tokenfilter {es-pull}115459[#115459] (issue: {es-issue}97849[#97849]) * Better sizing `BytesRef` for Strings in Queries {es-pull}115655[#115655] * Enable `_tier` based coordinator rewrites for all indices (not just mounted indices) {es-pull}115797[#115797] * Only aggregations require at least one shard request {es-pull}115314[#115314] Security:: * Add refresh `.security` index call between security migrations {es-pull}114879[#114879] Snapshot/Restore:: * Improve message about insecure S3 settings {es-pull}116915[#116915] * Retry `S3BlobContainer#getRegister` on all exceptions {es-pull}114813[#114813] * Split searchable snapshot into multiple repo operations {es-pull}116918[#116918] * Track shard snapshot progress during node shutdown {es-pull}112567[#112567] Vector Search:: * Add support for bitwise inner-product in painless {es-pull}116082[#116082] [[feature-9.0.0]] [float] === New features Data streams:: * Add default ILM policies and switch to ILM for apm-data plugin {es-pull}115687[#115687] ES|QL:: * Add support for `BYTE_LENGTH` scalar function {es-pull}116591[#116591] * Esql/lookup join grammar {es-pull}116515[#116515] * Remove snapshot build restriction for match and qstr functions {es-pull}114482[#114482] Search:: * ESQL - Add match operator (:) {es-pull}116819[#116819] [[upgrade-9.0.0]] [float] === Upgrades Search:: * Upgrade to Lucene 10 {es-pull}114741[#114741]