[[release-notes-8.0.0]] == {es} version 8.0.0 The following list are changes in 8.0.0 as compared to 7.17.0, and combines release notes from the 8.0.0-alpha1, -alpha2, -beta1, -rc1 and -rc2 releases. Also see <>. [[known-issues-8.0.0]] [float] === Known issues * If you installed {es} from an archive on an aarch64 platform like Linux ARM or macOS M1, the `elastic` user password and {kib} enrollment token are not generated automatically when starting your node for the first time. + -- After the node starts, generate the `elastic` password with the <> tool: [source,bash] ---- bin/elasticsearch-reset-password -u elastic ---- Then, create an enrollment token for {kib} with the <> tool: [source,bash] ---- bin/elasticsearch-create-enrollment-token -s kibana ---- -- [[breaking-8.0.0]] [float] === Breaking changes Aggregations:: * Percentiles aggregation: disallow specifying same percentile values twice {es-pull}52257[#52257] (issue: {es-issue}51871[#51871]) * Remove adjacency matrix setting {es-pull}46327[#46327] (issues: {es-issue}46257[#46257], {es-issue}46324[#46324]) * Remove `MovingAverage` pipeline aggregation {es-pull}39328[#39328] * Remove deprecated `_time` and `_term` sort orders {es-pull}39450[#39450] * Remove deprecated date histo interval {es-pull}75000[#75000] Allocation:: * Require single data nodes to respect disk watermarks {es-pull}73737[#73737] (issues: {es-issue}55805[#55805], {es-issue}73733[#73733]) * Remove `include_relocations` setting {es-pull}47717[#47717] (issues: {es-issue}46079[#46079], {es-issue}47443[#47443]) Analysis:: * Cleanup versioned deprecations in analysis {es-pull}41560[#41560] (issue: {es-issue}41164[#41164]) * Remove preconfigured `delimited_payload_filter` {es-pull}43686[#43686] (issues: {es-issue}41560[#41560], {es-issue}43684[#43684]) Authentication:: * Always add file and native realms unless explicitly disabled {es-pull}69096[#69096] (issue: {es-issue}50892[#50892]) * Do not set a NameID format in Policy by default {es-pull}44090[#44090] (issue: {es-issue}40353[#40353]) * Make order setting mandatory for Realm config {es-pull}51195[#51195] (issue: {es-issue}37614[#37614]) CCR:: * Avoid auto following leader system indices in CCR {es-pull}72815[#72815] (issue: {es-issue}67686[#67686]) Cluster Coordination:: * Remove join timeout {es-pull}60873[#60873] (issue: {es-issue}60872[#60872]) * Remove node filters for voting config exclusions {es-pull}55673[#55673] (issues: {es-issue}47990[#47990], {es-issue}50836[#50836]) * Remove support for delaying state recovery pending master {es-pull}53845[#53845] (issue: {es-issue}51806[#51806]) Distributed:: * Remove synced flush {es-pull}50882[#50882] (issues: {es-issue}50776[#50776], {es-issue}50835[#50835]) * Remove the `cluster.remote.connect` setting {es-pull}54175[#54175] (issue: {es-issue}53924[#53924]) Engine:: * Force merge should reject requests with `only_expunge_deletes` and `max_num_segments` set {es-pull}44761[#44761] (issue: {es-issue}43102[#43102]) * Remove per-type indexing stats {es-pull}47203[#47203] (issue: {es-issue}41059[#41059]) * Remove translog retention settings {es-pull}51697[#51697] (issue: {es-issue}50775[#50775]) Features/CAT APIs:: * Remove the deprecated `local` parameter for `_cat/indices` {es-pull}64868[#64868] (issue: {es-issue}62198[#62198]) * Remove the deprecated `local` parameter for `_cat/shards` {es-pull}64867[#64867] (issue: {es-issue}62197[#62197]) Features/Features:: * Remove deprecated `._tier` allocation filtering settings {es-pull}73074[#73074] (issue: {es-issue}72835[#72835]) Features/ILM+SLM:: * Add lower bound on `poll_interval` {es-pull}39593[#39593] (issue: {es-issue}39163[#39163]) * Make the ILM `freeze` action a no-op {es-pull}77158[#77158] (issue: {es-issue}70192[#70192]) * Always enforce default tier preference {es-pull}79751[#79751] (issue: {es-issue}76147[#76147]) * Validate that snapshot repository exists for ILM policies at creation/update time {es-pull}78468[#78468] (issues: {es-issue}72957[#72957], {es-issue}77657[#77657]) * Default `cluster.routing.allocation.enforce_default_tier_preference` to `true` {es-pull}79275[#79275] (issues: {es-issue}76147[#76147], {es-issue}79210[#79210]) Features/Indices APIs:: * Change `prefer_v2_templates` parameter to default to true {es-pull}55489[#55489] (issues: {es-issue}53101[#53101], {es-issue}55411[#55411]) * Remove deprecated `_upgrade` API {es-pull}64732[#64732] (issue: {es-issue}21337[#21337]) * Remove local parameter for get field mapping request {es-pull}55100[#55100] (issue: {es-issue}55099[#55099]) * Remove `include_type_name` parameter from REST layer {es-pull}48632[#48632] (issue: {es-issue}41059[#41059]) * Remove the `template` field in index templates {es-pull}49460[#49460] (issue: {es-issue}21009[#21009]) * Remove endpoint for freezing indices {es-pull}78918[#78918] (issues: {es-issue}70192[#70192], {es-issue}77273[#77273]) Features/Watcher:: * Move watcher history to data stream {es-pull}64252[#64252] Geo:: * Disallow creating `geo_shape` mappings with deprecated parameters {es-pull}70850[#70850] (issue: {es-issue}32039[#32039]) * Remove bounding box query `type` parameter {es-pull}74536[#74536] Infra/Circuit Breakers:: * Fixed synchronizing inflight breaker with internal variable {es-pull}40878[#40878] Infra/Core:: * Limit processors by available processors {es-pull}44894[#44894] (issue: {es-issue}44889[#44889]) * Remove `nodes/0` folder prefix from data path {es-pull}42489[#42489] * Remove `bootstrap.system_call_filter` setting {es-pull}72848[#72848] * Remove `fixed_auto_queue_size` threadpool type {es-pull}52280[#52280] * Remove `node.max_local_storage_nodes` {es-pull}42428[#42428] (issue: {es-issue}42426[#42426]) * Remove camel case named date/time formats {es-pull}60044[#60044] * Remove legacy role settings {es-pull}71163[#71163] (issues: {es-issue}54998[#54998], {es-issue}66409[#66409], {es-issue}71143[#71143]) * Remove `processors` setting {es-pull}45905[#45905] (issue: {es-issue}45855[#45855]) * Remove the `local` parameter of `/_cat/nodes` {es-pull}50594[#50594] (issues: {es-issue}50088[#50088], {es-issue}50499[#50499]) * Remove the listener thread pool {es-pull}53314[#53314] (issue: {es-issue}53049[#53049]) * Remove the node local storage setting {es-pull}54381[#54381] (issue: {es-issue}54374[#54374]) * Remove the `pidfile` setting {es-pull}45940[#45940] (issue: {es-issue}45938[#45938]) * Removes `week_year` date format {es-pull}63384[#63384] (issue: {es-issue}60707[#60707]) * System indices treated as restricted indices {es-pull}74212[#74212] (issue: {es-issue}69298[#69298]) * Remove Joda dependency {es-pull}79007[#79007] * Remove Joda support from date formatters {es-pull}78990[#78990] * All system indices are hidden indices {es-pull}79512[#79512] Infra/Logging:: * Remove slowlog level {es-pull}57591[#57591] (issue: {es-issue}56171[#56171]) Infra/Plugins:: * Remove deprecated basic license feature enablement settings {es-pull}56211[#56211] (issue: {es-issue}54745[#54745]) Infra/REST API:: * Remove content type required setting {es-pull}61043[#61043] * Remove deprecated endpoints containing `_xpack` {es-pull}48170[#48170] (issue: {es-issue}35958[#35958]) * Remove deprecated endpoints of hot threads API {es-pull}55109[#55109] (issue: {es-issue}52640[#52640]) * Allow parsing Content-Type and Accept headers with version {es-pull}61427[#61427] Infra/Resiliency:: * Fail node containing ancient closed index {es-pull}44264[#44264] (issues: {es-issue}21830[#21830], {es-issue}41731[#41731], {es-issue}44230[#44230]) Infra/Scripting:: * Consolidate script parsing from object {es-pull}59507[#59507] (issue: {es-issue}59391[#59391]) * Move `script_cache` into _nodes/stats {es-pull}59265[#59265] (issues: {es-issue}50152[#50152], {es-issue}59262[#59262]) * Remove general cache settings {es-pull}59262[#59262] (issue: {es-issue}50152[#50152]) Infra/Settings:: * Change default value of `action.destructive_requires_name` to `true` {es-pull}66908[#66908] (issue: {es-issue}61074[#61074]) * Forbid settings without a namespace {es-pull}45947[#45947] (issues: {es-issue}45905[#45905], {es-issue}45940[#45940]) Ingest:: * Remove default maxmind GeoIP databases from distribution {es-pull}78362[#78362] (issue: {es-issue}68920[#68920]) License:: * Set `xpack.security.enabled` to true for all licenses {es-pull}72300[#72300] * Enforce license expiration {es-pull}79671[#79671] Machine Learning:: * Remove deprecated `_xpack` endpoints {es-pull}59870[#59870] (issues: {es-issue}35958[#35958], {es-issue}48170[#48170]) * Remove the ability to update datafeed's `job_id` {es-pull}44752[#44752] (issue: {es-issue}44616[#44616]) * Remove `allow_no_datafeeds` and `allow_no_jobs` parameters from APIs {es-pull}80048[#80048] (issue: {es-issue}60732[#60732]) Mapping:: * Remove `boost` mapping parameter {es-pull}62639[#62639] (issue: {es-issue}62623[#62623]) * Remove support for chained multi-fields {es-pull}42333[#42333] (issues: {es-issue}41267[#41267], {es-issue}41926[#41926]) * Remove support for string in `unmapped_type` {es-pull}45675[#45675] * Removes typed URLs from mapping APIs {es-pull}41676[#41676] Network:: * Remove client feature tracking {es-pull}44929[#44929] (issues: {es-issue}31020[#31020], {es-issue}42538[#42538], {es-issue}44667[#44667]) * Remove escape hatch permitting incompatible builds {es-pull}65753[#65753] (issues: {es-issue}65249[#65249], {es-issue}65601[#65601]) Packaging:: * Remove SysV init support {es-pull}51716[#51716] (issue: {es-issue}51480[#51480]) * Remove support for `JAVA_HOME` {es-pull}69149[#69149] (issue: {es-issue}55820[#55820]) * Remove no-jdk distributions {es-pull}76896[#76896] (issue: {es-issue}65109[#65109]) * Require Java 17 for running Elasticsearch {es-pull}79873[#79873] Recovery:: * Remove dangling index auto import functionality {es-pull}59698[#59698] (issue: {es-issue}48366[#48366]) Reindex:: * Reindex from Remote encoding {es-pull}41007[#41007] (issue: {es-issue}40303[#40303]) * Reindex remove outer level size {es-pull}43373[#43373] (issues: {es-issue}24344[#24344], {es-issue}41894[#41894]) Rollup:: * `RollupStart` endpoint should return OK if job already started {es-pull}41502[#41502] (issues: {es-issue}35928[#35928], {es-issue}39845[#39845]) Search:: * Decouple shard allocation awareness from search and get requests {es-pull}45735[#45735] (issue: {es-issue}43453[#43453]) * Fix range query on date fields for number inputs {es-pull}63692[#63692] (issue: {es-issue}63680[#63680]) * Make fuzziness reject illegal values earlier {es-pull}33511[#33511] * Make remote cluster resolution stricter {es-pull}40419[#40419] (issue: {es-issue}37863[#37863]) * Parse empty first line in msearch request body as action metadata {es-pull}41011[#41011] (issue: {es-issue}39841[#39841]) * Remove `CommonTermsQuery` and `cutoff_frequency` param {es-pull}42654[#42654] (issue: {es-issue}37096[#37096]) * Remove `type` query {es-pull}47207[#47207] (issue: {es-issue}41059[#41059]) * Remove `use_field_mapping` format option for docvalue fields {es-pull}55622[#55622] * Remove deprecated `SimpleQueryStringBuilder` parameters {es-pull}57200[#57200] * Remove deprecated `search.remote` settings {es-pull}42381[#42381] (issues: {es-issue}33413[#33413], {es-issue}38556[#38556]) * Remove deprecated sort options: `nested_path` and `nested_filter` {es-pull}42809[#42809] (issue: {es-issue}27098[#27098]) * Remove deprecated vector functions {es-pull}48725[#48725] (issue: {es-issue}48604[#48604]) * Remove support for `_type` in searches {es-pull}68564[#68564] (issues: {es-issue}41059[#41059], {es-issue}68311[#68311]) * Remove support for sparse vectors {es-pull}48781[#48781] (issue: {es-issue}48368[#48368]) * Remove the object format for `indices_boost` {es-pull}55078[#55078] * Removes type from `TermVectors` APIs {es-pull}42198[#42198] (issue: {es-issue}41059[#41059]) * Removes typed endpoint from search and related APIs {es-pull}41640[#41640] * Set max allowed size for stored async response {es-pull}74455[#74455] (issue: {es-issue}67594[#67594]) * `indices.query.bool.max_clause_count` now limits all query clauses {es-pull}75297[#75297] Security:: * Remove obsolete security settings {es-pull}40496[#40496] * Remove support of creating CA on the fly when generating certificates {es-pull}65590[#65590] (issue: {es-issue}61884[#61884]) * Remove the `id` field from the `InvalidateApiKey` API {es-pull}66671[#66671] (issue: {es-issue}66317[#66317]) * Remove the migrate tool {es-pull}42174[#42174] * Compress audit logs {es-pull}64472[#64472] (issue: {es-issue}63843[#63843]) * Remove insecure settings {es-pull}46147[#46147] (issue: {es-issue}45947[#45947]) * Remove `kibana_dashboard_only_user` reserved role {es-pull}76507[#76507] Snapshot/Restore:: * Blob store compress default to `true` {es-pull}40033[#40033] * Get snapshots support for multiple repositories {es-pull}42090[#42090] (issue: {es-issue}41210[#41210]) * Remove repository stats API {es-pull}62309[#62309] (issue: {es-issue}62297[#62297]) * Remove frozen cache setting leniency {es-pull}71013[#71013] (issue: {es-issue}70341[#70341]) * Adjust snapshot index resolution behavior to be more intuitive {es-pull}79670[#79670] (issue: {es-issue}78320[#78320]) TLS:: * Reject misconfigured/ambiguous SSL server config {es-pull}45892[#45892] * Remove support for configurable PKCS#11 keystores {es-pull}75404[#75404] * Remove the client transport profile filter {es-pull}43236[#43236] [[breaking-java-8.0.0]] [float] === Breaking Java changes Authentication:: * Mandate x-pack REST handler installed {es-pull}71061[#71061] (issue: {es-issue}70523[#70523]) CCR:: * Remove the `CcrClient` {es-pull}42816[#42816] CRUD:: * Remove types from `BulkRequest` {es-pull}46983[#46983] (issue: {es-issue}41059[#41059]) * Remove `Client.prepareIndex(index, type, id)` method {es-pull}48443[#48443] Client:: * Remove `SecurityClient` from x-pack {es-pull}42471[#42471] Features/ILM+SLM:: * Remove the `ILMClient` {es-pull}42817[#42817] Features/Monitoring:: * Remove `MonitoringClient` from x-pack {es-pull}42770[#42770] Features/Watcher:: * Remove `WatcherClient` from x-pack {es-pull}42815[#42815] Infra/Core:: * Remove `XPackClient` from x-pack {es-pull}42729[#42729] * Remove the transport client {es-pull}42538[#42538] * Remove transport client from x-pack {es-pull}42202[#42202] Infra/REST API:: * Copy HTTP headers to `ThreadContext` strictly {es-pull}45945[#45945] Machine Learning:: * Remove the `MachineLearningClient` {es-pull}43108[#43108] Mapping:: * Remove type filter from `GetMappings` API {es-pull}47364[#47364] (issue: {es-issue}41059[#41059]) * Remove `type` parameter from `PutMappingRequest.buildFromSimplifiedDef()` {es-pull}50844[#50844] (issue: {es-issue}41059[#41059]) * Remove unused parameter from `MetadataFieldMapper.TypeParser#getDefault()` {es-pull}51219[#51219] * Remove `type` parameter from `CIR.mapping(type, object...)` {es-pull}50739[#50739] (issue: {es-issue}41059[#41059]) Search:: * Removes types from `SearchRequest` and `QueryShardContext` {es-pull}42112[#42112] Snapshot/Restore:: * Remove deprecated repository methods {es-pull}42359[#42359] (issue: {es-issue}42213[#42213]) [[deprecation-8.0.0]] [float] === Deprecations Authentication:: * Deprecate setup-passwords tool {es-pull}76902[#76902] CRUD:: * Remove `indices_segments` 'verbose' parameter {es-pull}78451[#78451] (issue: {es-issue}75955[#75955]) Engine:: * Deprecate setting `max_merge_at_once_explicit` {es-pull}80574[#80574] Machine Learning:: * Deprecate `estimated_heap_memory_usage_bytes` and replace with `model_size_bytes` {es-pull}80554[#80554] Monitoring:: * Add deprecation info API entries for deprecated monitoring settings {es-pull}78799[#78799] * Automatically install monitoring templates at plugin initialization {es-pull}78350[#78350] * Remove Monitoring ingest pipelines {es-pull}77459[#77459] (issue: {es-issue}50770[#50770]) Search:: * Configure `IndexSearcher.maxClauseCount()` based on node characteristics {es-pull}81525[#81525] (issue: {es-issue}46433[#46433]) Transform:: * Improve transform deprecation messages {es-pull}81847[#81847] (issues: {es-issue}81521[#81521], {es-issue}81523[#81523]) [[feature-8.0.0]] [float] === New features Security:: * Auto-configure TLS for new nodes of new clusters {es-pull}77231[#77231] (issues: {es-issue}75144[#75144], {es-issue}75704[#75704]) Snapshot/Restore:: * Support IAM roles for Kubernetes service accounts {es-pull}81255[#81255] (issue: {es-issue}52625[#52625]) Watcher:: * Use `startsWith` rather than exact matches for Watcher history template names {es-pull}82396[#82396] [[enhancement-8.0.0]] [float] === Enhancements Analysis:: * Move `reload_analyzers` endpoint to x-pack {es-pull}43559[#43559] Authentication:: * Reset elastic password CLI tool {es-pull}74892[#74892] (issues: {es-issue}70113[#70113], {es-issue}74890[#74890]) * Autogenerate and print elastic password on startup {es-pull}77291[#77291] * Enroll Kibana API uses Service Accounts {es-pull}76370[#76370] * Add `reset-kibana-system-user` tool {es-pull}77322[#77322] * New CLI tool to reset password for built-in users {es-pull}79709[#79709] * Auto-configure the `elastic` user password {es-pull}78306[#78306] Authorization:: * Granting `kibana_system` reserved role access to "all" privileges to `.internal.preview.alerts*` index {es-pull}80889[#80889] (issues: {es-issue}76624[#76624], {es-issue}80746[#80746], {es-issue}116374[#116374]) * Granting `kibana_system` reserved role access to "all" privileges to .preview.alerts* index {es-pull}80746[#80746] * Granting editor and viewer roles access to alerts-as-data indices {es-pull}81285[#81285] Cluster Coordination:: * Prevent downgrades from 8.x to 7.x {es-pull}78586[#78586] (issues: {es-issue}42489[#42489], {es-issue}52414[#52414]) * Prevent downgrades from 8.x to 7.x {es-pull}78638[#78638] (issues: {es-issue}42489[#42489], {es-issue}52414[#52414]) * Make `TaskBatcher` less lock-heavy {es-pull}82227[#82227] (issue: {es-issue}77466[#77466]) Data streams:: * Data stream support read and write with custom routing and partition size {es-pull}74394[#74394] (issue: {es-issue}74390[#74390]) EQL:: * Add option for returning results from the tail of the stream {es-pull}64869[#64869] (issue: {es-issue}58646[#58646]) * Introduce case insensitive variant `in~` {es-pull}68176[#68176] (issue: {es-issue}68172[#68172]) * Optimize redundant `toString` {es-pull}71070[#71070] (issue: {es-issue}70681[#70681]) Engine:: * Always use soft-deletes in `InternalEngine` {es-pull}50415[#50415] * Remove translog retention policy {es-pull}51417[#51417] (issue: {es-issue}50775[#50775]) Features/CAT APIs:: * Remove `size` and add `time` params to `_cat/threadpool` {es-pull}55736[#55736] (issue: {es-issue}54478[#54478]) Features/ILM+SLM:: * Allow for setting the total shards per node in the Allocate ILM action {es-pull}76794[#76794] (issue: {es-issue}76775[#76775]) * Inject migrate action regardless of allocate action {es-pull}79090[#79090] (issue: {es-issue}76147[#76147]) * Make unchanged ILM policy updates into noop {es-pull}82240[#82240] (issue: {es-issue}82065[#82065]) * Avoid unnecessary `LifecycleExecutionState` recalculation {es-pull}81558[#81558] (issues: {es-issue}77466[#77466], {es-issue}79692[#79692]) Features/Indices APIs:: * Batch rollover cluster state updates {es-pull}79945[#79945] (issues: {es-issue}77466[#77466], {es-issue}79782[#79782]) * Reuse `MappingMetadata` instances in Metadata class {es-pull}80348[#80348] (issues: {es-issue}69772[#69772], {es-issue}77466[#77466]) Features/Stats:: * Add bulk stats track the bulk per shard {es-pull}52208[#52208] (issues: {es-issue}47345[#47345], {es-issue}50536[#50536]) Features/Watcher:: * Remove Watcher history clean up from monitoring {es-pull}67154[#67154] Infra/Core:: * Remove aliases exist action {es-pull}43430[#43430] * Remove indices exists action {es-pull}43164[#43164] * Remove types exists action {es-pull}43344[#43344] * Retain reference to stdout for exceptional cases {es-pull}77460[#77460] * Check whether stdout is a real console {es-pull}79882[#79882] * Share int, long, float, double, and byte pages {es-pull}75053[#75053] * Revert "Deprecate resolution loss on date field (#78921)" {es-pull}79914[#79914] (issue: {es-issue}78921[#78921]) * Add two missing entries to the deprecation information API {es-pull}80290[#80290] (issue: {es-issue}80233[#80233]) * Prevent upgrades to 8.0 without first upgrading to the last 7.x release {es-pull}82321[#82321] (issue: {es-issue}81865[#81865]) Infra/Logging:: * Make Elasticsearch JSON logs ECS compliant {es-pull}47105[#47105] (issue: {es-issue}46119[#46119]) Infra/REST API:: * Allow for field declaration for future compatible versions {es-pull}69774[#69774] (issue: {es-issue}51816[#51816]) * Introduce stability description to the REST API specification {es-pull}38413[#38413] * Parsing: Validate that fields are not registered twice {es-pull}70243[#70243] * Support response content-type with versioned media type {es-pull}65500[#65500] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] Typed endpoints for index and get APIs {es-pull}69131[#69131] (issue: {es-issue}54160[#54160]) * [REST API Compatibility] Typed endpoints for put and get mapping and get field mappings {es-pull}71721[#71721] (issues: {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Allow `copy_settings` flag for resize operations {es-pull}75184[#75184] (issues: {es-issue}38514[#38514], {es-issue}51816[#51816]) * [REST API Compatibility] Allow for type in geo shape query {es-pull}74553[#74553] (issues: {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Always return `adjust_pure_negative` value {es-pull}75182[#75182] (issues: {es-issue}49543[#49543], {es-issue}51816[#51816]) * [REST API Compatibility] Clean up x-pack/plugin rest compat tests {es-pull}74701[#74701] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] Do not return `_doc` for empty mappings in template {es-pull}75448[#75448] (issues: {es-issue}51816[#51816], {es-issue}54160[#54160], {es-issue}70966[#70966], {es-issue}74544[#74544]) * [REST API Compatibility] Dummy REST action for `indices.upgrade` API {es-pull}75136[#75136] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] REST Terms vector typed response {es-pull}73117[#73117] * [REST API Compatibility] Rename `BulkItemResponse.Failure` type field {es-pull}74937[#74937] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] Type metadata for docs used in simulate request {es-pull}74222[#74222] (issues: {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Typed `TermLookups` {es-pull}74544[#74544] (issues: {es-issue}46943[#46943], {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Typed and x-pack graph explore API {es-pull}74185[#74185] (issues: {es-issue}46935[#46935], {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Typed endpoint for bulk API {es-pull}73571[#73571] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] Typed endpoint for multi-get API {es-pull}73878[#73878] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] Typed endpoints for `RestUpdateAction` and `RestDeleteAction` {es-pull}73115[#73115] (issues: {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Typed endpoints for `get_source` API {es-pull}73957[#73957] (issues: {es-issue}46587[#46587], {es-issue}46931[#46931], {es-issue}51816[#51816]) * [REST API Compatibility] Typed endpoints for explain API {es-pull}73901[#73901] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] Typed endpoints for search `_count` API {es-pull}73958[#73958] (issues: {es-issue}42112[#42112], {es-issue}51816[#51816]) * [REST API Compatibility] Typed indexing stats {es-pull}74181[#74181] (issues: {es-issue}47203[#47203], {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Types for percolate query API {es-pull}74698[#74698] (issues: {es-issue}46985[#46985], {es-issue}51816[#51816], {es-issue}54160[#54160], {es-issue}74689[#74689]) * [REST API Compatibility] Validate query typed API {es-pull}74171[#74171] (issues: {es-issue}46927[#46927], {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Voting config exclusion exception message {es-pull}75406[#75406] (issues: {es-issue}51816[#51816], {es-issue}55291[#55291]) * [REST API Compatibility] `MoreLikeThisQuery` with types {es-pull}75123[#75123] (issues: {es-issue}42198[#42198], {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Update and delete by query using size field {es-pull}69606[#69606] * [REST API Compatibility] Indicies boost in object format {es-pull}74422[#74422] (issues: {es-issue}51816[#51816], {es-issue}55078[#55078]) * [REST API Compatibility] Typed endpoints for search and related endpoints {es-pull}72155[#72155] (issues: {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Allow to use size `-1` {es-pull}75342[#75342] (issues: {es-issue}51816[#51816], {es-issue}69548[#69548], {es-issue}70209[#70209]) * [REST API Compatibility] Ignore `use_field_mapping` option for docvalue {es-pull}74435[#74435] (issue: {es-issue}55622[#55622]) * [REST API Compatibility] `_time` and `_term` sort orders {es-pull}74919[#74919] (issues: {es-issue}39450[#39450], {es-issue}51816[#51816]) * [REST API Compatability] `template` parameter and field on PUT index template {es-pull}71238[#71238] (issues: {es-issue}49460[#49460], {es-issue}51816[#51816], {es-issue}68905[#68905]) * [REST API Compatibility] Make query registration easier {es-pull}75722[#75722] (issue: {es-issue}51816[#51816]) * [REST API Compatibility] Typed query {es-pull}75453[#75453] (issues: {es-issue}47207[#47207], {es-issue}51816[#51816], {es-issue}54160[#54160]) * [REST API Compatibility] Deprecate the use of synced flush {es-pull}75372[#75372] (issues: {es-issue}50882[#50882], {es-issue}51816[#51816]) * [REST API Compatibility] Licence `accept_enterprise` and response changes {es-pull}75479[#75479] (issues: {es-issue}50067[#50067], {es-issue}50735[#50735], {es-issue}51816[#51816], {es-issue}58217[#58217]) Infra/Scripting:: * Update `DeprecationMap` to `DynamicMap` {es-pull}56149[#56149] (issue: {es-issue}52103[#52103]) * Add nio Buffers to Painless {es-pull}79870[#79870] (issue: {es-issue}79867[#79867]) * Restore the scripting general cache {es-pull}79453[#79453] (issue: {es-issue}62899[#62899]) Infra/Settings:: * Fixed inconsistent `Setting.exist()` {es-pull}46603[#46603] (issue: {es-issue}41830[#41830]) * Remove `index.optimize_auto_generated_id` setting (#27583) {es-pull}27600[#27600] (issue: {es-issue}27583[#27583]) * Implement setting deduplication via string interning {es-pull}80493[#80493] (issues: {es-issue}77466[#77466], {es-issue}78892[#78892]) Ingest:: * Add support for `_meta` field to ingest pipelines {es-pull}76381[#76381] * Remove binary field after attachment processor execution {es-pull}79172[#79172] * Improving cache lookup to reduce recomputing / searches {es-pull}77259[#77259] * Extract more standard metadata from binary files {es-pull}78754[#78754] (issue: {es-issue}22339[#22339]) License:: * Add deprecated `accept_enterprise` param to `/_xpack` {es-pull}58220[#58220] (issue: {es-issue}58217[#58217]) * Support `accept_enterprise` param in get license API {es-pull}50067[#50067] (issue: {es-issue}49474[#49474]) * Enforce Transport TLS check on all licenses {es-pull}79602[#79602] (issue: {es-issue}75292[#75292]) Machine Learning:: * The Windows build platform for the {ml} C++ code now uses Visual Studio 2019 {ml-pull}1352[#1352] * The macOS build platform for the {ml} C++ code is now Mojave running Xcode 11.3.1, or Ubuntu 20.04 running clang 8 for cross compilation {ml-pull}1429[#1429] * Add a new application for evaluating PyTorch models. The app depends on LibTorch - the C++ front end to PyTorch - and performs inference on models stored in the TorchScript format {ml-pull}1902[#1902] * Adding new PUT trained model vocabulary endpoint {es-pull}77387[#77387] * Creating new PUT model definition part API {es-pull}76987[#76987] * Add inference time configuration overrides {es-pull}78441[#78441] (issue: {es-issue}77799[#77799]) * Optimize source extraction for `categorize_text` aggregation {es-pull}79099[#79099] * The Linux build platform for the {ml} C++ code is now CentOS 7 running gcc 10.3. {ml-pull}2028[#2028] * Make ML indices hidden when the node becomes master {es-pull}77416[#77416] (issue: {es-issue}53674[#53674]) * Add `deployment_stats` to trained model stats {es-pull}80531[#80531] * The setting `use_auto_machine_memory_percent` now defaults `max_model_memory_limit` {es-pull}80532[#80532] (issue: {es-issue}80415[#80415]) Mapping:: * Sparse vector to throw exception consistently {es-pull}62646[#62646] * Add support for configuring HNSW parameters {es-pull}79193[#79193] (issue: {es-issue}78473[#78473]) * Extend `dense_vector` to support indexing vectors {es-pull}78491[#78491] (issue: {es-issue}78473[#78473]) Monitoring:: * Add previously removed Monitoring settings back for 8.0 {es-pull}78784[#78784] * Change Monitoring plugin cluster alerts to not install by default {es-pull}79657[#79657] * Adding default templates for Metricbeat ECS data {es-pull}81744[#81744] Network:: * Enable LZ4 transport compression by default {es-pull}76326[#76326] (issue: {es-issue}73497[#73497]) * Improve slow inbound handling to include response type {es-pull}80425[#80425] Packaging:: * Make the Docker build more re-usable in Cloud {es-pull}50277[#50277] (issues: {es-issue}46166[#46166], {es-issue}49926[#49926]) * Update docker-compose.yml to fix bootstrap check error {es-pull}47650[#47650] * Allow total memory to be overridden {es-pull}78750[#78750] (issue: {es-issue}65905[#65905]) * Convert repository plugins to modules {es-pull}81870[#81870] (issue: {es-issue}81652[#81652]) Recovery:: * Use Lucene index in peer recovery and resync {es-pull}51189[#51189] (issue: {es-issue}50775[#50775]) * Fix `PendingReplicationActions` submitting lots of `NOOP` tasks to `GENERIC` {es-pull}82092[#82092] (issues: {es-issue}77466[#77466], {es-issue}79837[#79837]) Reindex:: * Make reindexing managed by a persistent task {es-pull}43382[#43382] (issue: {es-issue}42612[#42612]) * Reindex restart from checkpoint {es-pull}46055[#46055] (issue: {es-issue}42612[#42612]) * Reindex search resiliency {es-pull}45497[#45497] (issues: {es-issue}42612[#42612], {es-issue}43187[#43187]) * Reindex v2 rethrottle sliced fix {es-pull}46967[#46967] (issues: {es-issue}42612[#42612], {es-issue}46763[#46763]) * Do not scroll if max docs is less than scroll size (update/delete by query) {es-pull}81654[#81654] (issue: {es-issue}54270[#54270]) Rollup:: * Adds support for `date_nanos` in Rollup Metric and `DateHistogram` Configs {es-pull}59349[#59349] (issue: {es-issue}44505[#44505]) SQL:: * Add text formatting support for multivalue {es-pull}68606[#68606] * Add xDBC and CLI support. QA CSV specs {es-pull}68966[#68966] * Export array values through result sets {es-pull}69512[#69512] * Improve alias resolution in sub-queries {es-pull}67216[#67216] (issue: {es-issue}56713[#56713]) * Improve the optimization of null conditionals {es-pull}71192[#71192] * Push `WHERE` clause inside subqueries {es-pull}71362[#71362] * Use Java `String` methods for `LTRIM/RTRIM` {es-pull}57594[#57594] * QL: Make canonical form take into account children {es-pull}71266[#71266] * QL: Polish optimizer expression rule declaration {es-pull}71396[#71396] * QL: Propagate nullability constraints across conjunctions {es-pull}71187[#71187] (issue: {es-issue}70683[#70683]) Search:: * Completely disallow setting negative size in search {es-pull}70209[#70209] (issue: {es-issue}69548[#69548]) * Make `0` as invalid value for `min_children` in `has_child` query {es-pull}41347[#41347] * Return error when remote indices are locally resolved {es-pull}74556[#74556] (issue: {es-issue}26247[#26247]) * [REST API Compatibility] Nested path and filter sort options {es-pull}76022[#76022] (issues: {es-issue}42809[#42809], {es-issue}51816[#51816]) * [REST API Compatibility] `CommonTermsQuery` and `cutoff_frequency` parameter {es-pull}75896[#75896] (issues: {es-issue}42654[#42654], {es-issue}51816[#51816]) * [REST API Compatibility] Allow first empty line for `_msearch` {es-pull}75886[#75886] (issues: {es-issue}41011[#41011], {es-issue}51816[#51816]) * Node level can match action {es-pull}78765[#78765] * TSDB: Add time series information to field caps {es-pull}78790[#78790] (issue: {es-issue}74660[#74660]) * Add new kNN search endpoint {es-pull}79013[#79013] (issue: {es-issue}78473[#78473]) * Disallow kNN searches on nested vector fields {es-pull}79403[#79403] (issue: {es-issue}78473[#78473]) * Ensure kNN search respects authorization {es-pull}79693[#79693] (issue: {es-issue}78473[#78473]) * Load kNN vectors format with mmapfs {es-pull}78724[#78724] (issue: {es-issue}78473[#78473]) * Support cosine similarity in kNN search {es-pull}79500[#79500] * Node level can match action {es-pull}78765[#78765] * Check nested fields earlier in kNN search {es-pull}80516[#80516] (issue: {es-issue}78473[#78473]) Security:: * Add a tool for creating enrollment tokens {es-pull}74890[#74890] * Add the Enroll Kibana API {es-pull}72207[#72207] * Change default hashing algorithm for FIPS 140 {es-pull}55544[#55544] * Create enrollment token {es-pull}73573[#73573] (issues: {es-issue}71438[#71438], {es-issue}72129[#72129]) * Enroll node API {es-pull}72129[#72129] * Configure security for the initial node CLI {es-pull}74868[#74868] * Generate and store password hash for elastic user {es-pull}76276[#76276] (issue: {es-issue}75310[#75310]) * Set elastic password and generate enrollment token {es-pull}75816[#75816] (issue: {es-issue}75310[#75310]) * Add `elasticsearch-enroll-node` tool {es-pull}77292[#77292] * Default hasher to `PBKDF2_STRETCH` on FIPS mode {es-pull}76274[#76274] * Add v7 `restCompat` for invalidating API key with the id field {es-pull}78664[#78664] (issue: {es-issue}66671[#66671]) * Print enrollment token on startup {es-pull}78293[#78293] * Startup check for security implicit behavior change {es-pull}76879[#76879] * CLI tool to reconfigure nodes to enroll {es-pull}79690[#79690] (issue: {es-issue}7718[#7718]) * Security auto-configuration for packaged installations {es-pull}75144[#75144] (issue: {es-issue}78306[#78306]) Snapshot/Restore:: * Introduce searchable snapshots index setting for cascade deletion of snapshots {es-pull}74977[#74977] * Unify blob store compress setting {es-pull}39346[#39346] (issue: {es-issue}39073[#39073]) * Add recovery state tracking for searchable snapshots {es-pull}60505[#60505] * Allow listing older repositories {es-pull}78244[#78244] * Optimize SLM Policy Queries {es-pull}79341[#79341] (issue: {es-issue}79321[#79321]) TLS:: * Add `ChaCha20` TLS ciphers on Java 12+ {es-pull}42155[#42155] * Add support for `KeyStore` filters to `ssl-config` {es-pull}75407[#75407] * Update TLS ciphers and protocols for JDK 11 {es-pull}41808[#41808] (issues: {es-issue}38646[#38646], {es-issue}41385[#41385]) Transform:: * Prevent old beta transforms from starting {es-pull}79712[#79712] TSDB:: * Automatically add timestamp mapper {es-pull}79136[#79136] * Create a coordinating node level reader for tsdb {es-pull}79197[#79197] * Fix TSDB shrink test in multi-version cluster {es-pull}79940[#79940] (issue: {es-issue}79936[#79936]) * Do not allow shadowing metrics or dimensions {es-pull}79757[#79757] [[bug-8.0.0]] [float] === Bug fixes Aggregations:: * Fix BWC issues for `x_pack/usage` {es-pull}55181[#55181] (issue: {es-issue}54847[#54847]) * Fix `DoubleBounds` null serialization {es-pull}59475[#59475] * Fix `TopHitsAggregationBuilder` adding duplicate `_score` sort clauses {es-pull}42179[#42179] (issue: {es-issue}42154[#42154]) * Fix `t_test` usage stats {es-pull}54753[#54753] (issue: {es-issue}54744[#54744]) * Throw exception if legacy interval cannot be parsed in `DateIntervalWrapper` {es-pull}41972[#41972] (issue: {es-issue}41970[#41970]) Autoscaling:: * Autoscaling use adjusted total memory {es-pull}80528[#80528] (issue: {es-issue}78750[#78750]) CCR:: * Fix `AutoFollow` version checks {es-pull}73776[#73776] (issue: {es-issue}72935[#72935]) Cluster Coordination:: * Apply cluster states in system context {es-pull}53785[#53785] (issue: {es-issue}53751[#53751]) Data streams:: * Prohibit restoring a data stream alias with a conflicting write data stream {es-pull}81217[#81217] (issue: {es-issue}80976[#80976]) Distributed:: * Introduce `?wait_for_active_shards=index-setting` {es-pull}67158[#67158] (issue: {es-issue}66419[#66419]) * Fixes to task result index mapping {es-pull}50359[#50359] (issue: {es-issue}50248[#50248]) Features/CAT APIs:: * Fix cat recovery display of bytes fields {es-pull}40379[#40379] (issue: {es-issue}40335[#40335]) Features/ILM+SLM:: * Ensuring that the `ShrinkAction` does not hang if total shards per node is too low {es-pull}76732[#76732] (issue: {es-issue}44070[#44070]) * Less verbose serialization of snapshot failure in SLM metadata {es-pull}80942[#80942] (issue: {es-issue}77466[#77466]) Features/Indices APIs:: * Fix `ComposableIndexTemplate` equals when `composed_of` is null {es-pull}80864[#80864] Features/Java High Level REST Client:: * The Java High Level Rest Client (HLRC) has been removed and replaced by a new {es} Java client. For migration steps, refer to {java-api-client}/migrate-hlrc.html[Migrate from the High Level Rest Client]. Geo:: * Preprocess polygon rings before processing it for decomposition {es-pull}59501[#59501] (issues: {es-issue}54441[#54441], {es-issue}59386[#59386]) Infra/Core:: * Add searchable snapshot cache folder to `NodeEnvironment` {es-pull}66297[#66297] (issue: {es-issue}65725[#65725]) * CLI tools: Write errors to stderr instead of stdout {es-pull}45586[#45586] (issue: {es-issue}43260[#43260]) * Precompute `ParsedMediaType` for XContentType {es-pull}67409[#67409] * Prevent stack overflow in rounding {es-pull}80450[#80450] Infra/Logging:: * Fix NPE when logging null values in JSON {es-pull}53715[#53715] (issue: {es-issue}46702[#46702]) * Fix stats in slow logs to be a escaped JSON {es-pull}44642[#44642] * Populate data stream fields when `xOpaqueId` not provided {es-pull}62156[#62156] Infra/REST API:: * Do not allow spaces within `MediaType's` parameters {es-pull}64650[#64650] (issue: {es-issue}51816[#51816]) * Handle incorrect header values {es-pull}64708[#64708] (issues: {es-issue}51816[#51816], {es-issue}64689[#64689]) * Ignore media ranges when parsing {es-pull}64721[#64721] (issues: {es-issue}51816[#51816], {es-issue}64689[#64689]) * `RestController` should not consume request content {es-pull}44902[#44902] (issue: {es-issue}37504[#37504]) * Handle exceptions thrown from `RestCompatibleVersionHelper` {es-pull}80253[#80253] (issues: {es-issue}78214[#78214], {es-issue}79060[#79060]) Infra/Scripting:: * Change compound assignment structure to support string concatenation {es-pull}61825[#61825] * Fixes casting in constant folding {es-pull}61508[#61508] * Several minor Painless fixes {es-pull}61594[#61594] * Fix duplicated allow lists upon script engine creation {es-pull}82820[#82820] (issue: {es-issue}82778[#82778]) Infra/Settings:: * Stricter `UpdateSettingsRequest` parsing on the REST layer {es-pull}79227[#79227] (issue: {es-issue}29268[#29268]) * Set Auto expand replica on deprecation log data stream {es-pull}79226[#79226] (issue: {es-issue}78991[#78991]) Ingest:: * Adjust default geoip logging to be less verbose {es-pull}81404[#81404] (issue: {es-issue}81356[#81356]) Machine Learning:: * Add timeout parameter for delete trained models API {es-pull}79739[#79739] (issue: {es-issue}77070[#77070]) * Tone down ML unassigned job notifications {es-pull}79578[#79578] (issue: {es-issue}79270[#79270]) * Use a new annotations index for future annotations {es-pull}79006[#79006] (issue: {es-issue}78439[#78439]) * Set model state compatibility version to 8.0.0 {ml-pull}2139[#2139] * Check that `total_definition_length` is consistent before starting a deployment {es-pull}80553[#80553] * Fail inference processor more consistently on certain error types {es-pull}81475[#81475] * Optimize the job stats call to do fewer searches {es-pull}82362[#82362] (issue: {es-issue}82255[#82255]) Mapping:: * Remove assertions that mappings have one top-level key {es-pull}58779[#58779] (issue: {es-issue}58521[#58521]) Packaging:: * Suppress illegal access in plugin install {es-pull}41620[#41620] (issue: {es-issue}41478[#41478]) Recovery:: * Make shard started response handling only return after the cluster state update completes {es-pull}82790[#82790] (issue: {es-issue}81628[#81628]) SQL:: * Introduce dedicated node for `HAVING` declaration {es-pull}71279[#71279] (issue: {es-issue}69758[#69758]) * Make `RestSqlQueryAction` thread-safe {es-pull}69901[#69901] Search:: * Check for negative `from` values in search request body {es-pull}54953[#54953] (issue: {es-issue}54897[#54897]) * Fix `VectorsFeatureSetUsage` serialization in BWC mode {es-pull}55399[#55399] (issue: {es-issue}55378[#55378]) * Handle total hits equal to `track_total_hits` {es-pull}37907[#37907] (issue: {es-issue}37897[#37897]) * Improve error msg for CCS request on node without remote cluster role {es-pull}60351[#60351] (issue: {es-issue}59683[#59683]) * Remove unsafe assertion in wildcard field {es-pull}78966[#78966] Security:: * Allow access to restricted system indices for reserved system roles {es-pull}76845[#76845] Snapshot/Restore:: * Fix `GET /_snapshot/_all/_all` if there are no repos {es-pull}43558[#43558] (issue: {es-issue}43547[#43547]) * Don't fill stack traces in `SnapshotShardFailure` {es-pull}80009[#80009] (issue: {es-issue}79718[#79718]) * Remove custom metadata if there is nothing to restore {es-pull}81373[#81373] (issues: {es-issue}81247[#81247], {es-issue}82019[#82019]) [[regression-8.0.0]] [float] === Regressions Search:: * Disable numeric sort optimization conditionally {es-pull}78103[#78103] [[upgrade-8.0.0]] [float] === Upgrades Authentication:: * Upgrade to UnboundID LDAP SDK v6.0.2 {es-pull}79332[#79332] Infra/Logging:: * Upgrade ECS logging layout to latest version {es-pull}80500[#80500] Search:: * Upgrade to Lucene 9 {es-pull}81426[#81426] Security:: * Update to OpenSAML 4 {es-pull}77012[#77012] (issue: {es-issue}71983[#71983]) Snapshot/Restore:: * Upgrade repository-hdfs plugin to Hadoop 3 {es-pull}76897[#76897]