[[release-notes-8.3.0]] == {es} version 8.3.0 coming[8.3.0] Also see <>. [[bug-8.3.0]] [float] === Bug fixes Aggregations:: * Allow `serial_diff` under `min_doc_count` aggs {es-pull}86401[#86401] * Allow bucket paths to specify `_count` within a bucket {es-pull}85720[#85720] * Fix a bug with flattened fields in terms aggregations {es-pull}87392[#87392] * Fix flaky `top_metrics` test {es-pull}86582[#86582] (issue: {es-issue}86377[#86377]) * Fix: check field existence before trying to merge running stats {es-pull}86926[#86926] * Fix: ordering terms aggregation on top metrics null values {es-pull}85774[#85774] * Serialize interval in auto date histogram aggregation {es-pull}85473[#85473] Audit:: * Fix audit logging to consistently include port number in `origin.address` {es-pull}86732[#86732] * Support removing ignore filters for audit logging {es-pull}87675[#87675] (issue: {es-issue}68588[#68588]) Authentication:: * An authorized user can disable a user with same name but different realm {es-pull}86473[#86473] * Fix clearing of `lastSuccessfulAuthCache` when clear all realm cache API is called {es-pull}86909[#86909] (issue: {es-issue}86650[#86650]) Authorization:: * Fix resolution of wildcard application privileges {es-pull}87293[#87293] CAT APIs:: * Get hidden indices stats in `GET _cat/shards` {es-pull}86601[#86601] (issue: {es-issue}84656[#84656]) CCR:: * Prevent invalid datastream metadata when CCR follows a datastream with closed indices on the follower {es-pull}87076[#87076] (issue: {es-issue}87048[#87048]) * Remove some blocking in CcrRepository {es-pull}87235[#87235] Cluster Coordination:: * Add `master_timeout` support to voting config exclusions APIs {es-pull}86670[#86670] * Small fixes to clear voting config excls API {es-pull}87828[#87828] Discovery-Plugins:: * [discovery-gce] Fix initialisation of transport in FIPS mode {es-pull}85817[#85817] (issue: {es-issue}85803[#85803]) Distributed:: * Enforce external id uniqueness during `DesiredNode` construction {es-pull}84227[#84227] Engine:: * Fork to WRITE thread before failing shard in `updateCheckPoints` {es-pull}87458[#87458] (issue: {es-issue}87094[#87094]) * Removing Blocking Wait for Close in `RecoverySourceHandler` {es-pull}86127[#86127] (issue: {es-issue}85839[#85839]) Features:: * Fix 'autoGeneratedTimestamp should not be set externally' error when retrying IndexRequest {es-pull}86184[#86184] (issue: {es-issue}83927[#83927]) Geo:: * Fix Geotile aggregations on `geo_shapes` for precision 0 {es-pull}87202[#87202] (issue: {es-issue}87201[#87201]) * Fix `null_value` for array-valued `geo_point` fields {es-pull}85959[#85959] * Guard for adding null value tags to vector tiles {es-pull}87051[#87051] * Quantize geo queries to remove true negatives from search results {es-pull}85441[#85441] (issue: {es-issue}40891[#40891]) Highlighting:: * `FastVectorHighlighter` should use `ValueFetchers` to load source data {es-pull}85815[#85815] (issues: {es-issue}75011[#75011], {es-issue}84690[#84690], {es-issue}82458[#82458], {es-issue}80895[#80895]) ILM+SLM:: * Make the ILM Move to Error Step Batched {es-pull}85565[#85565] (issue: {es-issue}81880[#81880]) Indices APIs:: * Make `GetIndexAction` cancellable {es-pull}87681[#87681] Infra/Circuit Breakers:: * Make CBE message creation more robust {es-pull}87881[#87881] Infra/Core:: * Adjust osprobe assertion for burst cpu {es-pull}86990[#86990] * Clean up `DeflateCompressor` after exception {es-pull}87163[#87163] (issue: {es-issue}87160[#87160]) * Error on direct creation of non-primary system index {es-pull}86707[#86707] * Fix null message in output {es-pull}86981[#86981] * Fix using `FilterOutputStream` without overriding bulk write {es-pull}86304[#86304] * Hide system indices and their aliases in upgraded clusters {es-pull}87125[#87125] * Refactor code to avoid JDK bug: JDK-8285835 {es-pull}86614[#86614] Infra/Logging:: * Temporarily provide `SystemPropertiesPropertySource` {es-pull}87149[#87149] Infra/Node Lifecycle:: * Upgrade folders after settings validation {es-pull}87319[#87319] Infra/Plugins:: * Use Windows newlines when listing plugin information on Windows {es-pull}86408[#86408] (issue: {es-issue}86352[#86352]) Infra/REST API:: * Fix min node version before state recovery {es-pull}86482[#86482] Infra/Scripting:: * Allow to sort by script value using `SemVer` semantics {es-pull}85990[#85990] (issues: {es-issue}85989[#85989], {es-issue}82287[#82287]) * Script: Fix setter shortcut for unbridged setters {es-pull}86868[#86868] * Script: Load Whitelists as Resource {es-pull}87539[#87539] Infra/Settings:: * Permit removal of archived index settings {es-pull}86107[#86107] Ingest:: * Execute self-reference checks once per pipeline {es-pull}85926[#85926] (issue: {es-issue}85790[#85790]) Java Low Level REST Client:: * Do not retry client requests when failing with `ContentTooLargeException` {es-pull}87248[#87248] (issue: {es-issue}86041[#86041]) License:: * Consistent response for starting basic license {es-pull}86272[#86272] (issue: {es-issue}86244[#86244]) Machine Learning:: * Fix ML task auditor exception early in cluster lifecycle {es-pull}87023[#87023] (issue: {es-issue}87002[#87002]) * Fix `WordPiece` tokenization of unknown words with known subwords {es-pull}87510[#87510] * Fix distribution change check for `change_point` aggregation {es-pull}86423[#86423] * Fixes inference timeout handling bug that throws unexpected `NullPointerException` {es-pull}87533[#87533] * Correct logic for restart from failover fine tuning hyperparameters for training classification and regression models {ml-pull}2251[#2251] * Fix possible source of "x = NaN, distribution = class boost::math::normal_distribution<..." log errors training classification and regression models {ml-pull}2249[#2249] * Fix some bugs affecting decision to stop optimizing hyperparameters for training classification and regression models {ml-pull}2259[#2259] * Fix cause of "Must provide points at which to evaluate function" log error training classification and regression models {ml-pull}2268[#2268] * Fix a source of "Discarding sample = nan, weights = ..." log errors for time series anomaly detection {ml-pull}2286[#2286] Mapping:: * Don't run `include_in_parent` when in `copy_to` context {es-pull}87123[#87123] (issue: {es-issue}87036[#87036]) Network:: * Reject `openConnection` attempt while closing {es-pull}86315[#86315] (issue: {es-issue}86249[#86249]) Recovery:: * Fail shard if STARTED after master failover {es-pull}87451[#87451] (issue: {es-issue}87367[#87367]) SQL:: * Fix FORMAT function to comply with Microsoft SQL Server specification {es-pull}86225[#86225] (issue: {es-issue}66560[#66560]) * Implement binary format support for SQL clear cursor {es-pull}84230[#84230] (issue: {es-issue}53359[#53359]) Search:: * Add status field to Multi Search Template Responses {es-pull}85496[#85496] (issue: {es-issue}83029[#83029]) * Fields API to allow fetching values when `_source` is disabled {es-pull}87267[#87267] (issue: {es-issue}87072[#87072]) * Fix `_terms_enum` on unconfigured `constant_keyword` {es-pull}86191[#86191] (issues: {es-issue}86187[#86187], {es-issue}86267[#86267]) * Fix status code when open point in time without `keep_alive` {es-pull}87011[#87011] (issue: {es-issue}87003[#87003]) * Handle empty point values in `DiskUsage` API {es-pull}87826[#87826] (issue: {es-issue}87761[#87761]) * Make sure to rewrite explain query on coordinator {es-pull}87013[#87013] (issue: {es-issue}64281[#64281]) Security:: * Make user and role name constraint consistent with max document ID {es-pull}86728[#86728] (issue: {es-issue}66020[#66020]) * Security plugin close releasable realms {es-pull}87429[#87429] (issue: {es-issue}86286[#86286]) Snapshot/Restore:: * DONE should mean fully processed in snapshot status {es-pull}86414[#86414] * Distinguish missing and invalid repositories {es-pull}85551[#85551] (issue: {es-issue}85550[#85550]) * Fork after calling `getRepositoryData` from `StoreRecovery` {es-pull}87264[#87264] (issue: {es-issue}87237[#87237]) * Fork after calling `getRepositoryData` from `StoreRecovery` {es-pull}87254[#87254] (issue: {es-issue}87237[#87237]) * Throw exception on illegal `RepositoryData` updates {es-pull}87654[#87654] * Upgrade Azure SDK to 12.16.0 {es-pull}86135[#86135] Stats:: * Run `TransportClusterInfoActions` on MANAGEMENT pool {es-pull}87679[#87679] TSDB:: * TSDB: fix the time_series in order collect priority {es-pull}85526[#85526] * TSDB: fix wrong initial value of tsidOrd in TimeSeriesIndexSearcher {es-pull}85713[#85713] (issue: {es-issue}85711[#85711]) Transform:: * Fix transform `_start` permissions to use stored headers in the config {es-pull}86802[#86802] * [Transforms] fix bug when unsetting retention policy {es-pull}87711[#87711] [[deprecation-8.3.0]] [float] === Deprecations Authentication:: * Configuring a bind DN in an LDAP or Active Directory (AD) realm without a corresponding bind password is deprecated {es-pull}85326[#85326] (issue: {es-issue}47191[#47191]) [[enhancement-8.3.0]] [float] === Enhancements Aggregations:: * Improve min and max performance while in a `random_sampler` aggregation {es-pull}85118[#85118] Authentication:: * Support configurable claims in JWT Realm Tokens {es-pull}86533[#86533] * Warn on user roles disabled due to licensing requirements for document or field level security {es-pull}85393[#85393] (issue: {es-issue}79207[#79207]) * `TokenService` decode JWTs, change warn to debug {es-pull}86498[#86498] Authorization:: * Add delete privilege to `kibana_system` for Synthetics {es-pull}85844[#85844] * Authorize painless execute as index action when an index is specified {es-pull}85512[#85512] (issue: {es-issue}86428[#86428]) * Better error message for run-as denials {es-pull}85501[#85501] (issue: {es-issue}72904[#72904]) * Improve "Has Privilege" performance for boolean-only response {es-pull}86685[#86685] * Relax restrictions for role names in roles API {es-pull}86604[#86604] (issue: {es-issue}86480[#86480]) * [Osquery] Extend `kibana_system` role with an access to osquery_manager… {es-pull}86609[#86609] Autoscaling:: * Add support for CPU ranges in desired nodes {es-pull}86434[#86434] Cluster Coordination:: * Block joins while applier is busy {es-pull}84919[#84919] * Compute master task batch summary lazily {es-pull}86210[#86210] * Log `cluster.initial_master_nodes` at startup {es-pull}86101[#86101] * Reduce resource needs of join validation {es-pull}85380[#85380] (issue: {es-issue}83204[#83204]) * Report pending joins in `ClusterFormationFailureHelper` {es-pull}85635[#85635] * Speed up map diffing (2) {es-pull}86375[#86375] Discovery-Plugins:: * Remove redundant jackson dependencies from discovery-azure {es-pull}87898[#87898] Distributed:: * Keep track of desired nodes cluster membership {es-pull}84165[#84165] Engine:: * Cache immutable translog lastModifiedTime {es-pull}82721[#82721] (issue: {es-issue}82720[#82720]) * Increase `force_merge` threadpool size based on the allocated processors {es-pull}87082[#87082] (issue: {es-issue}84943[#84943]) * More optimal forced merges when max_num_segments is greater than 1 {es-pull}85065[#85065] Geo:: * Support 'GeoJSON' in CartesianPoint for 'point' {es-pull}85442[#85442] * Support geo label position as runtime field {es-pull}86154[#86154] * Support geo label position through REST vector tiles API {es-pull}86458[#86458] (issue: {es-issue}86044[#86044]) Health:: * Add a basic check for tier preference and allocation filter clashing {es-pull}85071[#85071] * Add preflight checks to Health API to ensure health is obtainable {es-pull}86404[#86404] * Add tier information on health api migrate tiers user actions {es-pull}87486[#87486] * Health api add indicator doc links {es-pull}86904[#86904] (issue: {es-issue}86892[#86892]) * Health api copy editing {es-pull}87010[#87010] * Return a default user action if no actions could be determined {es-pull}87079[#87079] ILM+SLM:: * Make the ILM and SLM `history_index_enabled` settings dynamic {es-pull}86493[#86493] Indices APIs:: * Batch execute template and pipeline cluster state operations {es-pull}86017[#86017] Infra/Core:: * Add mapping for tags for the elastic agent {es-pull}86298[#86298] * Expand jar hell to include modules {es-pull}86622[#86622] * Faster GET _cluster/settings API {es-pull}86405[#86405] (issue: {es-issue}82342[#82342]) * Faster string writes by saving stream flushes {es-pull}86114[#86114] * Fleet: Add `start_time` and `minimum_execution_duration` attributes to actions {es-pull}86167[#86167] * Force property expansion for security policy {es-pull}87396[#87396] * Refactor array part into a `BytesRefArray` which can be serialized and … {es-pull}85826[#85826] * Speed up ip v4 parser {es-pull}86253[#86253] * Use varhandles for primitive type conversion in more places {es-pull}85577[#85577] (issue: {es-issue}78823[#78823]) Infra/Scripting:: * Script: add ability to alias classes in whitelist {es-pull}86899[#86899] Ingest:: * Iteratively execute synchronous ingest processors {es-pull}84250[#84250] (issue: {es-issue}84274[#84274]) * Skip `ensureNoSelfReferences` check in `IngestService` {es-pull}87337[#87337] License:: * Initialize active realms without logging a message {es-pull}86134[#86134] (issue: {es-issue}81380[#81380]) Machine Learning:: * A text categorization aggregation that works like ML categorization {es-pull}80867[#80867] * Add new _infer endpoint for all supervised models and deprecate deployment infer api {es-pull}86361[#86361] * Adds new `question_answering` NLP task for extracting answers to questions from a document {es-pull}85958[#85958] * Adds start and end params to `_preview` and excludes cold/frozen tiers from unbounded previews {es-pull}86989[#86989] * Adjust automatic JVM heap sizing for dedicated ML nodes {es-pull}86399[#86399] * Replace the implementation of the `categorize_text` aggregation {es-pull}85872[#85872] * Upgrade PyTorch to version 1.11 {ml-pull}2233[#2233], {ml-pull}2235[#2235],{ml-pull}2238[#2238] * Upgrade zlib to version 1.2.12 on Windows {ml-pull}2253[#2253] * Upgrade libxml2 to version 2.9.14 on Linux and Windows {ml-pull}2287[#2287] * Improve time series model stability and anomaly scoring consistency for data for which many buckets are empty {ml-pull}2267[#2267] * Address root cause for actual equals typical equals zero anomalies {ml-pull}2270[#2270] * Better handling of outliers in update immediately after detecting changes in time series {ml-pull}2280[#2280] Mapping:: * Intern field names in Mappers {es-pull}86301[#86301] * Replace BYTE_BLOCK_SIZE - 2 with indexWriter#MAX_TERM_LENGTH {es-pull}85518[#85518] Network:: * Log node identity at startup {es-pull}85773[#85773] Search:: * GeoBoundingBox query should work on bounding box with equal latitude or longitude {es-pull}85788[#85788] (issue: {es-issue}77717[#77717]) * Improve error message for search API url parameters {es-pull}86984[#86984] (issue: {es-issue}79719[#79719]) Security:: * Add run-as support for OAuth2 tokens {es-pull}86680[#86680] * Relax username restrictions for User APIs {es-pull}86398[#86398] (issue: {es-issue}86326[#86326]) * User Profile - Add hint support to SuggestProfiles API {es-pull}85890[#85890] * User Profile - Add new action origin and internal user {es-pull}86026[#86026] * User Profile - Support request cancellation on HTTP disconnect {es-pull}86332[#86332] * User Profile - add caching for `hasPrivileges` check {es-pull}86543[#86543] Snapshot/Restore:: * Add parameter to exclude indices in a snapshot from response {es-pull}86269[#86269] (issue: {es-issue}82937[#82937]) Stats:: * Add documentation for "io_time_in_millis" {es-pull}84911[#84911] TLS:: * Set `serverAuth` extended key usage for generated certificates and CSRs {es-pull}86311[#86311] (issue: {es-issue}81067[#81067]) TSDB:: * Aggregation Execution Context add timestamp provider {es-pull}85850[#85850] Transform:: * Prefer secondary auth headers for transforms {es-pull}86757[#86757] * Support `range` aggregation in transform {es-pull}86501[#86501] [[feature-8.3.0]] [float] === New features Authorization:: * Has privileges API for profiles {es-pull}85898[#85898] Geo:: * New geo_grid query to be used with geogrid aggregations {es-pull}86596[#86596] (issue: {es-issue}85727[#85727]) Health:: * Add support for `impact_areas` to health impacts {es-pull}85830[#85830] (issue: {es-issue}85829[#85829]) * Add troubleshooting guides to shards allocation actions {es-pull}87078[#87078] * Adding potential impacts to remaining health indicators {es-pull}86197[#86197] * Health api drill down {es-pull}85234[#85234] (issue: {es-issue}84793[#84793]) * New service to keep track of the master history as seen from each node {es-pull}85941[#85941] * Sorting impact index names by index priority {es-pull}85347[#85347] Mapping:: * Add support for dots in field names for metrics usecases {es-pull}86166[#86166] (issue: {es-issue}63530[#63530]) * Synthetic source {es-pull}85649[#85649] SQL:: * SQ: Allow partial results in SQL queries {es-pull}85897[#85897] (issue: {es-issue}33148[#33148]) Search:: * Snapshots as simple archives {es-pull}86261[#86261] (issue: {es-issue}81210[#81210]) TSDB:: * TSDB: Implement downsampling on time-series indices {es-pull}85708[#85708] (issues: {es-issue}69799[#69799], {es-issue}65769[#65769]) [[upgrade-8.3.0]] [float] === Upgrades Infra/CLI:: * Upgrade procrun executables to 1.3.1 {es-pull}86710[#86710] Infra/Core:: * Upgrade jackson to 2.13.2 {es-pull}86051[#86051] Ingest:: * Upgrading to tika 2.4 {es-pull}86015[#86015] Network:: * Upgrade to Netty 4.1.76 {es-pull}86252[#86252] Packaging:: * Update Iron Bank base image to 8.6 {es-pull}86796[#86796] SQL:: * Update dependency - JLine - to v 3.21.0 {es-pull}83767[#83767] (issue: {es-issue}83575[#83575]) Search:: * Update to public lucene 9.2.0 release {es-pull}87162[#87162] Snapshot/Restore:: * Upgrade GCS Plugin to 1.118.1 {es-pull}87800[#87800]