- Add PREVIEW annotations to all preview functions
- Update docs generation logic to use annotations instead of preview boolean
* Changed stack: ga 9.1 to stack: coming in multiple places
- Match.java: Updated the options parameter description
- MultiMatch.java: Updated the options parameter description
- ToLower.java: Reformatted parameter description and updated version annotation
- ToUpper.java: Removed the appliesTo annotation entirely and reformatted parameter description
- updated applies_to annotations to specify both preview 9.0.0 and ga 9.1.0 lifecycle stages
- added version-specific documentation examples with applies_to markers for ga 9.1.0 features
- enhanced to_lower and to_upper functions to indicate support for multi-valued expressions in ga 9.1.0
- added version guards around function parameters and descriptions using applies_to syntax
- updated function parameter descriptions to indicate ga 9.1.0 availability for named parameters
- use detailedDescription + and fix match_phrase applies_to syntax
- strip inline applies_to from kibana docs
- update roundto, matchphrase lifecycles
- fix match named params info
- various spatial functions are preview
- unsigned long is preview
- update qstr
- Update TO_LOWER/TO_UPPER parameter descriptions for clarity
- hide spatial functions per https://github.com/elastic/elasticsearch/pull/129839
- added `stack: ga 9.1.0` blocks to match_phrase.md and qstr.md examples
- simplified term.md version from `preview 9.0.0` to just `preview`
- added `applies_to = "stack: ga 9.1.0"` to matchphrase and querystring java annotations
- removed version `9.0.0` from term function annotation
- deleted unused `makeCallout()` and `appendLifeCycleAndVersion()` methods
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Liam Thompson <leemthompo@gmail.com>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
Co-authored-by: Nik Everett <nik9000@gmail.com>
We add a margin of error when comparing floats to the DynamicFieldMatcher to account for a small loss of accuracy when loading fields from synthetic source.
The vast majority of ingest pipelines are light CPU
operations. We don't want these to be put behind IO work on the write
executor. Instead, execute these on the coordination pool.
Occasional shard allocation issues were causing the YAML tests to fail
because the shard that had the document with the max timestamp in it
would be unavailable.
Fixes#118217
#125143 added 9 spatial grid functions and released them into Serverless. We think this is not the best long-term approach and the functions in #129581 are likely better.
As a first step, rmove the spatial grid functions added in #125143 from release builds so they don't get released into 8.19/9.1.
---------
Co-authored-by: Craig Taverner <craig@amanzi.com>
* Using the STREAMS_LOGS_SUPPORT_8_19 transport version
* Update StreamsMetadata.java
Returning null from getMinimalSupportedVersion
* Return minimal supported version as 8.19 for metadata object to fix test fail
---------
Co-authored-by: Luke Whiting <luke.whiting@elastic.co>
Add a new logical plan optimization:
When there is a Project (KEEP/DROP/RENAME/renaming EVALs) in a LOOKUP JOIN's left child (the "main" side), perform the Project after the LOOKUP JOIN. This prevents premature field extractions when the lookup join happens on data nodes.
A followup to #128440, which introduces a new `managed_by` field (`<1>`) that will be returned in the response of the Authenticate API.
Besides `managed_by` field, it also captures additional `internal` field (`<2>`) for cloud API key authentication and exposes it as part of the `api_key` fields.
```json
{
"username": "omSAd5YBK3gZiBcD-GvX",
"roles": [ "viewer" ],
"metadata": {
...
},
"enabled": true,
"authentication_realm": {
"name": "_cloud_api_key",
"type": "_cloud_api_key"
},
"lookup_realm": {
"name": "_cloud_api_key",
"type": "_cloud_api_key"
},
"authentication_type": "api_key",
"api_key": {
"id": "omSAd5YBK3gZiBcD-GvX",
"name": "my cloud API key",
"managed_by": "cloud", <1>
"internal": false <2>
}
}
```
- Additionally it implements the `Authentication#canAccessResourcesOf` for the cloud API keys. Ownership check allows access only to the same cloud API key.
- And lastly, adds a consistency check for cloud API keys in `Authentication#checkConsistencyForApiKeyAuthenticationType`.
This commit ports the IndexVersions.UPGRADE_TO_LUCENE_9_12_2 constant to the main branch.
This is required after the update of Lucene 9.12.2 in the 8.19 branch, see #129555.
This change makes the GeoIp persistent task executor/downloader multi-project aware.
- the database downloader persistent task will be at the project level, meaning there will be a downloader instance per project
- persistent task id is prefixed with project id, namely `<project-id>/geoip-downloader` for cluster in MP mode
Due to the way how stored fields get flushed when index sorting is active, it is possible that we encounter significant page cache faults when memory is scarce. In order to mitigate some of the slowness around this, we're planning to no longer mmap the fdt temp file. Initially behind a feature flag, to check for unforeseen side effects.
Typically using always mmap directory is better compared to noifs directory given there is a sufficient memory available to the OS for filesystem caching. However when that isn't the case, then indexing performance can vary a lot (often very slow). This is more true for files tmp files that stored fields create during flushing. These files exist for only a brief moment to sort stored fields in the order of the configured index sorting and are then removed. If these tmp files are mmapped there is risk to trash file system cache.
This change only avoids using mmap for the fdt tmp file. This the file that actually contains the data and can large compared to other files that get flushed. The fdm (metadata) and fdi (stored field index) remain being mmapped.
With this change we will create first the tmp file and the posting list and once the file is deleted we will
merge the vectors on the vec file. Therefore we only have two copies of the vector at the same time.
TransportStartDatafeedAction previously tried to validate remote index cluster
names in datafeed jobs, before checking if the local cluster had
remote_cluster_client role. Because this role enables retrieval of the remote
cluster names, the validation step would always fail with a no-such-cluster
exception. This was confusing. This change moves the remote_cluster_client check
ahead of cluster name validation, and adds a test.
Closes ES-11841
Closes#121149
There is an issue where for Flattened fields with synthetic source, if there is a key with a scalar value, and a duplicate key with an object value, one of the values will be left out of the produced synthetic source. This fixes the issue by replacing the object with paths to each of its keys. These paths consist of the concatenation of all keys going down to a given scalar, joined by a period. For example, they are of the form foo.bar.baz. This applies recursively, so that every value within the object, no matter how nested, will be accessible through a full specified path.
* Making progress on different request parameters
* Working tests
* Adding custom service validator for rerank
* Fixing embedding bug
* Adding transport version check
* Fixing tests
* Fixing license header
* Fixing writeTo
* Moving file and removing commented code
* Fixing test
* Fixing tests
* Refactoring and tests
* Fixing test
This test has been muted for a long time and only seemed to fail on v7
compatability tests. We're unmuting this to see if it's still relevant.
Closes#112189
