JDK 23 removes the COMPAT locale provider, leaving CLDR as the only option. This commit configures Elasticsearch
to use the CLDR provider when on JDK 23, but still use the existing COMPAT provider when on JDK 22 and below.
This causes some differences in locale behaviour; this also adapts various tests to still work whether run on COMPAT or CLDR.
This adds `hamming` distances, the pop-count of `xor` byte vectors as a
first class citizen in painless.
For byte vectors, this means that we can compute hamming distances via
script_score (aka, brute-force).
The implementation of `hamming` is the same that is available in Lucene,
and when lucene 9.11 is merged, we should update our logic where
applicable to utilize it.
NOTE: this does not yet add hamming distance as a metric for indexed
vectors. This will be a future PR after the Lucene 9.11 upgrade.
Hexidecimal strings are supported for index input and for kNN queries. We should support them for byte vectors in painless.
This commit addresses this for our common scoring functions.
closes: #109412
When security is enabled and a IndicesRequest.SingleIndexNoWildcards request uses a wildcard for
remote clusters that does not match (for example "*:myindex" when there are no configured remotes),
RBACEngine#authorizeIndexAction throws an assert error because it is given an empty list of indices.
This fix checks the IndicesRequest.SingleIndexNoWildcards special case in
IndicesAndAliasesResolver.resolveIndicesAndAliasesWithoutWildcards
and if splitLocalAndRemoteIndexNames filters out all indexes due to a non-matching remote cluster
wildcard, that code now just throws the NoSuchRemoteClusterException (which is what RBACEngine
does when asserts are disabled).
This handles painless/execute requests to remote clusters.
Added note to painless/execute API docs about wildcards not being allowed
Added a clusterAlias to the Painless execute Request object, so that index
expressions in the request of the form "myremote:myindex" will be parsed to
set clusterAlias to "myremote" and the index to "myindex".
If clusterAlias is null, then it is executed against a shard on the local cluster, as before.
If clusterAlias is non-null, then the SingleShardTransportAction is sent to the remote cluster,
where it will run the full request (doing remote coordination). Note that the new clusterAlias
field is not Writeable so that when it is sent to the remote cluster it will only see the index
name, not the clusterAlias (which it wouldn't know how to handle correctly).
Added PainlessExecuteIT test that tests cross-cluster calls
Updated painless-execute-script end user docs to indicate support for cross-cluster executions
This formats the result of the `fields` section of the `_search` API for
runtime `geo_point` fields using the `format` parameter like we do for
non-runtime `geo_point` fields. This changes the default format for
those fields from `lat, lon` to `geojson` with the option to get `wkt`
or any other format we support.
The fix does so by preserving the `double, double` nature of the
`geo_point` rather than encoding it immediately in the script. Callers can
use the results. The field fetchers use the `double, double` natively,
preserving as much precision as possible. The queries quantize the points
exactly like lucene indexing does. And like the script did before this Pr.
Closes#85245
The painless execute API supports the composite_field context since the composite field was added.
This commit adds docs for it where missing.
Relates to #78050
ctx['op'] should be set to 'noop', not 'none' when specifying no
operation.
Elasticsearch error when using 'none':
```json
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "Operation type [none] not allowed, only [noop, index, delete] are allowed"
}
],
"type" : "illegal_argument_exception",
"reason" : "Operation type [none] not allowed, only [noop, index, delete] are allowed"
},
"status" : 400
}
```
This change removes JodaCompatibleZonedDateTime and replaces it with ZonedDateTime for use in
scripting.
Breaking changes:
* JodaCompatibleDateTime no longer exists and cannot be cast to in Painless. Use ZonedDateTime
instead.
* The dayOfWeek method on ZonedDateTime returns the DayOfWeek enum instead of an int from
JodaCompatibleDateTime. dayOfWeekEnum still exists on ZonedDateTime as an augmentation to
support the transition to ZonedDateTime, but is now deprecated in favor of dayOfWeek on
ZonedDateTime.
* add runtime fields contexts to execute docs
* Changes for formatting throughout
* Add missing context and context_setup
* Updating runtime field context
* Moving parameters and adopting a more standard API layout
* Update several examples
* Update more examples for runtime context
* Fix links
* Add boolean_field example and remove extraneous headings
* Add example for date_time context
* Remove extra space in TEST
* Updating date_time example
* Incorporating review feedback
* Adding cross links
* Tweaking some language based on feedback
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Adam Locke <adam.locke@elastic.co>
This documents the different signatures of the `emit` method for runtime
fields. For fields like `long` the signature is fairly obvious -
`emit(long)`. But for `date`, `ip`, and `geo_point` its not obvious from
the name what the signature of the method will be.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
* Initial changes for scripting.
* Shorten script examples.
* Expanding types docs.
* Updating types.
* Fixing broken cross-link.
* Fixing map error.
* Incorporating review feedback.
* Fixing broken table.
* Adding more info about reference types.
* Fixing broken path.
* Adding more info an examples for def type.
* Adding more info on operators.
* Incorporating review feedback.
* Adding notconsole for example.
* Removing comments in example.
* More review feedback.
* Editorial changes.
* Incorporating more reviewer feedback.
* Rewrites based on review feedback.
* Adding new sections for storing scripts and shortening scripts.
* Adding redirect for stored scripts.
* Adding DELETE for stored script plus link.
* Adding section for updating docs with scripts.
* Incorporating final feedback from reviews.
* Tightening up a few areas.
* Minor change around other languages.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [DOCS] Adding grok support for runtime fields.
* Update response.
* Adding testresponse replacements.
* Update runtime field context and add dissect.
* Fixing backslash in the response.
* Fixing testresponse.
* Incorporating review feedback.
* Updates emit and adds cross link from ES runtime fields page.
* Fixing Painless tests.
* Update runtime field context to fix test cases.
* Remove watcher logging from usage API and replace test.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Adds datetime as a date, which is necessary in setup.
* Updating field context example.
* Fixing sample data, updating context example, and updating runtime example.
* Updating field context and changing runtime field to use seats data.
* Update filter context to use the seats data.
* Updating min-should-match context to use seats data.
* Replacing last mentions of TEST[skip].
* Update usage with watcher response for build error.
* Updating usage API again for watcher.
* Third time's a charm for fixing test cases.
* Adding specific test replacement for watcher logging total.
* Change actors to keyword based on review feedback.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Initial updates to the seats data.
* Enhance seats test in gradle.build.
* Updating bulk ingest example to use test data.
* Updating examples and context example intro.
* Adding runtime fields page for Painless context.
* Adds beta admonition to runtime fields and Painless docs.
* Fixing test errors and improving content sections.
* Adding refresh to fix test cases.
* Simplifying the ingest request to include refresh.
* Removing beta (will add in another PR) and updating examples.
* Moving examples to the page for retrieving runtime fields.
* Adding runtime_mappings to request body of search API.
* Updating runtime_mappings properties and adding runtime fields to search your data.
* Updating examples and hopefully fixing build failure.
* Fixing snippet formatting that was causing test failure.
* Adding page in Painless guide for runtime fields.
* Fixing typo.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Changes:
* Moves `Retrieve selected fields` to its own page and adds a title abbreviation.
* Adds existing script and stored fields content to `Retrieve selected fields`
* Adds a xref for `Retrieve selected fields` to `Search your data`
* Adds related redirects and updates existing xrefs
Plugin discovery documentation contained information about installing
Elasticsearch 2.0 and installing an oracle JDK, both of which is no
longer valid.
While noticing that the instructions used cleartext HTTP to install
packages, this commit replaces HTTPs links instead of HTTP where possible.
In addition a few community links have been removed, as they do not seem
to exist anymore.
Moves the search sort docs from the deprecated 'Request Body Search'
page to a new subpage of 'Run a search'.
No substantive changes were made to the content.
