This change leverages ordinals in enrich lookup. Instead of looking up
and extracting enrich fields for all input terms, this improvement only
looks up and extracts the dictionary, then applies the ordinals to the
enrich results.
```
| 50th percentile | esql_stats_enrich_rates_fares | 242.949 | 34.7007 | -208.248 | ms | -85.72% |
| 90th percentile | esql_stats_enrich_rates_fares | 245.479 | 36.3419 | -209.137 | ms | -85.20% |
|100th percentile | esql_stats_enrich_rates_fares | 252.877 | 49.0826 | -203.795 | ms | -80.59% |
```
* Enable corresponding validation in EsqlQueryRequest.
* Add the ESQL version to requests to /_query in integration tests.
* In mixed cluster tests for versions prior to 8.13.3, impersonate an 8.13
client and do not send any version.
---------
Co-authored-by: Nik Everett <nik9000@gmail.com>
Publish failures are - to a certain degree - expected as the master
might change before the corresponding task is executed.
This reduces logging to INFO level for publish failures that will be
retried by TransportMasterNodeAction.
Continuing my work from #104490, this PR moves the parameter compatibility checking for Equals into the type resolution check. This is a somewhat bigger change than for Add, as there was no ES|QL base class for binary comparison operators before this. I've added EsqlBinaryComparison as that base class, and migrated all of the binary comparisons to be based off of that (except for NullEquals, see note below).
In order to maintain compatibility with the current behavior, I've kept it so that unsigned longs are only inter-operable with other unsigned longs. We've talked a lot about changing that, and I consider this work a prerequisite for that.
I've also added a bunch of test cases to Equals and NotEquals, which should have the side effect of filling out the type support table in the equals docs. As noted in the comments, I'll have follow up PRs for the other binary comparisons to add tests, but this PR is already too long.
Note about NullEquals: There is an ES|QL NullEquals class, which inherits from the QL version, but I don't think it works. I didn't see any tests or docs for it, and trying it out in the demo instance gave me a syntax error. I think we need to delve into what's going on there, but this PR isn't the right place for it.
The method for `RoleDescriptor` parsing is becoming more complex due to
its usage being shared between API keys, file-based and native roles.
In some cases we allow 2.X format, in others we disallow
restrictions. Having a single method with multiple boolean flags that
control inclusion/exclusion of fields is becoming hard to extend.
This refactoring aims to allow easier introduction of new fields that
should be conditionally supported in some cases but not others.
One of such cases is introduction of `description` field that should only
be supported for file-based and native roles but not for roles embedded
in API keys.
Relates to: #107088
RestoreService will validate the snapshot based on max index version.
However, we do not increment index version on every single release.
To prevent attempting to restore an index of a future release, this change
rejects the restore already earlier when building the snapshot info from
a newer node matching the current index version.
Removes changes for code already removed from the `semantic_text`
feature branch.
This is prep work for opening up `semantic_text` PRs in order to remove
irrelevant changes.
This extends `BUCKET` function to accept a two-parameters-only
invocation: the first parameter remains as is, while the second is a
span. It can be a numeric (floating point) span, if the first argument
is numeric, or a date period or time duration, if the first argument is
a date.
Also, the function can now be invoked with the alias BIN.
Additionally, the function has been turned into a grouping-only function
and thus can only be used within a `STATS` command.
Today if `InternalTestCluster#getInstance` fails to find a matching
instance it throws an opaque `AssertionError`. This commit adds a
message describing the problem to make troubleshooting easier.
Relates #107392
This PR updates the privilege model to require `manage_security` cluster
privilege to invalidate cross cluster API keys, to better match the
access requirements of the creation and update APIs. Requests made with
lower privileges will receive descriptive errors in the response payload
indicating failure to invalidate, for each cross cluster API key. There
are no changes to invalidating REST API keys, nor to the Query or Get
APIs.
It's possible that the `executor` passed in to a `SubscribableListener`
rejects the task it should execute. This commit adds to the Javadocs a
description of what happens if so.
This adds a "tests.rest.project.id" system property that causes the
default test rest client to pass the "X-Elastic-Project-Id" header in
HTTP requests
Sometimes ESQL's Div test generates random division that'll produce an
out of range number. They should produce a warning and a `null`, but the
tests still thought they should make `Infinity`. This flips the tests
around to expecting the right thing in that case.
Closes#107475
The source index of a shrink action is made read-only to perform the shrink. During shrink, settings are copied from source index to target index, causing the target index to also be read-only. This change adds a shrink policy argument making the target index writable after shrinking. The default is to keep the read-only setting to preserve current behavior.
closes#106599
This PR adds counters for downsampling: success, failure and failure due
to invalid configuration. They will be used in TSDB dashboard to assess
health of downsampling functionality.
Soon we will require an ESQL version for all ESQL requests. Kibana is
sending it already. The official clients will start sending it in
version 8.14+. This defaults the version for the official clients before
8.14. It does so by reading a magic header the clients send,
`x-elastic-client-meta` and detecting the version string for any clients
that might possibly support ESQL - `es=8.11`, `es=8.12`, and `es=8.13`.
If we receive that we'll default to the first version of ESQL.
I've also made this work for kibana versions 8.12 and 8.13 - discover
will default to the old version of ESQL.
Transforms continue to run even when a node is shutting down. This may
lead to a transform failing and putting itself into a failed state,
which will prevent it from restarting when the node comes back online.
The transform will now abort rather than fail, which puts itself into a
started state. When the node comes back online, or another node in the
cluster starts the transform, then the transform will pick up from its
last successful saved state and checkpoint.
Close#100891
This change introduces a specialized exception RequestedRangeNotSatisfiedException
that is thrown when FsBlobContainer or S3BlobContainer try to read a range of bytes
from a position that is located after the real length of the blob.
This exception can then be caught to detect such situation and acts accordingly.
Wit this commit we add CO2 emission and cost information on the
top-level of the TopN functions API response. This is needed by the UI
to show summary info.
