* Use a task queue to ensure ILM policy change cluster state updates are batched
* Update docs/changelog/126529.yaml
* Update docs/changelog/126529.yaml
* Switch to using SimpleBatchedAckListenerTaskExecutor
* Get timeout from request
* Ditch the try-catch
Currently, the rate aggregation accepts two parameters: the first
specifies the counter field, and the second specifies the time unit of
the rate. The time unit parameter was introduced to conveniently compute
requests per minute or per hour. However, this can be replaced easily -
for example, rate(field, 1minute) with 60 * rate(field).
This change removes the time unit parameter and reserves it for
potential future usage, such as introducing a sliding window unit. If we
decide on other options later, we can reintroduce it. Removing it now
avoids breaking changes while the rate aggregation is not yet available.
This commit cleans up how entitlement test methods are discovered. It
also adds another robustness check to ensure an annotation doesn't exist
on a private method.
In the unexpected case that Elasticsearch dies due to a segfault or
other similar native issue, a core dump is useful in diagnosing the
problem. Yet core dumps are written to the working directory, which is
read-only for most installations of Elasticsearch. This commit changes
the working directory to the logs dir which should always be writeable.
Before we enabled sort optimization on long, double and date types,
but left other types for follow-up.
This enables sort optimization on float and half_float types.
Optimizations on INT, BYTE, SHORT are left for follow-up,
because they need more work: we currently use SORT.LONG
type for all integer types and this doesn't allow to use
optimization.
Today when we are adding a ElasticsearchException, we specify a versionAdded TransportVersion (the transport version from which we support it); this version is checked by the isRegistered method:
return version.onOrAfter(elasticsearchExceptionHandle.versionAdded);
This does not play well with backports; when we add a patch version for a backport, normally the procedure would be to change the code above take also the patch into account, like:
version.orOnAfter(versionAdded) || version.isPatchFrom(versionPatched)
This PR updates ElasticsearchException to have more than just "version added", so that we can do patches as described above.
The doc values codec iterates a few times over the doc value instance that needs to be written to disk. In case when merging and index sorting is enabled, this is much more expensive, as each time the doc values instance is iterated a merge sorting is performed (in order to get the doc ids of new segment in order of index sorting).
There are several reasons why the doc value instance is iterated multiple times:
* To compute stats (num values, number of docs with value) required for writing values to disk.
* To write bitset that indicate which documents have a value. (indexed disi, jump table)
* To write the actual values to disk.
* To write the addresses to disk (in case docs have multiple values)
This applies for numeric doc values, but also for the ordinals of sorted (set) doc values.
This PR addresses solving the first reason why doc value instance needs to be iterated. This is done only when in case of merging and when the segments to be merged with are also of type es87 doc values, codec version is the same and there are no deletes. Note this optimized merged is behind a feature flag for now.
This PR adds two new REST endpoints, for listing queries and getting information on a current query.
* Resolves#124827
* Related to #124828 (initial work)
Changes from the API specified in the above issues:
* The get API is pretty initial, as we don't have a way of fetching the memory used or number of rows processed.
List queries response:
```
GET /_query/queries
// returns for each of the running queries
// query_id, start_time, running_time, query
{ "queries" : {
"abc": {
"id": "abc",
"start_time_millis": 14585858875292,
"running_time_nanos": 762794,
"query": "FROM logs* | STATS BY hostname"
},
"4321": {
"id":"4321",
"start_time_millis": 14585858823573,
"running_time_nanos": 90231,
"query": "FROM orders | LOOKUP country_code ON country"
}
}
}
```
Get query response:
```
GET /_query/queries/abc
{
"id" : "abc",
"start_time_millis": 14585858875292,
"running_time_nanos": 762794,
"query": "FROM logs* | STATS BY hostname"
"coordinating_node": "oTUltX4IQMOUUVeiohTt8A"
"data_nodes" : [ "DwrYwfytxthse49X4", "i5msnbUyWlpe86e7"]
}
```
* Refactor remote cluster handling in Analyzer
- Initialize clusters earlier
- Simplify cluster set calculation
- No need to keep separate skipped list for enrich resolution
This commit adds support for system data streams reindexing. The system data stream migration extends the existing system indices migration task and uses the data stream reindex API.
The system index migration task starts a reindex data stream task and tracks its status every second. Only one system index or system data stream is migrated at a time. If a data stream migration fails, the entire system index migration task will also fail.
Port of #123926
The mostly-optional parameters to `createBlobContainer` are getting
rather numerous in this test harness which makes the tests hard to read.
This commit introduces a builder to help name the provided parameters
and skip the omitted ones.
In #88707 we changed the behaviour here to run the shard-snapshot
initialization tasks all in sequence. Yet these tasks do nontrivial work
since they may flush to acquire the relevant index commit, so with this
commit we go back to distributing them across the `SNAPSHOT` pool again.
* Adding validation to ElasticsearchInternalService
* Update docs/changelog/123044.yaml
* [CI] Auto commit changes from spotless
* Removing checkModelConfig
* Fixing IT
* [CI] Auto commit changes from spotless
* Remove DeepSeek checkModelConfig and fix tests
* Cleaning up comments, updating validation input type, and moving model deployment starting to model validator
---------
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
When creating the an empty lifecycle we used to use the default
constructor. This is not just for efficiency but it will allow us to
separate the default data and failures lifecycle in the future.
This converts this class to support multi-project.
There is a drive-by refactoring to split multiple independent tests in one method into separate methods. (Look at this change with whitespace changes hidden to see the 'real' change.)
The `elasticsearch-certutil http` command, and security auto-configuration,
generate the HTTP certificate and CA without setting the `keyUsage` extension.
This PR fixes this by setting (by default):
- `keyCertSign` and `cRLSign` for self-signed CAs
- `digitalSignature` and `keyEncipherment` for HTTP certificates and CSRs
These defaults can be overridden when running `elasticsearch-certutil http`
command. The user will be prompted to change them as they wish.
For `elasticsearch-certutil ca`, the default value can be overridden by passing
the `--keysage` option, e.g.
```
elasticsearch-certutil ca --keyusage "digitalSignature,keyCertSign,cRLSign" -pem
```
Fixes#117769
Transport actions have associated request and response classes. However,
the base type restrictions are not necessary to duplicate when creating
a map of transport actions. Relatedly, the ActionHandler class doesn't
actually need strongly typed action type and classes since they are lost
when shoved into the node client map. This commit removes these type
restrictions and generic parameters.
