We accidentally released the `9_051_0_00` and `9_052_0_00` transport
versions to serverless without their backing code. We can't take that
back. It's live. You can't unbake the cake.
That version of Elasticsearch will claim to be `9_052_0_00` but actually
speak `9_050_0_00` code. To make main compatible with that version we're
bumping all transport versions in main *past* those numbers.
This is a forbidden action. You may not do this. I may not do this. It
is not safe. But it is the only way to make the world whole again.
In all seriousness, everything will be fine. It's against the rules but
sometimes you have to break rules to put things right. We believe some
serverless dev environments may break which is sad but it's worth it to
bring everything back to sane land.
This PR introduces a time-series aggregation function that collects the
last value of each time series within each grouping.
For example:
TS index
| STATS sum(last_over_time(memory_usage)) BY cluster, bucket(@timestamp, 1minute)
With the change to using the logs dir as the working dir of the
Elasticsearch process we need to ensure the logs dir exists within the
CLI instead of later during startup.
relates #124966
The docs about the queue in a `fixed` pool are a little awkwardly
worded, and there is no mention of the queue in a `scaling` pool at all.
This commit cleans this area up.
Missed a spot here when moving this to delayed deserialization, we can leak pending batch results here on exceptions.
closes#126994closes#126995closes#126975closes#126999closes#127001closes#126974closes#127008
* updating documentation to remove duplicate and redundant wording from 9.x
* Update links to rerank model landing page
---------
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
We addressed the empty top docs issue with #126385 specifically for scenarios where
empty top docs don't go through the wire. Yet they may be serialized from data node
back to the coord node, in which case they will no longer be equal to Lucene#EMPTY_TOP_DOCS.
This commit expands the existing filtering of empty top docs to include also those that
did go through serialization.
Closes#126742
These tests had the potential to fail when subsequent requests would hit
different nodes with different versions of the cluster state.
Only one of these tests failed already, but we fix the other ones
proactively to avoid future failures.
Fixes#126746
Follow-up to #126138.
We can now release requst bytes directly after deserialization.
Also, request instances need not go through a ref-counting cycle when forking,
removing some contention from transport threads.
There is no actual need to reference-count InboundMessage instances. Their lifecycle is completely linear and we can simplify it away. This saves a little work directly but more importantly, it enables more eager releasing of the underlying buffers in a follow-up.
---------
Co-authored-by: David Turner <david.turner@elastic.co>
* Default new semantic_text fields to use BBQ when models are compatible
* Update docs/changelog/126629.yaml
* Gate default BBQ by IndexVersion
* Cleanup from PR feedback
* PR feedback
* Fix test
* Fix test
* PR feedback
* Update test to test correct options
* Hack alert: Fix issue where mapper service was always being created with current index version
Build jump table (disi) while iterating over SortedNumericDocValues for encoding the values, instead of separately iterating over SortedNumericDocValues just to build the jump table.
In case when indexing sorting is active, this requires an additional merge sort. Follow up from #125403
The compatibleWithOptimizedMerge() method doesn't the case when doc value fields don't exist in all segments. It is like that for smaller segments not all fields exist. The change addresses that by skipping such doc value producers instead of returning that a field can't be merged using the optimized method.
There are existing metrics for the active number of threads, but it seems tricky to go from those to a "utilisation" number because all the pools have different sizes.
This commit adds `es.thread_pool.{name}.threads.utilization.current` which will be published by all `TaskExecutionTimeTrackingEsThreadPoolExecutor` thread pools (where `EsExecutors.TaskTrackingConfig#trackExecutionTime` is true).
The metric is a double gauge indicating what fraction (in [0.0, 1.0]) of the maximum possible execution time was utilised over the polling interval.
It's calculated as actualTaskExecutionTime / maximumTaskExecutionTime, so effectively a "mean" value. The metric interval is 60s so brief spikes won't be apparent in the measure, but the initial goal is to use it to detect hot-spotting so the 60s average will probably suffice.
Relates ES-10530
* Temporarily bypass competitive iteration for filters aggregation (#126956)
* Bump versions after 9.0.0 release
* fix merge conflict
* Remove 8.16 from branches.json
* Bring version-bump related changes from main
* [bwc] Add bugfix3 project (#126880)
* Sync version bump changes from main again
---------
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: elasticsearchmachine <58790826+elasticsearchmachine@users.noreply.github.com>
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
We should use DelayableWriteable here as well just like we do with per-shard results.
The heap savings of making use of this tool are quite significant at times and without
using it we could actually regress in terms of heap use relative to non-batched execution
in corner cases of a low but larger than one number of shards per node.
* Updating text_similarity_reranker documentation
* Updating docs to include urls
* remove extra THE from the text
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Reindex task may not have completed by time status is called, so remove assertion that relies on this. Following assertions are set in the action which starts task, so will be correct whether or not task has completed.
CollectionUtils.uniquify is based on C++ std::unique. However, C++
iterators are not quite the same as Java iterators. In particular,
advancing them only allows grabbing the value once. This commit reworks
uniquify to be based on list indices instead of iterators.
closes#126883
