This PR contains the following updates:
| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `d74b1fd` ->
`6387bd4` |
---
### Configuration
📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoiOC4xNiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
* ESQL: Speed up VALUES for many buckets (#123073)
Speeds up the VALUES agg when collecting from many buckets.
Specifically, this speeds up the algorithm used to `finish` the
aggregation. Most specifically, this makes the algorithm more tollerant
to large numbers of groups being collected. The old algorithm was
`O(n^2)` with the number of groups. The new one is `O(n)`
```
(groups)
1 219.683 ± 1.069 -> 223.477 ± 1.990 ms/op
1000 426.323 ± 75.963 -> 463.670 ± 7.275 ms/op
100000 36690.871 ± 4656.350 -> 7800.332 ± 2775.869 ms/op
200000 89422.113 ± 2972.606 -> 21920.288 ± 3427.962 ms/op
400000 timed out at 10 minutes -> 40051.524 ± 2011.706 ms/op
```
The `1` group version was not changed at all. That's just noise in the
measurement. The small bump in the `1000` case is almost certainly worth
it and real. The huge drop in the `100000` case is quite real.
* Fix
* Compile
Fixes two incompatibilities with Gradle configuration cache in our build scan build finished hook:
referencing static methods from build script
referencing gradle object from closure
(cherry picked from commit 929d39820a)
* Fix failing test(s) in `TimeSeriesDataStreamsIT` (#123378)
When these tests were run around midnight, the use of
`DataStream#getDefaultBackingIndexName` could result in a potential
mismatch in the generated index name and the one that the cluster
actually created. Instead, we need to obtain the backing index and
extract the desired index name from there.
Fixes#123086
Relates #123376
(cherry picked from commit f0f0eeb077)
# Conflicts:
# x-pack/plugin/ilm/qa/multi-node/src/javaRestTest/java/org/elasticsearch/xpack/ilm/TimeSeriesDataStreamsIT.java
* Fix compilation
Rather than checking the license (updating the usage map) on every
single shard, just do it once at the start of a computation that needs
to forecast write loads.
Backport of #123346 to 8.x
Closes#123247
These things can be quite expensive and there's no need to recompute
them in parallel across all management threads as done today. This
commit adds a deduplicator to avoid redundant work.
Backport of #123246 to `8.x`
* Bump json-smart and oauth2-oidc-sdk (#122737)
* Bump json-smart and oauth2-oidc-sdk
---------
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
(cherry picked from commit e16664573e)
# Conflicts:
# gradle/verification-metadata.xml
* fixup! Add back verification data for test dep
If metrics that have the same timestamp and dimensions aren't grouped into the same document, ES will consider them to be a duplicate.
The _metric_names_hash field will be set by the OTel ES exporter.
As it's mapped as a time_series_dimensions, it creates a different _tsid for documents with different sets of metrics.
The tradeoff is that if the composition of the metrics grouping changes over time, a different _tsid will be created.
That has an impact on the rate aggregation for counters.
This PR contains the following updates:
| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `bd40170` ->
`d74b1fd` |
---
### Configuration
📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoiOC4xNiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
The block-on-data-node returns once the data node begins to process the
cluster state update for new snapshot. This is before master can see the
chnages. In edge cases, the listener may be completed too early before
the master can see the new snapshot. This PR flushes the master queue to
ensure the snapshot is visible.
Resolves: #116730
(cherry picked from commit 5d9385f1ca)
# Conflicts:
# muted-tests.yml
This PR contains the following updates:
| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `bfdeddb` ->
`bd40170` |
---
### Configuration
📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoiOC4xNiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
This commit makes sure we reuse the existing static instance when deserializing to avoid excessive heap usage.
# Conflicts:
# server/src/main/java/org/elasticsearch/ingest/IngestStats.java
The nebula info broker plugin takes the information for the manifest from the java project settings rather then from
the compile task configuration. Instead of setting the compiler task configuration explicitly we now set the project
configuration accordingly. Also tweaked the javaTestCompile tasks to keep compiling with general minimum runtime version as we did before
(cherry picked from commit 6e6e42f5d4)
The aggs timeout test waits for the agg to return and then double checks
that the agg is stopped using the tasks API. We're seeing some failures
where the tasks API reports that the agg is still running. I can't
reproduce them because computers. This adds two things:
1. Logs the hot_threads so we can see if the query is indeed still
running.
2. Retries the _tasks API for a minute. If it goes away soon after the
_search returns that's *fine*. If it sticks around for more than a
few seconds then the cancel isn't working. We wait for a minute
because CI can't be trusted to do anything quickly.
Closes#121993
This PR addresses issues around aggregations cancellation, mentioned in https://github.com/elastic/elasticsearch/issues/108701 and other places. In brief, during aggregations collection time, we respect cancellation via the mechanisms in the searcher to poison cancelled queries. But once the aggregation finishes collection, there is no further need to interact with the searcher, so we cannot rely on that for cancellation checking. In particular, deeply nested aggregations can spend a long time constructing the results tree.
Checking for cancellation is a trade off, as the check itself is somewhat expensive (it involves a volatile read), so we want to balance checking often enough that cancelled queries aren't taking up resources for a long time, but not so frequently that it slows down most aggregation queries. Our first attempt to this is to check once when we go to build sub-aggregations, as the worst cases for this that we've seen involve needing to build deep sub-aggregation trees. Checking at sub-aggregation construction time also provides a conveniently centralized method call to add the check to.
---------
Conflicts:
server/src/main/java/org/elasticsearch/search/aggregations/bucket/BucketsAggregator.java
test/framework/src/main/java/org/elasticsearch/search/aggregations/AggregatorTestCase.java
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
* [8.x] ESQL: use field_caps native nested fields filtering (#117201) (#117375) (#121645)
* Just filter the nested fields natively with field_caps support
(cherry picked from commit 73381dbeb1)
* Add import
We need to explicitly add the incubating vector API module to the third
party audit task on Java 24.
(cherry picked from commit 0c667ecd2a)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/precommit/ThirdPartyAuditTask.java
Under very unfortunate conditions tests that check xContent objects
roundtrip parsing (like i.e. SearchHitsTests#testFromXContent)
can fail when we happen to randomly pick YAML xContent type and create
random (realistic)Unicode character sequences that may contain the
character U+0085 (133) from the Latin1 code page. That specific character
doesn't get parsed back to its original form for YAML xContent, which can
lead to rare but hard to diagnose test failures.
This change adds logic to AbstractXContentTestCase#test() which lies at
the core of most of our xContent roundtrip tests that disallows test
instances containing that particular character when using YAML xContent
type.
Closes#97716
Add missing apm-server tail sampling monitoring metrics to stack monitoring mapping. They were missed in #110568.
(cherry picked from commit f3f5135f06)
# Conflicts:
# x-pack/plugin/monitoring/src/main/java/org/elasticsearch/xpack/monitoring/MonitoringTemplateRegistry.java
