This commit adds compatibility tests that target ES revisions that align with specific Lucene versions. In this case, we are intending to upgrade from Lucene 10.0 to 10.1. Since no on-prem Elasticsearch release exists with 10.0, we need another method to ensure compatibility with Lucene 10.0 indicies.
The work here is a bit hacky since all our compatibility testing infrastructure is centered around versions and we're now effectively doing compatibility tests between two different revisions of Elasticsearch that both report the same version. Ideally this specific testing would be replaced by unit tests, rather that reusing our full cluster restart tests for this purpose.
We'll also want to bump the commit referenced in the CI pipelines here to align with the last commit using Lucene 10.0.
Enable logsdb by default if logsdb.prior_logs_usage has not been set to true.
Meaning that if no data streams were created matching with the logs-- pattern in 8.x, then logsdb will be enabled by default for data streams matching with logs-*-* pattern.
Also removes LogsPatternUsageService as with version 9.0 and beyond, this component is no longer necessary.
Followup from #120708Closes#106489
The mrjar plugin adds support for sourcesets named in the form mainNN,
which adds the appropriate compiler and other settings for that version
of Java, and produces a multi-release jar. Having multi-release jars
only makes sense for versions of java newer than the minimum compile
version. This commit adds validation that the version is not too old.
Note that the check is slightly relaxed; it allows mainNN where NN is
equal to the min java version. This is due to the desire to keep
code using incubating modules separate because warnings must be
disabled.
Reverts elastic/elasticsearch#120214 Reverting this change because this
is causing failing tests on AmazonLinux 2023. The problem with wrong
version of `docker-compose` has been fixed in the VM images so builds
should no longer fail.
Upgrading from using V1 to be able to run on CI machines which
`docker-compose` command use some detached version which doesn't work
nicely with gradle-docker-compose-plugin.
This PR contains the following updates:
| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `eef54b3` ->
`dd66bee` |
---
### Configuration
📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
relying on extra properties turns out problematic in general. This
introduces a typed task that has a util method enablePreview which is used
in the elasticsearch and elasticsearch serverless idea setup
This PR contains the following updates:
| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `bfdeddb` ->
`eef54b3` |
---
### Configuration
📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
We add explicit support for running build with -Druntime.java=24 and
-Druntime.java=24) By default it takes the build that was available at
implementation time (b29 for 24 and b3 for 25)
You can pass the build number also dynamically by running your build
e.g. with
```
-Druntime.java=24 -Druntime.java.build=29
```
which will resolve java ea24 build 29
fixes#118329
This updates the gradle wrapper to 8.12
We addressed deprecation warnings due to the update that includes:
- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions
* Reapply "[Build] Do not invalidate configuration cache when branch is switched (#118894)" (#119300)
The original PR (#118894) has broken serverless.
* Fix gitinfo plugin for serverless usage
* Update buildscan git revision reference
We depend on the `reactive-streams` library in a couple of places. This
commit extracts the version to the top-level `version.properties` file
to make sure we keep these versions in sync.
Shell utility in hdfs tries to execute a local script statically to
determine whether setsid is available. With the security manager this
doesn't work, but hdfs catches the SecurityException and assumes false.
With entitlements this doesn't work since hdfs does not know about our
NotEntitledException.
This commit reworks the patching of hdfs-client-api to use asm. It then
adds patching of hdfs' Shell class to replace the method that tries to
execute.
When compiling we use a compiler for the minimum java version. However,
javadoc is left to whatever Java gradle uses. This commit adjusts
javadoc to also use a javadoc tool for the minimum java version.
We should basically be able to skip the third party dependency audit
tasks if no third party dependency has changed.
Filtering out the project dependencies allows way better uptodate and
caching behaviour. We are only interested in thirdparty libs anyhow in
this context.
A positive side effect of the reduced class path is a quicker execution
of the task
This removes support for running date aggregations over boolean fields. This has never been useful, was deprecated in 7.x, and is finally being disabled for 9.0.
In an effort to improve performance and continue to provide unique
seeded scores for documents in the same index, we are switching from _id
to _seq_no.
Requiring a field that is "unique" for a field and to help with random
scores is burdensome for the user. So, we should default to a unique
field (per index) when the user provides a seed.
Using `_seq_no` should be better as:
- We don't have to grab stored fields values
- Bytes used are generally smaller
Additionally this removes the deprecation warning.
Marking as "breaking" as it does change the scores & behavior, but the
API provide is the same.
This makes checkPart4 tasks in general configuration cache compliant.
We do not enable it by default in our pipelines yet. That will
eventually be done in separate PRs. To track any regressions we cover
the checkPart4 task in the weekly gradle cache validation build job
Also provide caching support for buildparams provider
* Extract BuildParameterExtension public api into interface
* Make tests better readable
* Fix test flakyness
This change introduces a new QA project to test Lucene
support for reading indices created in version N-2.
The test suite is inspired from the various full-cluster
restart suites we already have. It creates a cluster in
version N-2 (today 7.17.25), then upgrades the cluster
to N-1 (today 8.18.0) and finally upgrades the cluster
to the current version (today 9.0), allowing to execute
test methods after every upgrade.
The test suite has two variants: one for searchable
snapshots and one for snapshot restore. The suites
demonstrates that Elasticsearch does not allow
reading indices written in version N-2 but we hope
to make this feasible. Also, the tests can be used for
investigation and debug with the command
`./gradlew ":qa:lucene-index-compatibility:check" --debug-jvm-server`
Relates ES-10274
Today each relevant module defines the version of the AWS SDK that it
uses, which means there's a risk that we use different versions in
different modules. This commit pulls the version declarations to the top
level to make sure we keep everything in sync.
Somehow files can be lost before the build ends up uploading them,
presumable from temporarily file deletion after tests complete. This
commit guards against this case so that the build will not completely
fail, but instead log a warning.
