Static fields dont do well in Gradle with configuration cache enabled.
- Use buildParams extension in build scripts
- Keep BuildParams.ci for now for easy serverless migration
- Tweak testing doc
This PR contains the following updates:
| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `c16d3ad` ->
`90888b1` |
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MjUuMSIsInVwZGF0ZWRJblZlciI6IjM3LjQyNS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
Fix incompatibility with 8.8 and our internal api usages
- Update ospackage to a version that contains a fix we provided
- Tweak build logic to avoid deprecation warnings
- Use newer permission api
- Use custom shadowplugin
- Rework ElasticsearchDistribution dependencies resolution
- Update Gradle wrapper to 8.8
To make those docker images available for our testcontainer based
fixtures we need to publish them on a regular basis to ensure the source
and the images are in sync. This adds some convenience plugin to take
care of publishing our docker test fixtures
This commit makes zstd compression available to Elasticsearch. The
library is pulled in through maven in jar files for each platform, then
bundled in a new platform directory under lib. Access to the zstd
compression/decompression is through NativeAccess.
x-content embeds its jackson implementation inside its jar. This commit
formalizes the setup for this embedding with a gradle plugin so that it
can be reused by other libs.
This commit adds an elasticsearch gradle plugin which sets up a project
to build an MRJAR. By applying the plugin, the src dir is checked for
any directories matching mainXX where XX is the java version number.
A source set is automatically setup, an appropriate compiler tied to
that source set, and it's output placed in the correct part of the final
jar. Additionally, the sourceset allows use of preview features in that
verison of Java, and the preview bits are stripped from the resulting
class files.
Add an updateVersions gradle task that updates the version numbers in the ES repo as part of release automation. This will be used by the release automation tools, replacing old functionality to manually modify the java files.
* Update minio fixture and usages to rely on new test cluster framework and testcontainer
* Cache test fixture resources during packer caching
Add logic that ensures we resolve docker images resolved from docker registry as part of our packer
cache setup.
- Replace docker-compose based s3-fixture usages to use plain java
- Support lazy evaluated system properties for local cluster definitions
- Make it work with configuration cache enabled
- Port Minio fixture usage to leverage test container testing library.
* Make gradle build finished logic CC compatible
* Make ElasticsearchBuildFinishedPlugin configuration cache aware
* Add gradle enterprise plugin to buildlibs version catalogue
This commit adds a StringTemplate Gradle plugin based on Antlr's StringTemplate.
With this plugin we can template Java source code and generate different variants of it, e.g. implementations that vary only by primitive type.
This commit updates the plugin cli and scanner components to use ASM 9.5.
The update is required to successfully test with JDK 21. Tests in this component programatically run the java source compiler, which generates class files with major version 65, then tries to parse those generated class files. Without this change the tests fail with java.lang.IllegalArgumentException: Unsupported class file major version 65.
Using gradle toolchain support in gradle requires refactoring how the composite build is composed.
We added three toolchain resolver
1. Resolver for resolving defined bundled version from oracle as openjdk
2. Resolve all available jdks from Adoption
3. Resolve archived Oracle jdk distributions.
We should be able to remove the JdkDownloadPlugin altogether without having that in place, but we'll do that in a separate effort.
Fixes#95094
This commit adds a new test framework for configuring and orchestrating
test clusters for both Java and YAML REST testing. This will eventually
replace the existing "test-clusters" Gradle plugin and the build-time
cluster orchestration.
* Add support for building against defined build-ids of dependent artifacts
* Introduce DraResolvePlugin and functional tests
* Declare initial dra snapshot ci job for elasticsearch
We run the DRA builds against ubuntu as other release related ci jobs have in the past. Also we only have libs2xmlutills available on our ubuntu based ci workers
This adds the generation and upload logic of Gradle dependency graphs to snyk
We directly implemented a rest api based snyk plugin as:
the existing snyk gradle plugin delegates to the snyk command line tool the command line tool
uses custom gradle logic by injecting a init file that is
a) using deprecated build logic which we definitely want to avoid
b) uses gradle api we avoid like eager task creation.
Shipping this as a internal gradle plugin gives us the most flexibility as we only want to monitor
production code for now we apply this plugin as part of the elasticsearch.build plugin,
that usage has been for now the de-facto indicator if a project is considered a "production" project
that ends up in our distribution or public maven repositories. This isnt yet ideal and we will revisit
the distinction between production and non production code / projects in a separate effort.
As part of this effort we added the elasticsearch.build plugin to more projects that actually end up
in the distribution. To unblock us on this we for now disabled a few check tasks that started failing by applying elasticsearch.build.
Addresses #87620
This PR reworks the testing conventions precommit plugin. This plugin now:
- is compatible with yaml, java rest tests and internalClusterTest (aka different sourceSets per test type)
- enforces test base class and simple naming conventions (as it did before)
- adds one check task per test sourceSet
- uses the worker api to improve task execution parallelism and encapsulation
- is gradle configuration cache compatible
This also ports the TestingConventions integration testing to Spock and removes the build-tools-internal/test kit folder that is not required anymore. We also add some common logic for testing java related gradle plugins.
We will apply further cleanup on other tests within our test suite in a dedicated follow up cleanup
This PR upstreams a custom Gradle Plugin for compiling modular source
projects, that we've been using for a while now over on the modules
branch. The Plugin infers which dependencies should be on the module
path and which dependencies should be on the class path, when compilng
the main java sourceset. This supports the "bottom up" modularization of
Elasticsearch, where parent projects are modularized before child
projects. Allowing to modularize the core of the server independently of
the extension components, that can themselves be modularized later.
With this plugin various subprojects can be modularized (by adding a
module-info.java in the source root) with minimal fuss - no custom
gradle logic is required by the subproject, beyond a couple of
subproject specific compiler lint suppressions.
The inference in the plugin is driven by walking the project
dependencies, searching for the presence of a module-info.java,
afterwhich all subprojects will be put on the module path. Again, this
supports "bottom up" modularization.
Co-authored-by: ChrisHegarty <christopher.hegarty@elastic.co>
Co-authored-by: Mark Vieira <portugee@gmail.com>
Co-authored-by: Ryan Ernst <ryan@iernst.net>
Co-authored-by: Rene Groeschke <rene@breskeby.com>
We introduce the use of a Gradle version catalogue for handling build related dependencies.
This provides type safe accessors for dependencies and allow centralised version definitions.
Later we want to move all our dependency handling to version catalogues.
Since this is a Gradle feature we remove long term maintenance cost for custom version handling in
our build and make centralised version handling more straight forward and support better tooling
based on version catalogues
Fixes asm version alignment on the way using 9.3 everywhere in our build logic.
Also adding test coverage and fixing certain issues we stumbled into
when resolving project dependencies of compileClasspath we need to use allDependencies instead of dependencies as usually no dependencies are added directly to compileClasspath but via implementation, compileonly and api and friends
Fixed javadoc setup for projects using shadowed dependencies via shadow plugin
Fixed dealing with skipped javadoc tasks in referenced dependent projects.
In general this PR only fixes expected javadoc generation as it was intended before this PR.
This also contains some tweaks to our gradle integration test fixtures for setting up and debugging test projects
This removes the overhead of zipping up modules that are immediately unzipped again when packaging
the elasticsearch distribution.
We also move some logic for packaging the elasticsearch distribution into a plugin and remove some outdated
overhead dealing with 'meta plugins' when copying modules into a distribution.
Another follow up and related optimization out of scope of this PR is, to also not zip unzip modules declared
for usage in our test cluster setups.
This partially addresses #76726.
This was used to mark dependencies as optional. Within
Gradle those deps are handled as usual and optional is only
added to the according pom file. When packaging our distro those
modules are added anyhow including the optional dependencies
The initial motiviation to remove the plugin was it is incompatible
with the gradle configuration cache
