Commit graph

1393 commits

Author SHA1 Message Date
Brian Seeders
c2ad34b97f
[release-notes] Update automation to use new markdown format (#124161) 2025-05-28 14:53:02 -04:00
Chris Hegarty
2134a1e6e1
Remove pesky restricted method warnings in tests. (#128212)
Since our unit and IT tests run as unnamed modules we should add `--enable-native-access=ALL-UNNAMED` to suppress the pesky warning that arise from calls to `System::load`.
2025-05-27 09:23:09 +01:00
Brian Seeders
a13a46a2da
[tests] StandaloneRestIntegTestTask should set a minimum maxParallelForks of 1 (#126346) 2025-05-23 10:42:34 -04:00
Rene Groeschke
38c90ca8d4
Restructure docker files for docker distributions (#127960)
Restructures docker files for docker distributions

- Put Dockerfiles in specific distro specific folders keeping "Dockerfile" naming convention
- Allows better ide support
- Allows easier renovate integration
- Explicitly set base image in dockerfile
- simplify renovate configuration
- Cleanup DockerBase file to not contain ess fips base image information

This lives now in the Dockerfile content directly

* Workaround docker test issue

* Fix labels for fips image
2025-05-19 19:47:34 +02:00
Rene Groeschke
9b1815c84a
[Build] Update Develocity Plugin to 4.0.1 (#128053)
After updating Develocity to 2015.1.3 we can also update the according plugin
2025-05-13 21:37:44 -07:00
Ryan Ernst
60ad8ba744
Remove custom SecurityManager (#127778)
Since SecurityManager is no longer used, the custom subclass of
SecurityManager, SecureSM, is no longer needed.
2025-05-06 16:16:46 -07:00
Rene Groeschke
a2e580fb60
Update Gradle wrapper to 8.14 (#126519)
* Fix PatternSetFactory incompatibility
* Update ospackage plugin
* Remove ambigious method definitions
* Cleanup verification metadata
* Some cleanup on unused methods and attributes
2025-05-06 13:00:15 +02:00
Nik Everett
15c461d842
ESQL: Disable format checks on generated imports (#127648)
This builds the infrastructure to disable spotless and some checkstyle
rules on generated imports. This works around the most frustrating part
of ESQL's string template generated files - the imports. It allows
unused and out of order imports. This can let us remove a lot of
cumbersome, tricky, and fairly useless `$if$` blocks from the templates.
2025-05-05 18:02:56 +02:00
Rene Groeschke
d546fc8217
Adjust docker fips entrypoint and cmd (#127630)
Also extract docker fips configuration in explicit docker file
2025-05-02 14:43:55 +02:00
Chris Hegarty
898dd47c93
Upgrade to Lucene 10.2.1 (#127343)
This commit upgrades to Lucene 10.2.1 (from a previous 10.2.1-SNAPSHOT).

Given that we're on a snapshot, there are a few things to note:

 * No index version update is necessary, we already have IndexVersions.UPGRADE_TO_LUCENE_10_2_1, and there are no format changes in the final non-snapshot release.
2025-05-01 14:24:42 +01:00
Rene Groeschke
aa309515f8
Rename docker fips image to cloud-ess-fips (#127561) 2025-04-30 15:56:24 +02:00
David Turner
b028c0af56
Upgrade repository-s3 to AWS SDK v2 (#126843)
Closes #120993
2025-04-24 21:21:03 +10:00
Julio
76f6006a42
Update Elasticsearch main with snapshot version of Lucene (#127125) 2025-04-22 00:25:08 +02:00
Brian Seeders
3d78c9428c
[bwc] Add bugfix3 project (#126880) 2025-04-16 16:27:03 -04:00
David Turner
aa40147142
Add integ tests for ftp:// URL repository (#126757)
We document support for snapshot repositories using `ftp://` URLs but it
seems this functionality has not worked for many years because of
security-manager restrictions, although nobody noticed because it was
not covered by any tests. The migration to the Entitlements framework
means that this functionality now works again, so this commit adds tests
to make sure we do not break it again in future.
2025-04-15 12:57:00 +01:00
Lorenzo Dematté
2697a3a872
Patcher for AWS SDKv2 locale-dependent formatting (#126326)
AWS SDK v2 has a bug (aws/aws-sdk-java-v2#5968) where PathResolver uses locale-dependent formatting.

This PR adds a patcher to the discovery-ec2 build process to replace calls to String.format(<format>, <args>) with String.format(Locale.ROOT, <format>, <args>).

Relates to ES-11279
2025-04-15 12:49:56 +02:00
Ignacio Vera
ffdfcec334
Upgrade to Lucene 10.2.0 (#126594)
This commit upgrade Elasticsearch to lucene 10.2.0
2025-04-14 13:50:52 +02:00
Lorenzo Dematté
e4af657c12
Patcher improvements (HDFS) (#126449)
Patchers transform specific classes in some "broken" dependencies to ensure they behave correctly (fixing a bug, disabling some undesired or dangerous behaviour, updating calls to deprecated or removed method overloads).

If we upgrade one of the dependencies we patch, we have a concerns that the patchers may not work against the classes in the new version.
This PR addresses this concern by introducing a check on the SHA256 digest of the class, to ensure we are operating on the same bytes the patcher was designed for; if the digest changes that means the class has been changed (e.g. for a dependency update). If that happens, we break the build process with a specific error, so we can double check that the patchers still work against the new classes.

Extracted from #126326

Relates to ES-11279
2025-04-11 17:20:45 +02:00
Rene Groeschke
ca195736ca
Address review feedback on es default docker image (#126330)
This addresses feedback we got for our default image at https://github.com/docker-library/official-images/pull/18692
This also introduces separate docker source files to make maintaining those easier.

We cannot take over all suggested changes as we require certain settings to have our packaging tests pass as expected.
2025-04-10 17:14:22 +02:00
Ryan Ernst
3bac50e818
Use logs dir as working directory (#124966)
In the unexpected case that Elasticsearch dies due to a segfault or
other similar native issue, a core dump is useful in diagnosing the
problem. Yet core dumps are written to the working directory, which is
read-only for most installations of Elasticsearch. This commit changes
the working directory to the logs dir which should always be writeable.
2025-04-09 07:07:11 -07:00
Rene Groeschke
6e4cb8142b
Add initial esql test benchmark tests (#126224)
* Add initial esql test benchmark tests
* Fix build-benchmark pipeline
* Update gredle profiler
2025-04-08 13:28:34 +02:00
Rene Groeschke
6cfb2e9d4a
Resolve fips specific metricbeat and filebeat libs for ES fips image (#126312)
Also updated DRA handling to reflect resolving fips specific artifacts
2025-04-05 06:47:30 +11:00
Rene Groeschke
d39db02a74
Use official ubi9-minimal base image for es docker (#126280)
Based on feedback we got here: Based on feedback we got
https://github.com/docker-library/official-images/pull/18692\#issuecomment-2749045229
we can not rely on non official images here
2025-04-04 22:41:16 +11:00
elastic-renovate-prod[bot]
7b46621c19
Pin docker.elastic.co/wolfi/chainguard-base-fips Docker tag to ebfc3f1 (#126065)
This PR contains the following updates:

| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base-fips | pinDigest |  -> `ebfc3f1`
|

---

### Configuration

📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
2025-04-02 06:22:31 +11:00
elastic-renovate-prod[bot]
de5377df45
Update docker.elastic.co/wolfi/chainguard-base:latest Docker digest to 29150cd (#126066)
This PR contains the following updates:

| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `c4e10ec` ->
`29150cd` |

---

### Configuration

📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
2025-04-02 06:19:48 +11:00
Joe Gallo
5c9cdeeb87
Bump junit and hamcrest (#126055) 2025-04-01 14:05:58 -04:00
Mark Vieira
66ba3c2a53
Convert CCR module tests to new testing framework (#125894) 2025-03-31 14:16:19 -07:00
Mark Vieira
421392af99
Convert more x-pack modules to new test framework (#125874) 2025-03-31 12:27:11 -07:00
Rene Groeschke
511c50967d
[CI] Fix LegacyYamlRestTestPluginFuncTest failure for not available java toolchain (#125976)
Wire in toolchain plugins into AbstractRestResourcesFuncTest to allow
resolving jdk

fixes #125534 , #125533 ,  #125532, #125531
2025-04-01 05:30:49 +11:00
Mark Vieira
e75891e49a
Convert x-pack fleet project to new test framework (#125743) 2025-03-31 10:30:38 -07:00
Rene Groeschke
f79ff970ea
Ignore Fips docker images in packages tests for now (#125942)
* Ignore Fips docker images in packages tests for now

Handled in a later work package

* Fix typo
2025-03-31 18:23:26 +02:00
Rene Groeschke
611f5cba4f
[BUILD] Support AARCH64 for docker fips images (#125927)
This updates the docker fips base image which support arm and x84
architectures
2025-03-31 22:41:04 +11:00
Mark Vieira
0c50403ebb
Migrate x-pack core module to new testing framework (#125740) 2025-03-27 10:13:29 -07:00
Rene Groeschke
b476ee19b5
Try fixing mutedTest.yml file not found (#125763) 2025-03-27 13:31:52 +01:00
Mark Vieira
e149a3e10d
Convert :test projects to new testing framework (#125724) 2025-03-26 16:11:50 -07:00
Mark Vieira
930b4ab995
Convert remaining plugin projects to new test clusters framework (#125626) 2025-03-26 13:44:07 -07:00
Mark Vieira
0388a5980c
Migrate legacy QA projects to new test clusters framework (#125545) 2025-03-26 10:05:56 -07:00
elastic-renovate-prod[bot]
2204b4a13c
Update docker.elastic.co/wolfi/chainguard-base:latest Docker digest to c4e10ec (#124565)
This PR contains the following updates:

| Package | Update | Change | |---|---|---| |
docker.elastic.co/wolfi/chainguard-base | digest | `15a4191` ->
`c4e10ec` |

---

### Configuration

📅 **Schedule**: Branch creation - "after 1pm on tuesday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once
you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate
Bot](https://redirect.github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMDcuMCIsInVwZGF0ZWRJblZlciI6IjM5LjEwNy4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyI6RGVsaXZlcnkvUGFja2FnaW5nIiwiPm5vbi1pc3N1ZSIsIlRlYW06RGVsaXZlcnkiLCJhdXRvLW1lcmdlLXdpdGhvdXQtYXBwcm92YWwiXX0=-->
2025-03-27 04:01:35 +11:00
Mark Vieira
d72d81a0eb
Convert remaining module projects to new test clusters framework (#125613) 2025-03-26 08:42:55 -07:00
Rene Groeschke
653c179b08
[Build] Add FIPS docker image for GovCloud (#117152)
- Adds docker image based on chainguard base fips image
- x86 only for now as the base image is x86 only
- the image does not provide any elasticsearch.yml configuration. for testing purposes you can follow the elasticsearch fips guide available at https://github.com/elastic/FIPSGuide/tree/main/elasticsearch

The image is shipped with:
- org.bouncycastle:bc-fips:1.0.2.5 and org.bouncycastle:bctls-fips:1.0.19 in Elasticsearch libs folder
- config/jvm.options.d/fips.options for fips specific JVM options
- fips_java.security file
- fips_java.policy

Out of scope:
- Add packaging test coverage (part of later PR as we want to provide that image for testing early and packaging tests require more general restructuring for support fips scenarios)
2025-03-26 16:15:15 +01:00
Mark Vieira
65751062f7
Re-enable VerifyVersionConstantsIT (#125605) 2025-03-25 12:16:53 -07:00
Rene Groeschke
59a55c85f3
[Gradle] Wire java toolchain plugin into integration tests (#125347)
* [Gradle] Wire java toolchain plugin into integration tests

This addresses transient issues we see in the integration tests where previously resolved jdk toolchain
is not available. See https://gradle-enterprise.elastic.co/s/66msrp7n2nsdy/tests/task/:build-tools-internal:integTest/details/org.elasticsearch.gradle.internal.test.rest.LegacyYamlRestTestPluginFuncTest/yamlRestTest%20executes%20and%20copies%20api%20and%20tests%20to%20correct%20source%20set\?top-execution\=1 for example

* Fix fix
2025-03-23 19:11:24 +01:00
Armin Braun
50437e79d3
Cleanup missing use of StandardCharsets (#125424)
Random annoyance that I figured, I'd just fix globally:
We can do a bit of a cleaner job when doing byte <-> string conversion here and there.
2025-03-21 20:10:15 +01:00
Rene Groeschke
6f7a206a8a
Update Gradle Enterprise Plugin (Develocity) to 3.19.2 (#125137)
with develocity updated to 2024.3.4 we are unblocked on updating the gradle enterprise plugin
2025-03-19 19:19:54 +01:00
Ryan Ernst
a7c71669a4
Update bundled JDK to Java 24 (#125159)
Bumping bundled JDK to Java 24 since it is now GA.
2025-03-19 01:46:37 +02:00
Rene Groeschke
12c2dd5903
[Gradle] Autoprovision jvm for gradle daemon (#124071)
With Gradle 8.13 we can now auto provision the jdk used by the gradle daemon. Our configuration relies on jdk21.
With this autoprovisioning enabled each gradle build will use adoptium jdk 21 for gradle jvm
2025-03-18 19:50:05 +01:00
Rene Groeschke
df7c20acd8
[Gradle] Fix configuration cache issues in distribution subproject (#124281)
* [Gradle] Fix configuration cache issues in distribution subproject

- do not reference tasks from task execution time
- do not use project references at task execution time
- use layout.settingsDir for resolving root project dir

* Fix test coverage

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-03-18 15:27:47 +01:00
Rene Groeschke
7f722e98eb
[Build] Rework internal build plugin plugin to work with Isolated Projects (#123461)
This fixes a general flaw in our build logic where we reach out to configurations of other projects.
This is not best practice and breaks future initiatives like IsolatedProjects that allow parallel configuration
of subprojects.
2025-03-17 08:27:03 +01:00
Rene Groeschke
ae569def9c
[Build] Require reason for usesDefaultDistribution (#124707)
This makes using usesDefaultDistribution in our test setup for explicit by requiring a reason why it's needed.
This is helpful as part of revisiting the need for all those usages in our code base.
2025-03-17 08:25:39 +01:00
David Turner
d5d81a57db
Upgrade AWS v2 SDK to 2.30.38 (#124738)
Adopts fix for https://github.com/aws/aws-sdk-java-v2/issues/5754,
removing the need for read permissions of sensitive files in
`${HOME}/.aws`.
2025-03-14 07:38:39 +11:00