Commit graph

1375 commits

Author SHA1 Message Date
Rene Groeschke
a2d5aa4bc5
[Gradle] Fix deprecation warning in release tests after 8.12 update (#119608) (#119626) 2025-01-07 20:05:58 +11:00
Mark Vieira
0009de3575
Upgrade Bouncy Castle FIPS dependencies (#112989) (#119614)
This PR updates `bc-fips` and `bctls-fips` dependencies to the latest
minor versions.

(cherry picked from commit 6ea3e01958)

Co-authored-by: Slobodan Adamović <slobodanadamovic@users.noreply.github.com>
2025-01-07 08:03:17 +11:00
Mark Vieira
5386acee8f
Fix vcs revision label in docker images (#119531) (#119540) 2025-01-04 06:08:21 +11:00
Rene Groeschke
4d17b2193a
Update Gradle wrapper to 8.12 (#118683) (#119357)
This updates the gradle wrapper to 8.12

We addressed deprecation warnings due to the update that includes:

- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions

(cherry picked from commit ba61f8c7f7)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerCloudElasticsearchDistributionType.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerUbiElasticsearchDistributionType.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/Fixture.java
#	plugins/repository-hdfs/hadoop-client-api/build.gradle
#	server/src/main/java/org/elasticsearch/inference/ChunkingOptions.java
#	x-pack/plugin/kql/build.gradle
#	x-pack/plugin/migrate/build.gradle
#	x-pack/plugin/security/qa/security-basic/build.gradle
2024-12-31 08:37:28 +01:00
Mark Vieira
88f96b503d
chore: deps(ironbank): Bump ubi version to 9.5 (#119203) (#119229)
(cherry picked from commit 2c736f49b0)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-24 05:00:37 +11:00
Mark Vieira
d5cf398621
chore: deps(ironbank): Bump ubi version to 9.5 (#119039) (#119143)
<Actions>     <action
id="ad27da7f660d61c82c61599e0e6945827ced1590f4bf36a5f74db07e99c04215">
<h3>deps: Bump ironbank version</h3>         <details
id="c8ee27cd13736547d240c88751dee86bddfbda339d0ee25795d0cc066ff6ea01">
<summary>deps(ironbank): Bump ubi version to 9.5</summary>
<p>change detected:&#xA;&#x9;* key &#34;$.args.BASE_TAG&#34; updated
from &#34;\&#34;9.4\&#34;&#34; to &#34;\&#34;9.5\&#34;&#34;, in file
&#34;distribution/docker/src/docker/iron_bank/hardening_manifest.yaml&#34;</p>
</details>         <details
id="ddfe323476a100a4fecf1aab633e1209ffebe2f0e55366500df03a529b067050">
<summary>deps(ironbank): Bump ubi version to 9.5</summary>
<p>changed lines [25] of file
&#34;/tmp/updatecli/github/elastic/elasticsearch/distribution/docker/src/docker/Dockerfile&#34;</p>
</details>         <a
href="1240723046">GitHub
Action workflow link</a>     </action> </Actions>

---

<table>   <tr>     <td width="77">       <img
src="https://www.updatecli.io/images/updatecli.png" alt="Updatecli logo"
width="50" height="50">     </td>     <td>       <p>         Created
automatically by <a href="https://www.updatecli.io/">Updatecli</a>
</p>       <details><summary>Options:</summary>         <br />
<p>Most of Updatecli configuration is done via <a
href="https://www.updatecli.io/docs/prologue/quick-start/">its
manifest(s)</a>.</p>         <ul>           <li>If you close this pull
request, Updatecli will automatically reopen it, the next time it
runs.</li>           <li>If you close this pull request and delete the
base branch, Updatecli will automatically recreate it, erasing all
previous commits made.</li>         </ul>         <p>           Feel
free to report any issues at <a
href="https://github.com/updatecli/updatecli/issues">github.com/updatecli/updatecli</a>.<br
/>           If you find this tool useful, do not hesitate to star <a
href="https://github.com/updatecli/updatecli/stargazers">our GitHub
repository</a> as a sign of appreciation, and/or to tell us directly on
our <a
href="https://matrix.to/#/#Updatecli_community:gitter.im">chat</a>!
</p>       </details>     </td>   </tr> </table>

(cherry picked from commit 46356bd64d)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-12-20 07:47:23 +11:00
Rene Groeschke
f28df38704
[8.17] Revert cloud wolfi migration (#119068) (#119103)
This reverts the wolfi migration for cloud ess docker images.

(cherry picked from commit 81f725d3d8)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java
#	distribution/docker/build.gradle
#	qa/packaging/src/test/java/org/elasticsearch/packaging/test/DockerTests.java
2024-12-20 03:22:05 +11:00
Mark Vieira
c1b0a0791b
Update jvm.options (#118716) (#119006)
Co-authored-by: Blake Niemyjski <bniemyjski@gmail.com>
2024-12-19 06:44:55 +11:00
Mark Vieira
82ebd0aeb5
Update BWC version logic to support multiple bugfix versions (#117943) (#118116) (#118119) 2024-12-05 16:15:12 -08:00
Rene Groeschke
581b9ab7c0
[8.16] [Gradle] Remove static use of BuildParams (#115122) (#117434)
* [Gradle] Remove static use of BuildParams (#115122)

Static fields dont do well in Gradle with configuration cache enabled.

- Use buildParams extension in build scripts
- Keep BuildParams.ci for now for easy serverless migration
-  Tweak testing doc

(cherry picked from commit 13c8aaeffa)

# Conflicts:
#	TESTING.asciidoc
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/InternalDistributionBwcSetupPlugin.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/RestTestBasePlugin.java
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/rest/compat/compat/AbstractYamlRestCompatTestPlugin.java
#	build.gradle
#	modules/ingest-geoip/qa/full-cluster-restart/build.gradle
#	qa/mixed-cluster/build.gradle
#	x-pack/plugin/ent-search/qa/full-cluster-restart/build.gradle
#	x-pack/plugin/eql/qa/rest/build.gradle
#	x-pack/plugin/fleet/qa/rest/build.gradle
#	x-pack/plugin/kql/build.gradle
#	x-pack/plugin/mapper-unsigned-long/build.gradle
#	x-pack/plugin/ml/qa/multi-cluster-tests-with-security/build.gradle
#	x-pack/plugin/security/qa/multi-cluster/build.gradle
#	x-pack/plugin/sql/qa/jdbc/build.gradle
#	x-pack/plugin/transform/qa/multi-cluster-tests-with-security/build.gradle

* Fix merge

* [Build] Fix fips testing after buildparams rework (#116934)

* More Cleanup

* [Build] Fix checkstyle exclusions on windows (#115185)

* More merge fixes

* Delete x-pack/plugin/kql/build.gradle
2024-11-27 12:34:32 +01:00
Rene Groeschke
e50a2670e1
[Build] Reapply updating to Gradle 8.11.1 (#117394) (#117429)
This reverts commit b5c6d927c1.
* Simplify and fix dynamic project dependency handling

(cherry picked from commit d2b3dc51d1)
2024-11-25 02:29:31 +11:00
Rene Groeschke
6192818528
Replace cloud-ess docker image with wolfi-ess (#114413) (#114684)
* Replace cloud-ess docker image with wolfi-ess
   We just replaced the existing implementation of cloud-ess with what was wolfi-ess which is a wolfi based ess image.
   The cloud image itself will be removed in a future commit it was not used anywhere

* Switch to test cloud docker image instead of default docker in packaging pr tests.
  This adds way more coverage than the default docker image which is also barely touched

(cherry picked from commit f1f5ee06a3)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java
2024-10-14 16:07:11 +02:00
Rene Groeschke
45f57f11a2
[8.x] Update wolfi image and fix breaking change (#114390) (#114489)
* Update wolfi image and fix breaking change (#114390)

(cherry picked from commit 190513dce7)

# Conflicts:
#	build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java

* Fix spotlight
2024-10-11 21:08:33 +11:00
Rene Groeschke
54ecb2c666
Add wolfi ess docker image (#113810) (#114111)
(cherry picked from commit 54c83d7fa7)
2024-10-04 13:44:42 +02:00
Chris Hegarty
27fe7df700 Add JIT compiler directives to workaround performance regression in memory segment access in JDK 23 (#113817)
This commit adds a couple of JIT compiler directives to avoid a performance pitfall in JDK 23.

Ultimately this is a workaround for a JDK 23 bug, which has been reported and will be fixed in a future version of the JDK.

The nested rally track uncovered the JDK performance regression. Running JDK 23 with the compiler directives in this PR restores performance, and in fact improves it in several cases.
2024-10-01 14:11:18 +01:00
Brian Seeders
c0be2e2c1b
Fix packaging tests after addition of new wolfi-based image (#112831) (#113715) 2024-09-27 16:01:14 -04:00
Simon Cooper
31d5967d35
Remove use of SPI locale for JDK 23+ (#113182)
On JDK 23 we're just going with what CLDR specifies for week-date calculations - the built-in locales are available for ISO weekdate uses.
2024-09-20 16:48:17 +01:00
Tim Brooks
69c4a4f7e9
Backport incremental bulk execution (#113215)
This commit back ports all of the work introduced in:
https://github.com/elastic/elasticsearch/pull/113044

* #111438 - 5e1f655
* #111865 - 478baf1
* #112179 - 1b77421
* #112227 - cbcbc34
* #112267 - c00768a
* #112154 - a03fb12
* #112479 - 95b42a7
* #112608 - ce2d648
* #112629 - 0d55dc6
* #112767 - 2dbbd7d
* #112724 - 58e3a39
* dce8a0b
* #112974 - 92daeeb
* 529d349
* #113161 - e3424bd
2024-09-19 21:35:18 -06:00
Ryan Ernst
ec1785b3f7
Simplify java version checker (#112955) (#112960)
* Simplify java version checker (#112955)

The version checker ensures custom JDKs used to run Elasticsearch meet
the minimum Java version. It uses a multi-release jar to pass through
the "ok" versions, falling back to a java 8 version to print an error.
However, a multi-release jar is not necessary, we can instead do simple
string version parsing for the check, special casing Java 1.8.

* adjust for java 17
2024-09-19 06:29:43 +10:00
Joe Gallo
2cf435ac34
There's no need to BufferedInputStream within a GZIPInputStream (#113052) (#113068) 2024-09-19 01:00:42 +10:00
Mark Vieira
0279c0a909
Add AGPLv3 as a supported license 2024-09-13 14:30:33 -07:00
Brian Seeders
5c9710caf5
Fix wolfi docker tasks (#112721) 2024-09-10 17:39:53 -04:00
Rene Groeschke
4dee614707
Add chainguard docker image (#112103) 2024-09-06 10:32:42 -07:00
Simon Cooper
a36d90cf34
Use CLDR locale provider on JDK 23+ (#110222)
JDK 23 removes the COMPAT locale provider, leaving CLDR as the only option. This commit configures Elasticsearch
to use the CLDR provider when on JDK 23, but still use the existing COMPAT provider when on JDK 22 and below.

This causes some differences in locale behaviour; this also adapts various tests to still work whether run on COMPAT or CLDR.
2024-09-04 13:42:40 +01:00
Ryan Ernst
84bd31eb8e
Fix shutdown race condition in server start (#112300)
When the server CLI is preparing to start Elasticsearch, it may receive
a SIGTERM. That signal causes the close method of the CLI to be invoked,
which checks for a non-null server process, and stops it if it exists.
Since shutdown occurs in a different thread, it is possible close is
invoked, sees the server process is null, so skips calling close, but
then before close finishes the server process is started.

Normally the above case is ok; the close method will cause Java to exit,
and the child process will be sent a SIGTERM. However, in the case of
serverless, close is wrapped so that the CLI can wait on uploading
possible heapdumps. This presents the possibility that close thinks the
child process has been stopped, but it hasn't, so the wrapped close
waits indefinitely on the child process that was started up during the
race.

This commit guards against the startup race by locking the critical
sections that create the child process and stop it. Either the child
process is created first, and will then be stopped immediately, or close
is invoked first, marking that the CLI is shutting down, and creating
the child process will be skipped.
2024-08-29 10:20:43 -07:00
Victor Martinez
e7c0ba953a
Update IronBank docker image base to ubi:9.4 (#111743) 2024-08-28 19:18:58 +02:00
Ryan Ernst
0aa4758f02
Stop setting java.library.path (#112119)
Native libraries in Java are loaded by calling System.loadLibrary. This
method inspects paths in the java.library.path to find the requested
library. Elasticsearch previously used this to find libsystemd, but now
the only remaining use is to set the additional platform directory in
which Elasticsearch keeps its own native libraries.

One issue with setting java.library.path is that its not set for the cli
process, which makes loading the native library infrastructure from clis
difficult. This commit reworks how Elasticsearch native libraries are
found in order to avoid needing to set java.library.path. There are two
cases. The simplest is production, where the working directory is the
Elasticsearch installation directory, so the platform specific directory
can be constructed. The second case is for tests where we don't have an
installtion. We already pass in java.library.path there, so this change
renames the system property to be a test specific property that the new
loading infrastructure looks for.
2024-08-23 11:16:18 -07:00
Ryan Ernst
8417d3f141
Move preallocate functionality to native access (#110678)
This commit moves the file preallocation functionality into
NativeAccess. The code is basically the same. One small tweak is that
instead of breaking Java access boundaries in order to get an open file
handle, the new code uses posix open directly.

relates #104876
2024-07-11 09:42:44 -07:00
Patrick Doyle
43b2e877e0
Revert "Move PluginsService to its own internal package (#109872)" (#109946)
This reverts commit b9e7965184.
2024-06-19 18:10:50 -04:00
Patrick Doyle
b9e7965184
Move PluginsService to its own internal package (#109872)
* Mechanical package change in IntelliJ
* A couple of manual fixups
* Export plugins.loading to deprecation
* Put plugin-cli in a module so can export PluginsUtils to it.
2024-06-19 15:23:47 -04:00
Benjamin Trent
29288d6590 Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-11 06:54:23 -04:00
Ry Biesemeyer
f75afb0098
Update year in NOTICE.txt (#109548) 2024-06-10 15:12:38 -07:00
Benjamin Trent
ac53d6020b Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-05 12:38:23 -04:00
Rene Groeschke
7226100458
Make :distribution:docker:check Gradle configuration cache compatible (#109083)
fix cache compatibility issue in TestFixturesPlugin and support
yamlRestTest in `:distribution:docker:yamlRestTest`
2024-06-05 19:23:04 +10:00
Benjamin Trent
cf84416fc5 Merge remote-tracking branch 'upstream/main' into lucene_snapshot_9_11 2024-06-04 12:50:52 -04:00
Rene Groeschke
8ac3e3dd90
Update Gradle wrapper to 8.8 (#108021)
Fix incompatibility with 8.8 and our internal api usages

- Update ospackage to a version that contains a fix we provided
- Tweak build logic to avoid deprecation warnings
- Use newer permission api
- Use custom shadowplugin
- Rework ElasticsearchDistribution dependencies resolution
- Update Gradle wrapper to 8.8
2024-06-04 12:43:02 +02:00
elasticsearchmachine
83f51b477e Merge remote-tracking branch 'origin/main' into lucene_snapshot 2024-05-28 10:01:58 +00:00
Chris Hegarty
6b62c5129d
Replace use of reflection with MemorySegmentAccessInput (#109061)
This commit replaces the use of reflection with the newly added MemorySegmentAccessInput.
2024-05-27 17:49:49 +01:00
Rene Groeschke
2704d3a8d1
Remove cross project support in TestFixturesPlugin (#109077)
- One step closer to configuration cache support
- Crossproject support has been replaced by using testcontainer based fixtures
2024-05-27 17:59:14 +02:00
elasticsearchmachine
7bd291d726 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2024-05-21 10:01:52 +00:00
Ryan Ernst
9e6fe11d19
Update ASM to 9.7 for plugin scanner (#108822)
This commit updates the ASM library in order to support class files
written with Java 23.

closes #108776
2024-05-20 11:56:23 -04:00
elasticsearchmachine
0ce5dadc6b Merge remote-tracking branch 'origin/main' into lucene_snapshot 2024-05-15 10:02:23 +00:00
Ryan Ernst
db70623442
Fix G1 JDK bug workaround (#108641)
The workaround requires to JDK args, but SystemJvmOptions actually
operates on individual JDK args. This commit adjusts SystemJvmOptions to
allow adding sets of jdk args together.
2024-05-14 19:42:35 -04:00
Ryan Ernst
b2bc951ac5
Workaround G1 bug for JDK 22 and 22.0.1 (#108571)
See https://bugs.openjdk.org/browse/JDK-8329528. The applied workaround
was suggested on the linked issue, and was tested and confirmed to avoid
the G1 bug.
2024-05-14 06:17:14 -07:00
elasticsearchmachine
46a9663957 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2024-05-14 10:01:36 +00:00
Lorenzo Dematté
89d666bf90
Adding override for lintian false positive on libvec.so (#108521) 2024-05-13 19:22:12 +02:00
elasticsearchmachine
1d9cbbcded Merge remote-tracking branch 'origin/main' into lucene_snapshot 2024-05-08 10:01:34 +00:00
Moritz Mack
4c263e2ab4
CLI launcher / terminal improvements (#106470)
This removes getWriter() and getErrorWriter() from Terminal in order to prepare for using log4j in server-cli.
Additionally, ensure IO failures on the pump thread are handled once completed (closed) and
catch throwables in Command#main to print them (formatted) via Terminal.
2024-05-07 16:09:52 +02:00
elasticsearchmachine
6f46ee51c8 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2024-05-03 10:01:31 +00:00
Jake Landis
6d20cef931
Bump Tika dependencies to 2.9.2 (#108144)
This commit bumps Tika to 2.9.2 and manually bumps the transitive versions 
to match 2.9.2's parent POM. This commit also centralizes the dependency 
versions so that you only need to look at 1 list to see the full set of dependencies 
to manually check.
2024-05-02 10:19:31 -05:00