This commit fix the IronBank hardening_manifest CI test that expect the
hardening_manifest.yaml values to match `'^(?!\\s*$)[ -~|]*$'` regex:
```
| hardening_manifest | ERROR | Hardening Manifest failed jsonschema validation
| hardening_manifest | ERROR | Verify Hardening Manifest content
| hardening_manifest | ERROR | 'Rene Gröschke' does not match '^(?!\\s*$)[ -~|]*$'
```
43219286
cc @breskeby
<!-- Thank you for your interest in and contributing to Elasticsearch!
There are a few simple things to check before submitting your pull
request that can help with the review process. You should delete these
items from your submission, but they are here to help bring them to your
attention. -->
- Have you signed the [contributor license agreement](https://www.elastic.co/contributor-agreement)?
- Have you followed the [contributor guidelines](https://github.com/elastic/elasticsearch/blob/main/CONTRIBUTING.md)?
- If submitting code, have you built your formula locally prior to submission with `gradle check`?
- If submitting code, is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed.
- If submitting code, have you checked that your submission is for an [OS and architecture that we support](https://www.elastic.co/support/matrix#show_os)?
- If you are submitting this code for a class then read our [policy](https://github.com/elastic/elasticsearch/blob/main/CONTRIBUTING.md#contributing-as-part-of-a-class) for that.
(cherry picked from commit e5cf597782)
# Conflicts:
# distribution/docker/src/docker/iron_bank/hardening_manifest.yaml
Co-authored-by: Julien Mailleret <8582351+jmlrt@users.noreply.github.com>
* Fix Gradle Deprecation warning as declaring an is- property with a Boolean type has been deprecated.
* Make use of new layout.settingsFolder api to address some cross project references
* Fix buildParams snapshot check for multiprojet projects
(cherry picked from commit e19b2264af)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/BaseInternalPluginBuildPlugin.java
# docs/build.gradle
# qa/entitlements/build.gradle
# x-pack/qa/multi-project/core-rest-tests-with-multiple-projects/build.gradle
# x-pack/qa/multi-project/xpack-rest-tests-with-multiple-projects/build.gradle
This updates the gradle wrapper to 8.12
We addressed deprecation warnings due to the update that includes:
- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions
(cherry picked from commit ba61f8c7f7)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerUbiElasticsearchDistributionType.java
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/Fixture.java
# plugins/repository-hdfs/hadoop-client-api/build.gradle
# qa/entitlements/build.gradle
# server/src/main/java/org/elasticsearch/indices/IndicesFeatures.java
# x-pack/plugin/migrate/build.gradle
# x-pack/plugin/security/qa/security-basic/build.gradle
* Add initial entitlement policy parsing (#114448)
This change adds entitlement policy parsing with the following design:
* YAML file for readability and re-use of our x-content parsers
* hierarchical structure to group entitlements under a single scope
* no general entitlements without a scope or for the entire project
* Avoid double instrumentation via class annotation (#115398)
* Move entitlement jars to libs (#115883)
The distribution tools are meant to be CLIs. This commit moves the
entitlements jar projects to the libs dir, under a single
libs/entitlement root directory to keep the related jars together.
* Entitlement tools: SecurityManager scanner (#116020)
* Dynamic entitlement agent (#116125)
* Refactor: treat "maybe" JVM options uniformly
* WIP
* Get entitlement running with bridge all the way through, with qualified
exports
* Cosmetic changes to SystemJvmOptions
* Disable entitlements by default
* Bridge module comments
* Fixup forbidden APIs
* spotless
* Rename EntitlementChecker
* Fixup InstrumenterTests
* exclude recursive dep
* Fix some compliance stuff
* Rename asm-provider
* Stop using bridge in InstrumenterTests
* Generalize readme for asm-provider
* InstrumenterTests doesn't need EntitlementCheckerHandle
* Better javadoc
* Call parseBoolean
* Add entitlement to internal module list
* Docs as requested by Lorenzo
* Changes from Jack
* Rename ElasticsearchEntitlementChecker
* Remove logging javadoc
* exportInitializationToAgent should reference EntitlementInitialization, not EntitlementBootstrap.
They're currently in the same module, but if that ever changes, this code would have become wrong.
* Some suggestions from Mark
---------
Co-authored-by: Ryan Ernst <ryan@iernst.net>
* Remove unused EntitlementInternals (#116473)
* Revert "Entitlement tools: SecurityManager scanner (#116020)"
This reverts commit 023fb663de.
---------
Co-authored-by: Jack Conradson <osjdconrad@gmail.com>
Co-authored-by: Lorenzo Dematté <lorenzo.dematte@elastic.co>
Co-authored-by: Ryan Ernst <ryan@iernst.net>
* [TEST] Migrated REST tests extending JsonLogsIntegTestCase (#115188)
REST tests extending JsonLogsIntegTestCase migrated to the new REST
testing framework, using 'elasticsearch.internal-java-rest-test' Gradle
plugin
Explicit handling of test single-cluster config by checking if cluster
is explicitly configured as single-node to avoid an incorrect configuration
which may cause split brain.
Replaced `ElasticsearchJavaPlugin` with `ElasticsearchJavaBasePlugin` in
`RestTestBasePlugin` for better granularity.
Additionally, updated the `DistributionDownloadPlugin` to set configurations as
non-consumable and the `InternalDistributionArchiveSetupPlugin` to mark
certain configurations as consumable. This ensures that configurations
are correctly utilized during build setup and execution phases.
(cherry picked from commit 130cc74d8a)
* Fixed DefaultSettingsProvider to use Java 17 (#116029)
It is needed to be able to backport the change to the 8.x.
Having same code for both versions can help us to avoid some merge
conflicts in the future.
* Use directory name as project name for libs (#115720)
The libs projects are configured to all begin with `elasticsearch-`.
While this is desireable for the artifacts to contain this consistent
prefix, it means the project names don't match up with their
directories. Additionally, it creates complexities for subproject naming
that must be manually adjusted.
This commit adjusts the project names for those under libs to be their
directory names. The resulting artifacts for these libs are kept the
same, all beginning with `elasticsearch-`.
* fixes
* Replace cloud-ess docker image with wolfi-ess
We just replaced the existing implementation of cloud-ess with what was wolfi-ess which is a wolfi based ess image.
The cloud image itself will be removed in a future commit it was not used anywhere
* Switch to test cloud docker image instead of default docker in packaging pr tests.
This adds way more coverage than the default docker image which is also barely touched
(cherry picked from commit f1f5ee06a3)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java
This commit adds a couple of JIT compiler directives to avoid a performance pitfall in JDK 23.
Ultimately this is a workaround for a JDK 23 bug, which has been reported and will be fixed in a future version of the JDK.
The nested rally track uncovered the JDK performance regression. Running JDK 23 with the compiler directives in this PR restores performance, and in fact improves it in several cases.
* Simplify java version checker (#112955)
The version checker ensures custom JDKs used to run Elasticsearch meet
the minimum Java version. It uses a multi-release jar to pass through
the "ok" versions, falling back to a java 8 version to print an error.
However, a multi-release jar is not necessary, we can instead do simple
string version parsing for the check, special casing Java 1.8.
* adjust for java 17
JDK 23 removes the COMPAT locale provider, leaving CLDR as the only option. This commit configures Elasticsearch
to use the CLDR provider when on JDK 23, but still use the existing COMPAT provider when on JDK 22 and below.
This causes some differences in locale behaviour; this also adapts various tests to still work whether run on COMPAT or CLDR.
When the server CLI is preparing to start Elasticsearch, it may receive
a SIGTERM. That signal causes the close method of the CLI to be invoked,
which checks for a non-null server process, and stops it if it exists.
Since shutdown occurs in a different thread, it is possible close is
invoked, sees the server process is null, so skips calling close, but
then before close finishes the server process is started.
Normally the above case is ok; the close method will cause Java to exit,
and the child process will be sent a SIGTERM. However, in the case of
serverless, close is wrapped so that the CLI can wait on uploading
possible heapdumps. This presents the possibility that close thinks the
child process has been stopped, but it hasn't, so the wrapped close
waits indefinitely on the child process that was started up during the
race.
This commit guards against the startup race by locking the critical
sections that create the child process and stop it. Either the child
process is created first, and will then be stopped immediately, or close
is invoked first, marking that the CLI is shutting down, and creating
the child process will be skipped.
Native libraries in Java are loaded by calling System.loadLibrary. This
method inspects paths in the java.library.path to find the requested
library. Elasticsearch previously used this to find libsystemd, but now
the only remaining use is to set the additional platform directory in
which Elasticsearch keeps its own native libraries.
One issue with setting java.library.path is that its not set for the cli
process, which makes loading the native library infrastructure from clis
difficult. This commit reworks how Elasticsearch native libraries are
found in order to avoid needing to set java.library.path. There are two
cases. The simplest is production, where the working directory is the
Elasticsearch installation directory, so the platform specific directory
can be constructed. The second case is for tests where we don't have an
installtion. We already pass in java.library.path there, so this change
renames the system property to be a test specific property that the new
loading infrastructure looks for.
This commit moves the file preallocation functionality into
NativeAccess. The code is basically the same. One small tweak is that
instead of breaking Java access boundaries in order to get an open file
handle, the new code uses posix open directly.
relates #104876
* Mechanical package change in IntelliJ
* A couple of manual fixups
* Export plugins.loading to deprecation
* Put plugin-cli in a module so can export PluginsUtils to it.
Fix incompatibility with 8.8 and our internal api usages
- Update ospackage to a version that contains a fix we provided
- Tweak build logic to avoid deprecation warnings
- Use newer permission api
- Use custom shadowplugin
- Rework ElasticsearchDistribution dependencies resolution
- Update Gradle wrapper to 8.8
