The no-jdk distributions exist in 7.x and before. They were removed with
8.0. This commit removes the remaining deprecation messages for using
the no-jdk distribution. Note that when talking with an older node, we
drop the bundledJdk attribute. This is ok because it is only possible
for this to not be true when talking with a 7.17 node, during an upgrade,
and the usingBundledJdk is retained, which is the important thing if
debugging a problem.
relates #76896
relates #85758
The java version checker tests whether the current jdk is a sufficient
version to run Elasticsearch. Since it is compatible with java 7, it
must parse Java's version sysprop.
This commit reworks the checker to no longer need to parse the version.
Instead, it uses a multi-release-jar, so that there are two versions of
the main class. The Java 7 compatible version simply exits with an error
message. The Java 17 version simple exits with no error.
relates #85758
When adding aarch64 support to Elasticsearch, a bug existed in the jdk
which caused an error message to be printed on every invocation of Java
due to page size misalignment (see
https://bugs.openjdk.java.net/browse/JDK-8236847). This bug was fixed in
Java 17. Since the main branch always uses Java 17, the workaround is no
longer necessary.
relates #85758
The java version checker is intended to ensure the version of the JDK
that Elasticsearch will use is at least the version Elasticsearch was
compiled against. It prevents cryptic class version errors upon loading.
The bundled jdk is always of the appropriate version.
This commit moves the version check to only occur when a custom
ES_JAVA_HOME is used. If a user were to swap out the bundled jdk with
their own, it could result in the cryptic errors being protected
against, but replacing the bundled jdk is not something we need nice
error messages for.
relates #85758
This commit removes the one conditional jvm option that was dependent on
the Java version priority 17, which was whether to add
`-XX:+ShowCodeDetailsInExceptionMessages`. This option is available
since Java 14, and actually the default since Java 15, so the flag is simply
not needed anymore.
relates #85758
Since 8.0.0, we only support Java 17 and up. This change cleans up unsupported Java versions from jvm.options and updates documentation to reference supported Java versions, in case it is confusing to users.
Deprecation log indexing can make test very fragile
and randomly failing during cluster cleanup.
This commit disables indexing of deprecation logs for :distribution:docker tests
closes#79803
This removes the overhead of zipping up modules that are immediately unzipped again when packaging
the elasticsearch distribution.
We also move some logic for packaging the elasticsearch distribution into a plugin and remove some outdated
overhead dealing with 'meta plugins' when copying modules into a distribution.
Another follow up and related optimization out of scope of this PR is, to also not zip unzip modules declared
for usage in our test cluster setups.
This partially addresses #76726.
Plugin downloads can sometimes take unexpectedly
long because of slow random number generation on certain
operating system versions, e.g. Debian based OS
distros with kernels older than 5.6. This PR adds
a warning message for users explaining why plugin
installation from remote URLs is taking unexpectedly long.
This commit adds a jar separate from the test framework to provide
utilities for testing x-content related code. The first thing moved
there is the base schema validation test case, which also pulls along
the com.networknt dependency and jackson. For now these are direct
dependencies, though we could consider shading them in the future so as
not to expose downstream projects to them, which may have version
conflicts.
This change isolates the Jackson implementation of x-content parsers and generators to a separate classloader. The code is loaded dynamically upon accessing any x-content functionality.
The x-content implementation is embedded inside the x-content jar, as a hidden set of resource files. These are loaded through a special classloader created to initialize the XContentProvider through service loader. One caveat to this approach is that IDEs will no longer trigger building the x-content implementation when it changes. However, running any test from the command line, or running a full Build in IntelliJ will trigger the directory to be built.
Co-authored-by: ChrisHegarty <christopher.hegarty@elastic.co>
Replace any old references of master with main. These are mostly
cosmetic, as the name of the default branch doesn't actually affect the
build in any meaningful way.
The tools used in launching Elasticsearch have a separate jar file and
dependencies. This commit moves the launcher tools to its own lib
directory, so that it does not bleed into the main Elasticsearch
classpath.
Closes#82433. If the environment variable `RESTART_ON_UPGRADE` is true,
then ensure that we delay restarting Elasticseach until after the
keystore is upgraded, or else we can run into permissions problems.
This flag has already been removed by ES on Oct 21, 2019 after they created their own allocator: 547e399dbf
However, in parallel, on Oct 28, 2019 all command line flags were moved from jvm.options file into the launcher, and this option has been re-introduced: c4fbda3310
See original commit from Oct 21 for the reasoning on why this flag is not needed anymore.
Lucene issues that resulted in elasticsearch changes:
LUCENE-9820 Separate logic for reading the BKD index from logic to intersecting it.
LUCENE-10377: Replace 'sortPos' with 'enableSkipping' in SortField.getComparator()
LUCENE-10301: make the test-framework a proper module by moving all test
classes to org.apache.lucene.tests
LUCENE-10300: rewrite how resources are read in ukrainian morfologik analyzer:
LUCENE-10054 Make HnswGraph hierarchical
2.4.0 Introduces support for linux and macOS aarch64. Without this
we cannot detect whether a terminal is attached to elasticsearch
on macoOS/linux aarch64 and so we do not generate and print the
elastic password and enrollment token on startup
In order to display formatted (bolded) autoconfiguration text on
Windows cmd prompt using JANSI, one needs to invoke the
SetConsoleMode Kernel32 API, via JANSI's AnsiConsole#install.
But we lack the testing infra to properly assert the formatting behavior,
so this PR disables ANSI-formatted output on Windows cmd prompt.
Fixes: #83316
This PR slightly improves the format of the security auto-configuration
information that is printed on the terminal when the initial cluster
node first starts up.
It uses eye-catching Unicode characters for bullet points.
It also uses Unicode to display a continuous border, for the whole
width of the terminal, before and after the information. In addition,
it uses ANSI escape sequences to render some of the information
in bold fonts.
It will fallback to using regular characters if the JVM is set-up with
a non-UTF encoding for the standard out.
Try to represent immutable data with Java records introduced in JEP 395
Convert only existing immutable classes, no "POJO with setters to a record" refactorings.
Fixes#81999.
Change `EnrollmentProcessTests` and the testing infrastructure so
that `ServerUtils` can send requests to the correct local port.
This is also a step towards addressing #79688.
Also change the Iron Bank build to that it works on aarch64. This
isn't currently a supported configuration, but it's useful for
testing.
Closes#81652.
Convert the `repository-azure`, `repository-gcs` and `repository-s3`
plugins into modules, so that they are always included in the
Elasticsearch distribution. Also change plugin installation, removal
and syncing so that attempting to add or remove these plugins still
succeeds but is now a no-op.
JEP 361[https://openjdk.java.net/jeps/361] added support for switch expressions
which can be much more terse and less error-prone than switch statements.
Another useful feature of switch expressions is exhaustiveness: we can make
sure that an enum switch expression covers all the cases at compile time.
We (mostly I) were initially advocating for the auto-generated files to
use unique names (the name containing a timestamp particle), in order to
avoid that subsequent invocations of the config step conflict with
itself. Moreover, I was wishing that these files will not have to be
handled directly by admins (that the enrollment process was to be used).
However, experience proved us otherwise, admins have to manipulate these
files, and unique configuration names are hard to deal with in scripts
and docs, so this PR is all about using a fixed name for all the
generated files. _Labeling as a bug fix because the feedback is that it
very negatively impacts usabilty._ Closes
https://github.com/elastic/elasticsearch/issues/81057
For some reason, the way we generate `/app/elasticsearch.sh` for Cloud
images has broken. It may be the result of changing the default shell to
`bash`. Fix by adding `-e` to the `echo` command.
As a result of changing the base Docker to Ubuntu in #80820, the default shell
i.e. `/bin/sh` changed to `dash`, rather than `bash`, which could impact anyone
invoking `/bin/sh` and expecting it to still propagate environment variables with
periods in their names.
Reconfigure the default shell back to `bash` so that this type of situation works
again.
The ES code base is quite JSON heavy. It uses a lot of multi-line JSON requests in tests which need to be escaped and concatenated which in turn makes them hard to read. Let's try to leverage Java 15 text blocks for representing them.
Apply suggestions from Docker Inc about how to update the `cacerts` in
our Ubuntu-based Docker image. Instead of copying around files and
symlinking, instead install `ca-certificates` and `p11-kit`, and use the
latter to regenerate Java's `cacerts`, as well as ensuring it is
regenerated if the system ca certs are updated.
Allow users to pass an ENROLLMENT_TOKEN environment variable that
would be passed as the `--enrollment-token` parameter to the
elasticsearch executable running in the container. This allows an
elasticsearch node running in docker to enroll itself to an
existing secured cluster.
Resolves: #81068
Closes#81208. Elasticsearch uses zlib for two purposes: *
Compression of stored fields with `index.codec: best_compression`,
which we use for observability and security data. * Request /
response compression. Historically, zlib was packaged within the JDK, so
that users wouldn't have to have zlib installed for basic usage of Java.
However, the original zlib optimizes for portability and misses a number
of important optimizations such as leveraging vectorization support for
x86 and ARM architectures. Several forks have been created in order to
address this. Since version 9, the JDK uses the system's zlib when
available and falls back to the zlib that is packaged within the JDK if
a system zlib cannot be found. This commit changes the Docker image to
install the Cloudflare fork of zlib, and run Java using the fork instead
of the original zlib, so that users of the Docker image can get better
performance. Other ES distribution types are out-of-scope, since
configuring the JVM to use an alternative zlib requires an environment
config as well as installed another zlib, and Docker is the only
distribution type where we can control both.
Part of #81150. Sometimes, building the Cloud Docker images fails due to
some issue with the `.tar.gz` files. Add extra checks and error messages
so that we have a better idea what the precise problem is.
Fix the split package org.elasticsearch.cli, between server and the cli library. Move the server org.elasticsearch.cli package to org.elasticsearch.common.cli. Removing split packages is a prerequisite to modularization.
Post JDK 17 the security manager is disabled by default - setSecurityManager throws UOE - see JEP 411.
This change adds a command line option to explicitly allow to set the security manager, which enables early testing with releases greater than JDK 17.
Closes#70219.
Introduce a declarative way for the Elasticsearch server to manage plugins,
which reads the `elasticsearch-plugins.yml` file and works which out
plugins need to be added and / or removed to match the configuration. Also
make it possible to configure a proxy in the config file, instead of
through the environment.
Most of the work of adding and removing is still done in the
`InstallPluginAction` and `RemovePluginAction` classes, so the
behaviour should be the same as with the `install` and `remove`
commands. However, these commands will now abort if the above config
file exists. The intent is to make it harder for the configuration
to drift.
This new method only applies to `docker` distribution types at the
moment.
Since this syncing mechanism declarative, rather than imperative,
the Cloud-specific plugin wrapper script is no longer required.
Instead, an environment variable informs `InstallPluginAction` to
install plugins from an archive directory instead of downloading
them, where possible.
