* Make YamlRestTest and ElasticDistributionPlugin cc compatible
These changes are addressing configuration cache incompatibilities of running
rest tests with the new ClassRule based Elasticsearch test cluster setup.
This allows running those tests with configuration cache enabled:
`./gradlew :modules:ingest-user-info:yamlRestTest --configuration-cache`
node.processors determines the size of Elasticsearch threadpools. This
commit sets the JDK flag -XX:ActiveProcessorCount when node.processors
is set so that the JDK similarly sizes its threadpools accordingly.
relates #100244
* Make forbidden apis check cacheable and cc compatible
* Port CheckForbiddenApiTask to use worker api
* Simplify runtime classpath for CheckForbiddenApisTask
This commit forbids the use of Thread constructors that do not take a
name. In general nameless threads are more difficult to understand their
purpose when debugging. Note that this is only added to production
signatures. Tests are not forbidden here so as not to be pedantic (or
require a larger change since many tests create anonymous threads).
relates #101628
This commit makes the apm log a little more usable. First, it makes the
path to the log explicit by passing the ES logs dir through, instead of
relying on the location of the apm jar file. Second, it tweaks the log
level to be warn, not error. Third, it switches the apm log file to be
json, which is more easily processable.
The build version is made up of a few parts in non-release builds. Both
the snapshot and pre-release qualifiers are appended to it. These
qualifiers used to be part of Version, but in 7.0 the qualifiers were
made to be found only in the build info. The Build class retains these
qualifiers through the compile ES version extracted from the server jar
at runtime.
Build.qualifiedVersion() is suppose to provide the fully qualified
version, including snapshot and pre-release qualifiers. Yet
Build.version() also includes this information; there is no distinction
since the qualifier was moved to be only in the build info.
This commit separates the pre-release qualifier from the version. It
maintains bwc in talking to older nodes, passing the fully qualified
version there, but in current nodes splits out the pre-release qualifier
into a new member of Build.
The Plugin CLI can generally treat strings opaquely. We had some logic
comparing earlier versions, but what we really care about with most of our
plugins is whether or not they were built with the current version of
Elasticsearch, not whether they were built before or after. (This question
will be trickier with stable plugins, but none of that code is in the CLI.)
The CLI classes can be cleaned up even more once Version is removed from
PluginDescriptor. Some of the tests can't use opaque strings for versions
until PluginDescriptor can handle them.
* Remove Version from Install and List plugin actions
* Remove Version from SyncPluginsAction
Another round of automated fixes to this, marking things that can be
made static as static. Saves some JIT cycles but also turns some lambdas
from capturing to non-capturing and makes the "utilityness" of some
classes visible.
There is a bug in the nebula os package gradle plugin that breaks copy spec specific setgid handling.
We have created a patch for the plugin that we use for now to unblock us. This will be ported upstream
to the nebula main branch and part of a release but that requires some more polishing and will
be taken care of in a later PR
The latest version contains a fix to allow sending metrics to APM server. also adds a apm agent jvm options
"enable_experimental_instrumentations", "true"
which is required to enable the otel-metrics-instrumentation.
relates https://github.com/elastic/elasticsearch/pull/99832
* Replaced some occurrences of Version.CURRENT toString
* Added unit tests for the 2 different warn header patterns in HeaderWarnings
* Comment to clarify non-semantic versions + revert change on Version.major message
Gradle now fully supports compiling, testing and running on Java 20.
Among other general performance improvements this release introduces --test-dry-run command line option that allows checking if tests are filtered or not by gradle.
Required updating nebula ospackage plugin as setuid was broken in gradle 8.3.
This commit updates the version range in jvm.options for the Panama Vector API. The range is updated from just a single value 20, to a range starting from 20 with no upper bound. This effectively enables the Panama Vector API for use in Lucene on all JDK versions greater than 19.
with the support of metrics the TracerPlugin name is no longer adequate. Renaming this to TelemetryPlugin.
Also introducing TelemetryProvider interface. While it is only used in Node.java at the moment to fetch Tracer instance, it is intended to be used in Plugin::createComponents (to be done in separate commit due to
the broad scope of this method)
This will allow for plugins to get access to both Tracer and Metric interfaces
without the need to add yet another argument to createComponents
Also adding internal subpackage in module/apm so that it is more obvious
which packages are not exported
Elasticsearch use lib as its module path, which includes all the
dependencies of server. However, there are sometimes additional modules
such as preallocate that are used dynamically and not direct
dependencies of server. This commit switches the addition of these
modules to be automatic by including all modules present in the module
path.
In order for build info to be pluggable for serverless, the current
build needs to be lazily determined. This commit moves the CURRENT
constant to a static method.
relates #96861
ES logging has to be explicitly initialised with a call to LoggerFactory.setInstance This is normally done as LogConfigurator.configure(Environment,boolean) is calling this as part of initPhase1
However CLI tools were not using that method. Cli tools are using LogConfigurator.configureWithoutConfig and that method was not setting up ES logging
This commit modifies configureWithoutConfig to also configure esLogging It also adds a LogConfigurator.configureESLogging call explicitly in CliToolLauncher This allows to build a new cli-launcher.jar and replace it in previous 8.7-8.9 clusters
closes#97350
Lots of spots where we did weird things around streams like redundant stream creation, redundant collecting
before adding all the collected elements to another collection or so, redundant streams for joining strings
and using less efficient `Collectors.toList` and in a few cases also incorrectly relying on the result being mutable.
This commit updates the plugin cli and scanner components to use ASM 9.5.
The update is required to successfully test with JDK 21. Tests in this component programatically run the java source compiler, which generates class files with major version 65, then tries to parse those generated class files. Without this change the tests fail with java.lang.IllegalArgumentException: Unsupported class file major version 65.
Bump the minimum targetCompatibility of java version checker to 1.8 ( from 1.7 ).
JDK 20 has removed support for source and target 1.7 in javac [1], so to successfully build with JDK 20 (through JAVA_TOOLCHAIN_HOME), we can no longer specify targetCompatibility = JavaVersion.VERSION_1_7, for the java version checker.
The idea of the java version checker is to successfully run with "older" JDK's, but I think that 1.8 is presently more than sufficient. While we still don't "support" compiling with JDK 20, it is good to keep the project up to date, so we can more easily experiment with recent JDK's.
This change adds the jdk.incubator.vector module, so that we can enable the Panamaized vector utils in Lucene.
The module is added by default if running on JDK 20, which is the only current supported implementation in Lucene, but JDK 21 will likely come soon.
If for some reason this needs to be disable, just remove or otherwise comment out the --add-modules=jdk.incubator.vector line in the jvm.options.
The log output from Lucene shows the preferred vector bit width that is in operation.
The preallocate module needs access to java.io internals. However, in
order to open java.io to a specific module, rather than the unnamed
module as was previously done, the said module must be in the boot
layer.
This commit moves the preallocate module to libs. It adds it to the main
lib dir, though it does not add it as a compile dependency of server.
Fixes#82794. Upgrade the spotless plugin, which addresses the issue
around formatting `instanceof` expressions. Formatting of statements
including lambdas seems to have improved too.
- Remove custom checksum build logic in wrapper task
- Adjust jdk home handling adjusting the change in behaviour in gradle. Requires providing canonical paths for provisioned jdk homes.
- Fix test by add workaround to bug in configuration cache
It's pretty common to run a block of code in a `try ... catch` block
that just passes exceptions off to a listener's `onFailure` method. This
commit adds a small utility to encapsulate this, enabling some
one-liners.
`JvmErgonomics` requires various JVM options to be present, but if they
are omitted then we throw a `NullPointerException` which looks to the
user like an ES bug. They would have to be doing something a little odd
to get into this state, but nonetheless it is possible to hit these
NPEs. We don't need to handle such a config gracefully, but we should
clarify why Elasticsearch won't start to help the user fix their config.
