We're leaking quite a few of these parsers. That doesn't seem to be much
of a problem but results in some memory inefficiencies in Jackson here
and there. This PR bulk fixes a bunch of instances that I could easily
automatically fix. I'll open a follow-up for closing the parser on the
document parsing context which also suffers from this but is non-trivial
to fix.
* Refactoring: move command line and environment build to a separate class
* Refactoring: adding a builder
* Moving tmp dir setup and JVM option parsing outside of builder
This commit upgrades the Bouncy Castle jars. Bouncy Castle is used for
some internal build concners as well as a comnand line application.
Most notably Bouncy Castle is also used as the FIPs certified JCE/JSEE provider
we use to test our ability to use a FIPs compliant crypto provider.
The following changes here are a result of the upgraded Bouncy Castle jars:
* TLSv1.3 is now supported when running in FIPs mode
* RSA PKCS#1 v1.5 is no longer allowed in FIPS mode
* Triple DES (3DES) is no longer allowed in FIPS mode
* Minor updates the security manager configuration used to test FIPs (to read permissions from the security provider)
* Minor adjustments to tests to accommodate the above changes.
* Minor adjustments to the gradle build to accommodate new dependencies
Note - update to the documentation will come in a later commit.
This introduces a new getHistoricalFeatures() method on ESRestTestCase
which returns a map of historical feature version mappings loaded from
FeatureSpecification implementations from any plugins/modules in use
by the current test suite. The mappings are generated by a new Gradle
task at build time, and then injected into the test runtime as a
System property.
* Make YamlRestTest and ElasticDistributionPlugin cc compatible
These changes are addressing configuration cache incompatibilities of running
rest tests with the new ClassRule based Elasticsearch test cluster setup.
This allows running those tests with configuration cache enabled:
`./gradlew :modules:ingest-user-info:yamlRestTest --configuration-cache`
node.processors determines the size of Elasticsearch threadpools. This
commit sets the JDK flag -XX:ActiveProcessorCount when node.processors
is set so that the JDK similarly sizes its threadpools accordingly.
relates #100244
* Make forbidden apis check cacheable and cc compatible
* Port CheckForbiddenApiTask to use worker api
* Simplify runtime classpath for CheckForbiddenApisTask
This commit forbids the use of Thread constructors that do not take a
name. In general nameless threads are more difficult to understand their
purpose when debugging. Note that this is only added to production
signatures. Tests are not forbidden here so as not to be pedantic (or
require a larger change since many tests create anonymous threads).
relates #101628
This commit makes the apm log a little more usable. First, it makes the
path to the log explicit by passing the ES logs dir through, instead of
relying on the location of the apm jar file. Second, it tweaks the log
level to be warn, not error. Third, it switches the apm log file to be
json, which is more easily processable.
The build version is made up of a few parts in non-release builds. Both
the snapshot and pre-release qualifiers are appended to it. These
qualifiers used to be part of Version, but in 7.0 the qualifiers were
made to be found only in the build info. The Build class retains these
qualifiers through the compile ES version extracted from the server jar
at runtime.
Build.qualifiedVersion() is suppose to provide the fully qualified
version, including snapshot and pre-release qualifiers. Yet
Build.version() also includes this information; there is no distinction
since the qualifier was moved to be only in the build info.
This commit separates the pre-release qualifier from the version. It
maintains bwc in talking to older nodes, passing the fully qualified
version there, but in current nodes splits out the pre-release qualifier
into a new member of Build.
The Plugin CLI can generally treat strings opaquely. We had some logic
comparing earlier versions, but what we really care about with most of our
plugins is whether or not they were built with the current version of
Elasticsearch, not whether they were built before or after. (This question
will be trickier with stable plugins, but none of that code is in the CLI.)
The CLI classes can be cleaned up even more once Version is removed from
PluginDescriptor. Some of the tests can't use opaque strings for versions
until PluginDescriptor can handle them.
* Remove Version from Install and List plugin actions
* Remove Version from SyncPluginsAction
Another round of automated fixes to this, marking things that can be
made static as static. Saves some JIT cycles but also turns some lambdas
from capturing to non-capturing and makes the "utilityness" of some
classes visible.
There is a bug in the nebula os package gradle plugin that breaks copy spec specific setgid handling.
We have created a patch for the plugin that we use for now to unblock us. This will be ported upstream
to the nebula main branch and part of a release but that requires some more polishing and will
be taken care of in a later PR
The latest version contains a fix to allow sending metrics to APM server. also adds a apm agent jvm options
"enable_experimental_instrumentations", "true"
which is required to enable the otel-metrics-instrumentation.
relates https://github.com/elastic/elasticsearch/pull/99832
* Replaced some occurrences of Version.CURRENT toString
* Added unit tests for the 2 different warn header patterns in HeaderWarnings
* Comment to clarify non-semantic versions + revert change on Version.major message
Gradle now fully supports compiling, testing and running on Java 20.
Among other general performance improvements this release introduces --test-dry-run command line option that allows checking if tests are filtered or not by gradle.
Required updating nebula ospackage plugin as setuid was broken in gradle 8.3.
This commit updates the version range in jvm.options for the Panama Vector API. The range is updated from just a single value 20, to a range starting from 20 with no upper bound. This effectively enables the Panama Vector API for use in Lucene on all JDK versions greater than 19.
with the support of metrics the TracerPlugin name is no longer adequate. Renaming this to TelemetryPlugin.
Also introducing TelemetryProvider interface. While it is only used in Node.java at the moment to fetch Tracer instance, it is intended to be used in Plugin::createComponents (to be done in separate commit due to
the broad scope of this method)
This will allow for plugins to get access to both Tracer and Metric interfaces
without the need to add yet another argument to createComponents
Also adding internal subpackage in module/apm so that it is more obvious
which packages are not exported
Elasticsearch use lib as its module path, which includes all the
dependencies of server. However, there are sometimes additional modules
such as preallocate that are used dynamically and not direct
dependencies of server. This commit switches the addition of these
modules to be automatic by including all modules present in the module
path.
In order for build info to be pluggable for serverless, the current
build needs to be lazily determined. This commit moves the CURRENT
constant to a static method.
relates #96861
ES logging has to be explicitly initialised with a call to LoggerFactory.setInstance This is normally done as LogConfigurator.configure(Environment,boolean) is calling this as part of initPhase1
However CLI tools were not using that method. Cli tools are using LogConfigurator.configureWithoutConfig and that method was not setting up ES logging
This commit modifies configureWithoutConfig to also configure esLogging It also adds a LogConfigurator.configureESLogging call explicitly in CliToolLauncher This allows to build a new cli-launcher.jar and replace it in previous 8.7-8.9 clusters
closes#97350
Lots of spots where we did weird things around streams like redundant stream creation, redundant collecting
before adding all the collected elements to another collection or so, redundant streams for joining strings
and using less efficient `Collectors.toList` and in a few cases also incorrectly relying on the result being mutable.
This commit updates the plugin cli and scanner components to use ASM 9.5.
The update is required to successfully test with JDK 21. Tests in this component programatically run the java source compiler, which generates class files with major version 65, then tries to parse those generated class files. Without this change the tests fail with java.lang.IllegalArgumentException: Unsupported class file major version 65.
Bump the minimum targetCompatibility of java version checker to 1.8 ( from 1.7 ).
JDK 20 has removed support for source and target 1.7 in javac [1], so to successfully build with JDK 20 (through JAVA_TOOLCHAIN_HOME), we can no longer specify targetCompatibility = JavaVersion.VERSION_1_7, for the java version checker.
The idea of the java version checker is to successfully run with "older" JDK's, but I think that 1.8 is presently more than sufficient. While we still don't "support" compiling with JDK 20, it is good to keep the project up to date, so we can more easily experiment with recent JDK's.
This change adds the jdk.incubator.vector module, so that we can enable the Panamaized vector utils in Lucene.
The module is added by default if running on JDK 20, which is the only current supported implementation in Lucene, but JDK 21 will likely come soon.
If for some reason this needs to be disable, just remove or otherwise comment out the --add-modules=jdk.incubator.vector line in the jvm.options.
The log output from Lucene shows the preferred vector bit width that is in operation.
