The workaround requires to JDK args, but SystemJvmOptions actually
operates on individual JDK args. This commit adjusts SystemJvmOptions to
allow adding sets of jdk args together.
This removes getWriter() and getErrorWriter() from Terminal in order to prepare for using log4j in server-cli.
Additionally, ensure IO failures on the pump thread are handled once completed (closed) and
catch throwables in Command#main to print them (formatted) via Terminal.
This commit bumps Tika to 2.9.2 and manually bumps the transitive versions
to match 2.9.2's parent POM. This commit also centralizes the dependency
versions so that you only need to look at 1 list to see the full set of dependencies
to manually check.
This commit adds an optimised int8 vector distance implementation for aarch64. Additional platforms like, say, x64, will be added as a follow-up.
The vector distance implementation outperforms Lucene's Pamana Vector implementation for binary comparisons by approx 5x (depending on the number of dimensions). It does so by means of compiler intrinsics built into a separate native library and link by Panama's FFI. Comparisons are performed on off-heap mmap'ed vector data.
The implementation is currently only used during merging of scalar quantized segments, through a custom format ES814HnswScalarQuantizedVectorsFormat, but its usage will likely be expanded over time.
Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
Co-authored-by: Lorenzo Dematté <lorenzo.dematte@elastic.co>
Co-authored-by: Mark Vieira <portugee@gmail.com>
Co-authored-by: Ryan Ernst <ryan@iernst.net>
On some systems Java appears to return amd64 (even if not an amd
processor), but on others it returns x86_64. This commit handles the
latter case to correctly associate the arch with the appropriate
platform dir.
For small nodes, we need a bit more wiggle room for new size
and concurrent GC threads in order to stay below real memory
circuit breaker limits on dedicated search nodes.
ES-8087
This commit makes zstd compression available to Elasticsearch. The
library is pulled in through maven in jar files for each platform, then
bundled in a new platform directory under lib. Access to the zstd
compression/decompression is through NativeAccess.
The build handles platform specific code which may be for arm or x86.
Yet there are multiple ways to describe 64bit x86, and the build
converts between the two in several places. This commit consolidates on
the x64 nomenclature in most places, except where necessary (eg ML still
uses x86_64).
relates #105715
The difference is that our matcher uses .find() to search for a regex match anywhere in the string, whereas the hamcrest one uses .matches() to check the whole string against the regex. This leads to more specific regex checks.
I've left our own one for YAML tests, as that way we don't need to mangle the regex to add .* either side, which might be confusing in test failures.
Elasticsearch requires access to some native functions. Historically
this has been achieved with the JNA library. However, JNA is a
complicated, magical library, and has caused various problems booting
Elasticsearch over the years. The new Java Foreign Function and Memory
API allows access to call native functions directly from Java. It also
has the advantage of tight integration with hotspot which can improve
performance of these functions (though performance of Elasticsearch's
native calls has never been much of an issue since they are mostly at
boot time).
This commit adds a new native lib that is internal to Elasticsearch. It
is built to use the foreign function api starting with Java 21, and
continue using JNA with Java versions below that.
Only one function, checking whether Elasticsearch is running as root, is
migrated. Future changes will migrate other native functions.
When hotspot encounters an error, it will emite a log file which can be
used for reproducing the error. This file is dumped to /tmp by default.
This commit configures the replay file to be alongside the hs_err file.
In certain circumstances if Elasticsearch encounters an error while
starting up, the server cli may exit with no error. This commit fixes
the cli to always check and wait on the Elasticsearch process and exit
with the same exit code.
relates #104055
Telemetry / APM settings are renamed from "tracing.apm.{name}" to "telemetry.tracing.{name}" for tracing related settings. General APM settings are renamed to "telemetry.{name}". The old legacy settings are kept for now and applied as fallback.
Prevent invalid and miss-configuration of APM agent using an explicit allow-list of setting keys.
Additionally, configuration defaults of APMAgentSettings are consolidated in APMJvmOptions to keep defaults in a single location.
(ES-6916)
We're leaking quite a few of these parsers. That doesn't seem to be much
of a problem but results in some memory inefficiencies in Jackson here
and there. This PR bulk fixes a bunch of instances that I could easily
automatically fix. I'll open a follow-up for closing the parser on the
document parsing context which also suffers from this but is non-trivial
to fix.
* Refactoring: move command line and environment build to a separate class
* Refactoring: adding a builder
* Moving tmp dir setup and JVM option parsing outside of builder
This commit upgrades the Bouncy Castle jars. Bouncy Castle is used for
some internal build concners as well as a comnand line application.
Most notably Bouncy Castle is also used as the FIPs certified JCE/JSEE provider
we use to test our ability to use a FIPs compliant crypto provider.
The following changes here are a result of the upgraded Bouncy Castle jars:
* TLSv1.3 is now supported when running in FIPs mode
* RSA PKCS#1 v1.5 is no longer allowed in FIPS mode
* Triple DES (3DES) is no longer allowed in FIPS mode
* Minor updates the security manager configuration used to test FIPs (to read permissions from the security provider)
* Minor adjustments to tests to accommodate the above changes.
* Minor adjustments to the gradle build to accommodate new dependencies
Note - update to the documentation will come in a later commit.
This introduces a new getHistoricalFeatures() method on ESRestTestCase
which returns a map of historical feature version mappings loaded from
FeatureSpecification implementations from any plugins/modules in use
by the current test suite. The mappings are generated by a new Gradle
task at build time, and then injected into the test runtime as a
System property.
