* Update Gradle wrapper to 8.13 (#122421)
* Fix Gradle Deprecation warning as declaring an is- property with a Boolean type has been deprecated.
* Make use of new layout.settingsFolder api to address some cross project references
* Fix buildParams snapshot check for multiprojet projects
(cherry picked from commit e19b2264af)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/BaseInternalPluginBuildPlugin.java
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/InternalDistributionBwcSetupPlugin.java
# docs/build.gradle
# qa/lucene-index-compatibility/build.gradle
# x-pack/qa/multi-project/core-rest-tests-with-multiple-projects/build.gradle
# x-pack/qa/multi-project/xpack-rest-tests-with-multiple-projects/build.gradle
* More fixes
This updates the gradle wrapper to 8.12
We addressed deprecation warnings due to the update that includes:
- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions
(cherry picked from commit ba61f8c7f7)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerCloudElasticsearchDistributionType.java
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerUbiElasticsearchDistributionType.java
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/Fixture.java
# plugins/repository-hdfs/hadoop-client-api/build.gradle
# server/src/main/java/org/elasticsearch/inference/ChunkingOptions.java
# x-pack/plugin/kql/build.gradle
# x-pack/plugin/migrate/build.gradle
# x-pack/plugin/security/qa/security-basic/build.gradle
* Reapply "[Build] Do not invalidate configuration cache when branch is switched (#118894)" (#119300) (#119325)
* Reapply "[Build] Do not invalidate configuration cache when branch is switched (#118894)" (#119300)
The original PR (#118894) has broken serverless.
* Fix gitinfo plugin for serverless usage
* Update buildscan git revision reference
(cherry picked from commit 5278159987)
# Conflicts:
# build-conventions/src/main/java/org/elasticsearch/gradle/internal/conventions/PublishPlugin.java
* Fix merge conflict
Today the overloads of `XContentBuilder#timeField` do two rather
different things: one formats an object as a `String` representation of
a time (where the object is either an unambiguous time object or else a
`long`) and the other formats only a `long` as one or two fields
depending on the `?human` flag.
This is trappy in a number of ways:
- `long` means an absolute (epoch) time, but sometimes folks will
mistakenly use this for time intervals too.
- `long` means only milliseconds, there is no facility to specify a
different unit.
- the dependence on the `?human` flag in exactly one of the overloads is
kinda weird.
This commit removes the confusion by dropping support for considering a
`Long` as a valid representation of a time at all, and instead requiring
callers to either convert it into a proper time object or else call a
method that is explicitly expecting an epoch time in milliseconds.
* Allow incubating Panama Vector in simdvec, and add vectorized ipByteBin (#112933)
Add support for vectorized ipByteBin.
The structure of the implementation and loading framework mirror that of Lucene, but is simplified by avoiding reflective loading since ES has support for a MRJar section for 21.
For now, we just disable warnings-as-errors in this small sourceset, since -Xlint:-incubating is only support since JDK 22. The number of source files is small here. Will investigate how to assert that just the single incubating warning is emitted by javac, at a later point.
* fix build runtime java check
* Fix Gradle configuration in idea for :libs:simdvec (#114251)
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Rene Groeschke <rene@elastic.co>
* Add CircuitBreaker to TDigest, Step 4: Take into account shallow classes size (#113613)
* Removed muted tests from merge conflict
* Added missing empty line in muted tests
The overridden `RefCounted` methods on `AbstractRefCounted` are all
`final`, but `mustIncRef` uses the default implementation and can still
be overridden. For the avoidance of doubt, this commit marks
`mustIncRef` as `final`.
Extensible plugins use a custom classloader for other plugin jars. When
extensible plugins were first added, the transport client still existed,
and elasticsearch plugins did not exist in the transport client (at
least not the ones that create classloaders). Yet the transport client
still created a PluginsService. An indirection was used to avoid
creating separate classloaders when the transport client had created the
PluginsService.
The transport client was removed in 8.0, but the indirection still
exists. This commit removes that indirection layer.
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Part of https://github.com/elastic/elasticsearch/issues/99815
## Steps
1. Migrate TDigest classes to use a custom Array implementation. Temporarily use a simple array wrapper (https://github.com/elastic/elasticsearch/pull/112810)
2. Implement CircuitBreaking in the `MemoryTrackingTDigestArrays` class. Add `Releasable` and ensure it's always closed within TDigest (This PR)
3. Pass the CircuitBreaker as a parameter to TDigestState from wherever it's being used
4. Account remaining TDigest classes size ("SHALLOW_SIZE")
Every step should be safely mergeable to main:
- The first and second steps should have no impact.
- The third and fourth ones will start increasing the CB count partially.
## Remarks
To simplify testing the CircuitBreaker, added a helper method + `@After` to ESTestCase.
Right now CBs are usually tested through MockBigArrays. E.g:
f7a0196b45/x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/expression/function/AbstractFunctionTestCase.java (L1263-L1265)
So I guess there was no need for this yet. But I may have missed something somewhere.
Also, I'm separating this PR from the "step 3" as integrating this (CB) in the current usages may require some refactor of external code, which may be somewhat more _dangerous_
It seemed if there wasn't any text to parse, this is not an internal
issue but instead an argument issue.
I simply changed the exception thrown. If we don't agree with this, I
can adjust `query` parsing directly, but this seemed like the better
choice.
closes: https://github.com/elastic/elasticsearch/issues/112296
Currently we wrap JsonEOFException from advancing the json parser into our own
XContentEOFException, but this has the drawback that is results in 500 errors on
the client side. Instead this should be 400 errors.
This changes XContentEOFException to extend XContentParseException so we report
a 400 error instead.
Closes#111542
No need to precompute the toString for `ActionListener` and
`Releasable`, that's quite expensive at times. Also string concat is way
faster than formating these days, so use that in the transport channels.
Lastly, short-circuit some obvious spots in network address
serialization and remove code that duplicates the JDK (remove the IPV4
specific forbidden API because it makes no sense, but still needed to
disable the check to make the build green because of the exclude on the
parent class).
Recent refactoring of native library paths broke jna loading zstd. This
commit fixes jna to set the jna.library.path during init so that jna
calls to load libraries still work.
Native libraries in Java are loaded by calling System.loadLibrary. This
method inspects paths in the java.library.path to find the requested
library. Elasticsearch previously used this to find libsystemd, but now
the only remaining use is to set the additional platform directory in
which Elasticsearch keeps its own native libraries.
One issue with setting java.library.path is that its not set for the cli
process, which makes loading the native library infrastructure from clis
difficult. This commit reworks how Elasticsearch native libraries are
found in order to avoid needing to set java.library.path. There are two
cases. The simplest is production, where the working directory is the
Elasticsearch installation directory, so the platform specific directory
can be constructed. The second case is for tests where we don't have an
installtion. We already pass in java.library.path there, so this change
renames the system property to be a test specific property that the new
loading infrastructure looks for.
Systemd notification now happens by directly communicating with the
systemd socket. This commit removes the native access to libsystemd,
which is no longer used.
When xcontent is copied, the parse tree is walked and each element is
passed to the given generator. In the case of numbers, BigInteger is
currently not handled. Although arbitrary precision BigIntegers are not
supported in Elasticsearch, they appear in xcontent when using unsigned
long fields. This commit adds handling for that case, and also ensures
all token types are handled. Note that BigDecimal are not supported at
all since double is the largest floating point mapper supported.
closes#111812
The libsystemd library function sd_notify is just a thin wrapper
around opeing and writing to a unix filesystem socket. This commit
replaces using libsystemd with opening the socket provided by systemd
directly.
relates #86475
Memory locking on Windows with the bundled jdk was broken by native
access refactoring. This commit fixes the linking issue, as well as adds
a packaging test to ensure memory locking is invoked on all supported
platforms.
* Add maximum nested depth check to WKT parser
This prevents StackOverflowErrors, replacing them with ParseException errors, which is more easily managed by running servers.
* Update docs/changelog/111843.yaml
Rename `xContent.streamSeparator()` and
`RestHandler.supportsStreamContent()` to `xContent.bulkSeparator()` and
`RestHandler.supportsBulkContent()`.
I want to reserve use of "supportsStreamContent" for current work in
HTTP layer to [support incremental content
handling](https://github.com/elastic/elasticsearch/pull/111438) besides
fully aggregated byte buffers. `supportsStreamContent` would indicate
that handler can parse chunks of http content as they arrive.
The VectorSystemPropertyTests need to run a child process. Normally this
isn't possible since we run with security manager, but the
`@WithoutSecurityManager` annotation causes the test suite to run with
the security manager disabled. However, that annotation only works with
ESTestCase. This commit changes the base class of the test suite to
ESTestCase.
closes#110949
The preallocate tests assumed that preallocation was using the fallback
implementation which calls setLength on Windows. However, that fallback
only happens inside the SharedBytes class, so windows doesn't actually
do anything when tryPreallocate is called. This commit skips the test on
windows.
closes#110948
When JNA loads libraries it creates a proxy object for the library.
Unfortunately it doesn't actually inspect any of the methods, those get
bound lazily at runtime when the method is called through the proxy. For
fstat64 we need to know at load time whether the symbol exists, so that
we can fallback to an alternate function if it doesn't.
This commit looks up the NativeLibrary object from JNA for libc and
checks if fstat64 exists during load time.
