This updates the gradle wrapper to 8.12
We addressed deprecation warnings due to the update that includes:
- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions
(cherry picked from commit ba61f8c7f7)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerCloudElasticsearchDistributionType.java
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/distribution/DockerUbiElasticsearchDistributionType.java
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/test/Fixture.java
# plugins/repository-hdfs/hadoop-client-api/build.gradle
# server/src/main/java/org/elasticsearch/inference/ChunkingOptions.java
# x-pack/plugin/kql/build.gradle
# x-pack/plugin/migrate/build.gradle
# x-pack/plugin/security/qa/security-basic/build.gradle
This adds infrastructure to make the legacy test cluster plugin and
the legacy test cluster based test plugins generally configuration cache compatible.
(cherry picked from commit 8c20ac5884)
# Conflicts:
# qa/mixed-cluster/build.gradle
This commit filters out jna cleaner threads specifically in the spawner
tests (which have a different set of filters from ESTestCase because
they extend LuceneTestCase).
closes#114555
This PR exposes operator-defined, cluster-state role mappings in the
[Get role mappings
API](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-get-role-mapping.html).
Cluster-state role mappings are returned with a reserved suffix
`-read-only-operator-mapping`, to disambiguate with native role mappings
stored in the security index. CS role mappings are also marked with a
`_read_only` metadata flag. It's possible to query a CS role mapping
using its name both with and without the suffix.
CS role mappings can be viewed via the API, but cannot be modified. To
clarify this, the PUT and DELETE role mapping endpoints return header
warnings if native role mappings that name-clash with CS role mappings
are created, modified, or deleted.
The PR also prevents the creation or role mappings with names ending in
`-read-only-operator-mapping` to ensure that CS role mappings and native
role mappings can always be fully disambiguated.
Finally, the PR changes how CS role mappings are persisted in
cluster-state. CS role mappings are written (and read from disk) in the
`XContent` format. This format omits the role mapping's name. This means
that if CS role mappings are ever recovered from disk (e.g., during a
master-node restart), their names are erased. To address this, this PR
changes CS role mapping serialization to persist the name of a mapping
in a reserved metadata field, and recover it from metadata during
serialization. This allows us to persist the name without BWC-breaks in
role mapping `XContent` format. It also allows us to ensure that role
mappings are re-written to cluster state in the new, name-preserving
format the first time operator file settings are processed.
Depends on: https://github.com/elastic/elasticsearch/pull/114295
Relates: ES-9628
* [TEST] Migrated ccs-unavailable-clusters QA tests (#114764)
Ccs-unavailable-clusters QA tests migrated to the new REST testing
framework, using 'elasticsearch.internal-java-rest-test' Gradle plugin
(cherry picked from commit 551a7d6d94)
* [TEST] Fix ccs-unavailable-clusters QA tests build (#114833)
Properly use `configureEach` on the task configuration to postpone the
tasks creation and configuration in the build process
* Replace cloud-ess docker image with wolfi-ess
We just replaced the existing implementation of cloud-ess with what was wolfi-ess which is a wolfi based ess image.
The cloud image itself will be removed in a future commit it was not used anywhere
* Switch to test cloud docker image instead of default docker in packaging pr tests.
This adds way more coverage than the default docker image which is also barely touched
(cherry picked from commit f1f5ee06a3)
# Conflicts:
# build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java
This adds support for a new `index_template_substitutions` field to the
body of an ingest simulate API request. These substitutions can be used
to change the pipeline(s) used for ingest, or to change the mappings
used for validation. It is similar to the
`component_template_substitutions` added in #113276. Here is an example
that shows both of those usages working together:
```
## First, add a couple of pipelines that set a field to a boolean:
PUT /_ingest/pipeline/foo-pipeline?pretty
{
"processors": [
{
"set": {
"field": "foo",
"value": true
}
}
]
}
PUT /_ingest/pipeline/bar-pipeline?pretty
{
"processors": [
{
"set": {
"field": "bar",
"value": true
}
}
]
}
## Now, create three component templates. One provides a mapping enforces that the only field is "foo"
## and that field is a keyword. The next is similar, but adds a `bar` field. The final one provides a setting
## that makes "foo-pipeline" the default pipeline.
## Remember that the "foo-pipeline" sets the "foo" field to a boolean, so using both of these templates
## together would cause a validation exception. These could be in the same template, but are provided
## separately just so that later we can show how multiple templates can be overridden.
PUT _component_template/mappings_template
{
"template": {
"mappings": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
}
}
}
}
}
PUT _component_template/mappings_template_with_bar
{
"template": {
"mappings": {
"dynamic": "strict",
"properties": {
"foo": {
"type": "keyword"
},
"bar": {
"type": "boolean"
}
}
}
}
}
PUT _component_template/settings_template
{
"template": {
"settings": {
"index": {
"default_pipeline": "foo-pipeline"
}
}
}
}
## Here we create an index template pulling in both of the component templates above
PUT _index_template/template_1
{
"index_patterns": ["foo*"],
"composed_of": ["mappings_template", "settings_template"]
}
## We can index a document here to create the index, or not. Either way the simulate call ought to work the same
POST foo-1/_doc
{
"foo": "FOO"
}
## This will not blow up with validation exceptions because the substitute "index_template_substitutions"
## uses `mappings_template_with_bar`, which adds the bar field.
## And the bar-pipeline is executed rather than the foo-pipeline because the substitute
## "index_template_substitutions" uses a substitute `settings_template`, so the value of "foo"
## does not get set to an invalid type.
POST _ingest/_simulate?pretty&index=foo-1
{
"docs": [
{
"_id": "asdf",
"_source": {
"foo": "foo",
"bar": "bar"
}
}
],
"component_template_substitutions": {
"settings_template": {
"template": {
"settings": {
"index": {
"default_pipeline": "bar-pipeline"
}
}
}
}
},
"index_template_substitutions": {
"template_1": {
"index_patterns": ["foo*"],
"composed_of": ["mappings_template_with_bar", "settings_template"]
}
}
}
```
SpanTermQueryBuilder currently creates a valid SpanTermQuery against unmapped fields.
In practice, if the field is unmapped, there won't be a match. This commit changes
the toQuery impl to return a MatchNoDocsQuery instead like we do in similar scenarios.
Currently we have a relatively basic decider about when to throttling
indexing. This commit adds two levels of watermarks with configurable
bulk size deciders. Additionally, adds additional settings to control
primary, coordinating, and replica rejection limits.
This test was failing due to a race between an early cancellation check
and the cancel operation. With this commit we wait until the action is
definitely blocked before cancelling the task.
Closes#100062
Introduces `MockLog#awaitAllExpectationsMatched` to allow tests to wait
until all the expected log messages have been seen without having to use
`assertBusy()`.
This commit flips the incremental bulk setting to false. Additionally,
it removes some test code which intermittently causes issues with
security test cases.
* Move raw path into HttpPreRequest (#113231)
Currently, the raw path is only available from the RestRequest. This
makes the logic to determine if a handler supports streaming more
challenging to evaluate. This commit moves the raw path into pre request
to allow easier streaming support logic.
* Fix
Almost every implementation of `AckedRequest` is an
`AcknowledgedRequest` too, and the distinction is rather confusing.
Moreover the other implementations of `AckedRequest` are a potential
source of `null` timeouts that we'd like to get rid of. This commit
simplifies the situation by dropping the unnecessary `AckedRequest`
interface entirely.
The addition of the logger requires several updates to tests to deal with the possible warning, or muting if there is not way to specify an allowed (but not mandatory) warning
* Deprecate dot-prefixed indices and composable template index patterns (#112571)
This commit adds a module emitting a deprecation warning when a
dot-prefixed index is manually or automatically created, or when a
composable index template with an index pattern that uses a dot-prefix
is created. This pattern warns that in the future these indices will not
be allowed. In a future breaking change (10.0.0 maybe?) the deprecation
can then be changed to an exception.
These deprecations are only displayed when a non-operator user is using
the API (one that does not set the `X-elastic-product-origin` header).
* Attempt to fix build for V7 gradle stuff
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
JDK 23 removes the COMPAT locale provider, leaving CLDR as the only option. This commit configures Elasticsearch
to use the CLDR provider when on JDK 23, but still use the existing COMPAT provider when on JDK 22 and below.
This causes some differences in locale behaviour; this also adapts various tests to still work whether run on COMPAT or CLDR.
Memory locking on Windows with the bundled jdk was broken by native
access refactoring. This commit fixes the linking issue, as well as adds
a packaging test to ensure memory locking is invoked on all supported
platforms.
Test LogsDB backward compatibility with a rolling upgrade and full cluster restart.
We try to start indexing logs using a `standard` index, then we switch to a `LogsDB` index.
We also improve the existing test which switches between the two index modes, `standard`
and `logs`.
