* Update Gradle Wrapper to 8.2 (#96686)
- Convention usage has been deprecated and was fixed in our build files
- Fix test dependencies and deprecation
This commit adds a new test framework for configuring and orchestrating
test clusters for both Java and YAML REST testing. This will eventually
replace the existing "test-clusters" Gradle plugin and the build-time
cluster orchestration.
Since https://github.com/elastic/elasticsearch/pull/89951 we use
ML snapshot builds in all elasticsearch-ci jobs, even the
release-tests ones.
The test of the reported ML native code version in the HLRC tests
needs to account for this.
This change is 7.17 only, as the HLRC is removed in 8.x.
Test had been muted because of issue #67749
This PR unmutes test RollupIT.testPutStartAndGetRollupJob() because most probably it has
been fixed by #86992Closes#67749
Backports the following commits to 7.17:
- Add verification metadata for dependencies (#88814)
- Add missing dependency verification checksums (#89139)
The High Level REST Client provides a MountSnapshotRequest#ignoredIndexSettings(String[]) method to define some index settings to ignore when mounting a snapshot as a searchable snapshot index.
Sadly the client generates a wrong request body field ignored_index_settings instead of ignore_index_settings. This wasn't caught until #75982 was reported because the parser of Mount API request ignores unknown fields in request body.
We fixed the wrong leniency of the request parser on the Elasticsearch side (#88987) starting version 8.5.0, and we decided to continue to ignore the ignored_index_settings generated by the HLRC bug to avoid breaking HLRC client usages.
This change fixes the request body generated by the HLRC to pass the correct field name. This fix is for versions 7.17.6+ (we do not expect to release new versions of HLRC in 8.x).
It also renames the HLRC methods to expose the change to client users.
Co-authored-by: bellengao <bellengao@tencent.com>
When a gzip-encoded response is decompressed the response should no more
have a content-encoding header and content-length should be set to
"unknown". GzipDecompressingEntity correctly does this for the entity
but the response still reported the original response's content-encoding
and content-length headers.
This expands the example in the High Level Rest Client's Put Role
documentation to include an example of creating a role with index
privileges, include FLS and DLS Relates: #81354
Thanks to https://bugs.eclipse.org/bugs/show_bug.cgi?id=574437,
we've run into a situation where Spotless is incorrectly formatting
a particular piece of syntax (due the underlying Eclipse bug). We
were able to turn off formatting of this syntax using `// @formatter:off`
and `// @formatter:on`, but there was a further problem. We configure
IntelliJ to use the Eclipse formatter plugin, but this doesn't
respect the `@formatter` tags since these are set at the Spotless
level, not the Eclipse formatter level. Note that these tags aren't
set in the Eclipse formatter config, because there we use `// tag::`
and `// end::` in order to avoid reformatting docs snippets, which
have a much narrower line width.
What a mess.
So, to get around all this, drop the `@formatter` tags and tweak
our custom `SnippetLengthCheck` Checkstyle rule so that
`// tag:noformat` regions are not subject to the narrower line length
check, but are still exempt from formatting.
* Change HLRC's LifecyclePolicy to allow all valid ILM actions (#81483)
This commit changes the High Level Rest Client's `LifecyclePolicy`
implementation to allow all the same valid actions for each phase that
ILM supports. It was previously missing actions in the hot and cold
phases, as well as missing the frozen phase entirely. This also adds a
test that uses the "real" `TimeseriesLifecycleType.ORDERED_*ACTIONS`
lists so that if an action is added or removed in the future the test
will fail. Resolves#81461
# Conflicts:
# client/rest-high-level/src/test/java/org/elasticsearch/client/indexlifecycle/LifecyclePolicyTests.java
# x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ilm/TimeseriesLifecycleType.java
* Fix test
Do not fail the transform if pit search fails with index not found as a result of an index that got deleted via ILM,
if that index is part of a search that selects indices using a wildcard, e.g. logs-*. If pit search fails, the search
is retried using search without a pit context. The 2nd search might fail if the source targets an explicit index.
In addition the usage of the pit API can not be disabled by transform.
fixes elastic#81252
relates elastic#81256
* [7.16] Add replicated field to get data stream api response.
Backporting #80988 to 7.16 branch.
Internally we already kept track of whether a data stream is replicated by CCR.
It is part of the `DataStream` class. This just adds it to the xcontent serialization
of the get data stream api response class.
Relates to elastic/kibana#118899
* adjust to 7.16 reality
* Revert "Allow deprecation warning for the return_200_for_cluster_health_timeout parameter (#80178) (#80445)"
This reverts commit 5a5300753d.
* Revert "[7.x] Use query param instead of a system property for opting in for new cluster health response code (#79397) (#79435)"
This reverts commit d8c14281
* Revert "[7.x] Deprecate returning 408 for a server timeout on `_cluster/health` (#78180) (#78940)"
This reverts commit 03bd55d1
This deprecates estimated_heap_memory_usage_bytes on model put and replaces it with model_size_bytes.
On GET, both fields are returned (unless storing in the index) and are populated with the same field.
For the ml/info API, both fields are returned as well.
Part of #19752.
Fix a number of cases of shadows vars under `client/rest-high-level`. As
part of this, fork the Checkstyle `HiddeFieldCheck` class so that it
understand the pattern of settings with no "set" prefix.
Mockito 4.0 removes several deprecated methods. This commit updates
usages of those deprecated methods and upgrades mockito. The changes
include: * Replace anyMapOf,anyListOf,anySetOf,anyCollectionOf with the
same method name without `Of` and no longer taking any arguments. *
Replace anyObject with any * Removing argument from isNull * Replace
verifyZeroInteractions with verifyNoMoreInteractions The changes here
were completely mechanical, done entirely with forms of find/replace
within IntelliJ.
Matchers is deprecated in Mockito, in favor of the newer
ArgumentMatchers class. In fact, internally Matchers just extends
ArgumentMatchers as all the methods there were moved. This commit
changes all imports of org.mockito.Matchers to
org.mockito.ArgumentMatchers.
* [Transform] Improve transform upgrader (#79601)
This changes fixes some issues with the upgrader. It inlines the index name resolver as wildcard
deletes are not supported if action.destructive_requires_name is set to true(default). It changes
response to always return all counters in order to avoid an empty response. After documentation has
been created, it switches the deprecation URL to the upgrade API docs.
* change url to 7.16 specific one
* fix 7.x specific test
* fix put mapping call
Securemock is a wrapper around Mockito that monkey patches internals of
Mockito to work with the SecurityManager. However, the library has not
been updated in several years due to the complicated nature of this
monkey patching. This has left us with an ancient version of Mockito,
missing out on updates to the library in the last half decade.
While Securemock currently works with Mockito 1.x, in 2.x an official
means of plugging into mockito was added, MockMaker. This commit removes
securemock as a dependnecy of the test framework, replacing it with a
modern version of Mockito, and implementing a MockMaker that integrates
with SecurityManager.
Note that while there is a newer version of Mockito available, 4.0, it
has several deprecations removed that are used throughout Elasticsearch.
Those can be addressed in followups, and then a subsequent upgrade to
4.0 should be possible.
relates #79567closes#40334
This PR adds a framework for migrating system indices as necessary prior
to Elasticsearch upgrades. This framework uses REST APIs added in
another commit:
- GET _migration/system_features
This API, which gets the status of "features" (plugins which own system
indices) with regards to whether they need to be upgraded or not. As of
this PR, this API also reports errors encountered while migrating system
indices alongside the index that was being processed when this occurred.
As an example of this error reporting:
```json
{
"feature_name": "logstash_management",
"minimum_index_version": "8.0.0",
"upgrade_status": "ERROR",
"indices": [
{
"index": ".logstash",
"version": "8.0.0",
"failure_cause": {
"error": {
"root_cause": [
{
"type": "runtime_exception",
"reason": "whoopsie",
"stack_trace": "<omitted for brevity>"
}
],
"type": "runtime_exception",
"reason": "whoopsie",
"stack_trace": "<omitted for brevity>"
}
}
}
]
}
```
- POST _migration/system_features
This API starts the migration process. The API for this has no changes,
but when called, any system indices which need to be migrated will be
migrated, with status information stored in the cluster state for later
use by the GET _migration/system_features API.
This change the way we store mappings in `Template` during serialization
to disk - instead storing it as map we use byte array that we already
have. This avoids deserialization-serialization cycle during storing
cluster state on disk.
# Conflicts:
# server/src/main/java/org/elasticsearch/cluster/coordination/ElasticsearchNodeCommand.java
A recent change for the deprecation logs provided the capability to emit deprecation's at critical vs. warning levels, #77482.
However deprecated settings always log at critical level without the ability to express that the setting deprecation is only a
warning.
This commit exposes the ability to set the deprecation level when deprecating a setting.
Relates #78781
Backport #79351 to 7.x:
The original change was implemented in #78940, but we have decided to move from a system property
to a request parameter, so Cloud users/clients have an easier way to opt-in for the new status code.
Relates #70849
* Replace eager gradle task creation with task avoidance api
Some more eagerly created tasks sneaked into the 7.x branch lately
* Fix typo in integTest configuration
