Fix incompatibility with 8.8 and our internal api usages
- Update ospackage to a version that contains a fix we provided
- Tweak build logic to avoid deprecation warnings
- Use newer permission api
- Use custom shadowplugin
- Rework ElasticsearchDistribution dependencies resolution
- Update Gradle wrapper to 8.8
- Fix one more deprecated permission api usage
- Fix DistroTest dependencies
- Fix permissions in package init.d folder
The new subcommand elasticsearch-node remove-index-settings can be used
to remove index settings from the cluster state in case where it
contains incompatible index settings that prevent the cluster from
forming. This tool can cause data loss and its use should be your last
resort.
Relates #96075
This commit introduced stricter DLS rules and is a manual backport of #105709 and #105714
with additional node level settings to optionally disable the stricter DLS rules.
Since these settings are not present in 8.x the needed deprecation info API entries have also
been added to help inform any users that may have set these values to remove them before upgrading.
This setting was removed via #25734, because the setting no longer used
since 6.0.0
However, the validation only kicked when trying to set this setting on a
closed index. Applying the setting on an open index would just work.
With severe consequences later on. For example when upgrading the
cluster, nodes would refuse to boot, because the validation would kick
in.
Relates to #96075
Block specific config files from being accessed after startup (#107481)
Some files should never be accessed by ES or plugin code once startup has completed. Use the security manager to block these files from being accessed by anything at all. The current blocked files are elasticsearch.yml, jvm.options, and the jvm.options.d directory.
Not really necessary to allow the scheduled task to race against the
blocks, and this race is a source of test flakiness. Fixed by imposing
the blocks first.
Closes#106618
We can't assert no leaked blobs here because today the first cleanup
leaves the original `RepositoryData` in place so the second cleanup is
not a no-op.
Relates #100718
Backport of #100855 to 7.17
If the search threadpool fills up then we may reject execution of
`SearchService.Reaper` which means it stops retrying. We must instead
force its execution so that it keeps on going.
With #106542, closes#106543
Backport of #106544 to 7.17
Today `ThreadPool#scheduleWithFixedDelay` does not interact as expected
with `AbstractRunnable`: if the task fails or is rejected then this
isn't passed back to the relevant callback, and the task cannot specify
that it should be force-executed. This commit fixes that.
Backport of #106542 to 7.17
* Fix deleting index during snapshot finalization
Today if an index is deleted during a very specific order of snapshot
finalizations then it's possible we'll miscalculate the latest shard
generations for the shards in that index, causing the deletion of a
shard-level `index-UUID` blob which prevents further snapshots of that
shard.
Backports #103817 to 7.17
Closes#101029
* Test fixup
* Update ExecutorScalingQueue to workaround LinkedTransferQueue JDK bug (#104347)
This commit adds a few overrides to ExecutorScalingQueue (subclass of LinkedTransferQueue) to workaround a JDK bug in LinkedTransferQueue.
* remove usage of var
* fix compilation
* Add full GC fallback within the current attempt (#102563)
* Add full GC fallback within the current attempt
* Try re-acquire the lock if we timed out due to a full GC
* Condition full GC to memory reclaimed only
* Fix: transform record to private static class
If waiting for a particular cluster state (e.g. on the CCR leader) we
will compute the resulting cluster state and serialize it on the cluster
applier thread, which can be too expensive in a large cluster for this
thread. With this commit we dispatch the final action back to the
original executor.
* Unwrap exception more tenaciously in testQueuedOperationsAndBrokenRepoOnMasterFailOver (#102352)
There can be more than 10 layers of wrapping RTEs, see #102351. As a
workaround to address the test failure, this commit just manually
unwraps them all.
Closes#102348
* Fixup
A call to `ConnectionTarget#connect` which happens strictly after all
calls that close connections should leave us connected to the target.
However concurrent calls to `ConnectionTarget#connect` can overlap, and
today this means that a connection returned from an earlier call may
overwrite one from a later call. The trouble is that the earlier
connection attempt may yield a closed connection (it was concurrent with
the disconnections) so we must not let it supersede the newer one.
With this commit we prevent concurrent connection attempts, which avoids
earlier attempts from overwriting the connections resulting from later
attempts.
Backport of #92558
When combined with #101910, closes#100493
Today we call `Transport.Connection#onRemoved`, notifying any
removed-listeners, when the connection is closed and removed from the
`connectedNodes` map. However, it's possible for the connection to be
closed while we're still adding it to the map and setting up the
listeners, so this now-dead connection will still be found in the
`pendingConnections` and may be returned to a future call to
`connectToNode` even if this call was made after all the
removed-listeners have been called.
With this commit we delay calling the removed-listeners until the
connection is closed and removed from both the `connectedNodes` and
`pendingConnections` maps.
Backport of #92546 to 7.17
Relates #100493
