This PR moves the doPrivileged wrapper closer to the actual deletion
request to ensure the necesary security context is established at all
times. Also added a new repository setting to configure max size for s3
deleteObjects request.
Fixes: #108049
One user reached out mentioning that it would be a good idea to remind
users to re-upload the license after full cluster recovery from snapshot
as one can easily miss this when trying to figure out why some features
aren't working after the restore.
* Remove `es-test-dir` book-scoped variable
* Remove `plugins-examples-dir` book-scoped variable
* Remove `:dependencies-dir:` and `:xes-repo-dir:` book-scoped variables
- In `index.asciidoc`, two variables (`:dependencies-dir:` and `:xes-repo-dir:`) were removed.
- In `sql/index.asciidoc`, the `:sql-tests:` path was updated to fuller path
- In `esql/index.asciidoc`, the `:esql-tests:` path was updated idem
* Replace `es-repo-dir` with `es-ref-dir`
* Move `:include-xpack: true` to few files that use it, remove from index.asciidoc
The S3 SDK permits changing the maximum number of concurrent connections
that it will open, but today there's no way to adjust this setting
within Elasticsearch. This commit adds a setting for this parameter.
Specifying `?master_timeout=-1` on an API which performs a cluster state
update means that the cluster state update task will never time out
while waiting in the pending tasks queue. However this parameter is also
re-used in a few places where a timeout of `-1` means something else,
typically to timeout immediately. This commit fixes those places so that
`?master_timeout=-1` consistently means to wait forever.
Allegedly-S3-compatible APIs are very popular these days, but many
third-party systems offering such an API also support a shared
filesystem interface. Shared filesystem protocols such as NFS are much
better specified than the S3 API, and experience shows that they lead to
fewer compatibility headaches. This commit adds a recommendation to the
`repository-s3` docs to consider such an interface instead.
To prevent leaking sensitive information such as credentials and keys in logs, this
commit prevents configuring some restricted loggers (currently `org.apache.http`
and `com.amazonaws.request`) at high verbosity unless the NetworkTraceFlag
(`es.insecure_network_trace_enabled`) is enabled.
Moves some of the detail about S3 storage classes to their own section
for easier linking, and adds a note about `intelligent_tiering` archive
classes.
* docs: fix numbering in restore-snapshot.asciidoc
Fix numbering in "Restore an entire cluster" section.
Remove "3." for "Universal Profiling" and add "3." just before "If you use Elasticsearch security features"
* Keep ID, fix list rendering
---------
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
This commit updates the docs to call out that custom certificate
authorities for S3 repositories will need to be reinstalled every time
ES is upgraded, is the node is using the bundled JDK
These docs have various subsections under the _Client settings_ and
_Repository settings_ sections which are nothing to do with those
settings. The reasons for this are historical and no longer relevant.
This commit moves these subsections together and up a level so that they
better reflect the structure of the information.
We don't mention linearizable registers in the snapshot/restore docs
today, but these things are verified for correctness by the repository
analysis API and some users with incorrect repository implementations
struggle to understand the verification errors. This commit adds some
docs to describe them and their various implementations.
Adds the `?register_operation_count` parameter that allows to control
the number of register operations separately from the number of regular
blob operations.
It's often useful to quote these docs to users encountering problems
with their not-quite-S3-compatible storage system. In practice we don't
need to quote the bits in the middle but we do need the last sentence
about working with the supplier to address incompatibilities. This
commit reorders things so that the most commonly quoted sentences form a
standalone paragraph.
👋 howdy, team! Expanding reference to [internal](https://github.com/elastic/cloud/pull/118105) update, we've just confirmed ILM requires the repository name to be the same among migrating clusters. This is a hard block for Searchable Snapshots which requires un-Searchable-Snapshotting or redoing migration to resolve.
* [DOC+] snapshot-restore single index example
👋🏼 howdy, team! I'd like to append an example to snapshot-restore a single index. Support usually points users to [this page](https://www.elastic.co/guide/en/elasticsearch/reference/master/restore-snapshot-api.html) but then users attempt the `rename_pattern` example (which makes sense!). I'd like to point them to a more literal "close index > restore on that index" example in the future.
* Fix test failure and reword
---------
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
👋🏼 Regardless of if we decide to validation enforce #78276, may we please drop a doc note that users should avoid duplicating repositories (particularly bucket / base paths).
To make it clear that repository snapshots should be available and reliable for any mounted searchable snapshots.
Co-authored-by: David Turner <david.turner@elastic.co>
If the recovery node bandwidth settings exist,
then the default value for max snapshot speed will
be infinite, and the speed will be rate limited
by the recovery rate limit as well.
Fixes#57023
The get snapshot status API will currently return a value of `STARTED` for the state of a snapshot that is currently running. The documentation says that the `state` value for a running snapshot is `IN_PROGRESS`. This documentation change will align the docs with the actual result of the get snapshot status API.
Co-authored-by: Austin Smith <76973609+asmith-elastic@users.noreply.github.com>
