👋🏼 Regardless of if we decide to validation enforce #78276, may we please drop a doc note that users should avoid duplicating repositories (particularly bucket / base paths).
To make it clear that repository snapshots should be available and reliable for any mounted searchable snapshots.
Co-authored-by: David Turner <david.turner@elastic.co>
If the recovery node bandwidth settings exist,
then the default value for max snapshot speed will
be infinite, and the speed will be rate limited
by the recovery rate limit as well.
Fixes#57023
The get snapshot status API will currently return a value of `STARTED` for the state of a snapshot that is currently running. The documentation says that the `state` value for a running snapshot is `IN_PROGRESS`. This documentation change will align the docs with the actual result of the get snapshot status API.
Co-authored-by: Austin Smith <76973609+asmith-elastic@users.noreply.github.com>
1. Adds a note that you can restore older snapshots (to recover from a
failed upgrade) even after newer snapshots were taken.
2. Copies the note about incompatible S3 repo implementations to the top
level to avoid misunderstandings.
Adds a parameter `index_names` to the get snapshots API so that users may exclude the potentially very long index name lists when listing out snapshots.
closes#82937
Today the docs say that new major versions need new snapshot
repositories or else corruption may occur. This isn't true, we support
using the same repository across upgrades.
This commit adds back some notes which were lost when we consolidated
the snapshot/restore documentation into a single location.
The notes in question are that:
1) If a Snapshot repository contains Security's feature state, then
that repository contains security-sensitive information. This may
be obvious to some, but is good to state explicitly.
2) Some files, such as the keystore and TLS keys, are not included in
snapshots and are important to back up via other means.
Today the note in the docs about S3-compatible repositories notes that
the repo must behave correctly, but it's also important that it has the
same performance profile. This commit extends the docs to include this
info.
This commit corrects the snapshot creation and restoration docs to
describe the usage of `"none"` to restore no feature states. Previously,
they incorrectly stated that using an empty array would accomplish this,
but specifying an empty array results in the default behavior (rather
than preventing feature state snapshot/restoration).
Elastic Cloud Enterprise (ECE) shares snapshot repositories across multiple deployments. As a result, the `base_path` is generated by ECE, and the `base_path` setting is not allowed. This PR adds a note to the S3, Azure, and GCS snapshot repository docs.
* Documents `GET _snapshot/_status` and `GET _snapshot/<repository>/_status`.
* Notes the `<repository>` and `<snasphot>` parameters are optional.
* Removes erroneous mention of the `<snapshot>` parameter supporting the `_current` value.
Closes#81600
Relates to #80931
With https://github.com/elastic/elasticsearch/pull/81870, the Azure, GCS, and S3 repository types have separate, dedicated pages in the Elasticsearch guide. For consistency, this PR creates separate pages for the shared file system, read-only URL, and source-only repository types.
Related changes:
- Adds redirects to the plugins docs
- Fixes a few breaking changes that refer to the Azure, GCS, and S3 repositories as plugins.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Today we note that the `repository-s3` plugin uses the JVM-wide
truststore in the docs for the `protocol` client setting, but it turns
out that this is easy to overlook since most installations will not need
to change the `protocol`. This commit adds the same detail to the
section on S3-compatible repositories where it is more likely to be
found.
* Add support for HTTP Proxies for the GCS repository
The change adds 3 new client properties for the GCS repository:
* gcs.client.default.proxy.type
* gcs.client.default.proxy.host
* gcs.client.default.proxy.port
They allow to configure a [java.net.Proxy](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/net/Proxy.html)
for the GCS SDK to use when communicating with the GCS API.
Resolves#82444
There have been many requests to support repository-s3 authentication via IAM roles in Kubernetes service accounts.
The AWS SDK is supposed to support them out of the box with the aws-java-sdk-sts library. Unfortunately, we can't use WebIdentityTokenCredentialsProvider from the SDK. It reads the token from AWS_WEB_IDENTITY_TOKEN_FILE environment variable which is usually mounted to /var/run/secrets/eks.amazonaws.com/serviceaccount/token and the S3 repository doesn't have the read permission to read it. We don't want to hard-code a file permission for the repository, because the location of AWS_WEB_IDENTITY_TOKEN_FILE can change at any time in the future and we would also generally prefer to restrict the ability of plugins to access things outside of their config directory.
To overcome this limitation, this change adds a custom WebIdentityCredentials provider that reads the service account from a symlink to AWS_WEB_IDENTITY_TOKEN_FILE created in the repository's config directory. We expect the end user to create the symlink to indicate that they want to use service accounts for authentification.
Service accounts are checked and exchanged for session tokens by the AWS STS. To test the authentification flow, this change adds a test fixture which mocks the assume-role-with-web-identity call to the service and returns a response with test credentials.
Fixes#52625
### Changes
* Updates the snapshot version compatibility table to use minor versions rather than major versions.
* Adds a index creation version and cluster compatibility table. Updates the index compatibility section to use minor versions.
* Moves the tables to separate files. This'll help prevent merge conflicts.
* Fixes the heading level for the "Warnings" section.
Changes:
* Moves the get SLM policy API example _after_ the get SLM stats API. This seems to fit the normal workflow better where a user will drill-down into a particular policy to get more information.
* Notes some more information about what the get SLM policy API returns. In particular, it notes that you can get the error message for the last policy failure.
Closes#81652.
Convert the `repository-azure`, `repository-gcs` and `repository-s3`
plugins into modules, so that they are always included in the
Elasticsearch distribution. Also change plugin installation, removal
and syncing so that attempting to add or remove these plugins still
succeeds but is now a no-op.
We say to mark repos as readonly to prevent corruption, but there's
other ways to prevent corruption that people sometimes use instead (e.g.
denying writes at the filesystem/bucket level). It's reasonable to think
that the readonly flag is redundant in that situation but it's not: they
should still mark the repo as readonly tho to bypass the cache and
re-read its contents on each access. This commit adds docs to that
effect.
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
Manipulating the contents of a snapshot repository is a very bad idea,
but it turns out we don't call this out in the docs anywhere. This
commit adds a warning about this.
* [DOCS] Update ES quick start for security ON by default
* Remove code.asciidoc, which is part of the overall doc build now
* Update node names for cleanup
* Add note with links to tools
* Add --net elastic network
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
