Recent versions of MinIO will sometimes leak multi-part uploads under
concurrent load, leaving them in the `ListMultipartUploads` output even
though they cannot be aborted. Today this causes repository analysis to
fail since compare-and-exchange operations will not even start if there
are any pre-existing uploads. This commit makes it possible to skip this
pre-flight check (and accept the performance consequences) by adjusting
the relevant settings.
Workaround for minio/minio#21189
Closes#122670
Catching Exception instead of AmazonClientException in copyBlob and
executeMultipart led to failures in S3RepositoryAnalysisRestIT due to
the injected exceptions getting wrapped in IOExceptions that prevented
them from being caught and handled in BlobAnalyzeAction.
Closes#126576
- Translate a 404 during a multipart copy into a `FileNotFoundException`
- Use multiple threads in `S3HttpHandler` to avoid `CopyObject`/`PutObject` deadlock
Closes#126576
* BlobContainer: add copyBlob method
If a container implements copyBlob, then the copy is
performed by the store, without client-side IO. If the store
does not provide a copy operation then the default implementation
throws UnsupportedOperationException.
This change provides implementations for the FS and S3 blob containers.
More will follow.
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: David Turner <david.turner@elastic.co>
The mostly-optional parameters to `createBlobContainer` are getting
rather numerous in this test harness which makes the tests hard to read.
This commit introduces a builder to help name the provided parameters
and skip the omitted ones.
Some `AbstractBlobContainerRetriesTestCase#createBlobContainer`
implementations choose a path for the container randomly, but we have a
need for a test which re-creates the same container against a different
`S3Service` and `BlobStore` and must therefore specify the same path
each time. This commit exposes a parameter that lets callers specify a
container path.
Rather than hard-coding a region name we should always auto-generate it
randomly during test execution. This commit replaces the remaining fixed
`String` arguments with a `Supplier<String>` argument to enable this.
Today we have some special-case test classes in `:modules:repository-s3`
within the same source root as the regular tests, with some trickery to
define separate Gradle tasks to run them with their special-case
configs. This commit simplifies the build by just moving each of these
classes into its own Gradle project.
These tests only don't work in a FIPS JVM because they use a secret key
that is unacceptably short. This commit replaces the relevant uses of
`randomIdentifier` with `randomSecretKey` so they work whether in FIPS
mode or not.
`S3BlobContainerRetriesTests` uses `MD5DigestCalculatingInputStream`
from the AWS v1 SDK to compute a MD5 checksum, but this feature is not
available in the v2 SDK. With this commit we remove this dependency and
compute the MD5 checksums directly instead.
The `METHOD /path/components?and=query` string representation of a
request is becoming increasingly difficult to parse, with slight
variations in parsing between the implementation in `S3HttpHandler` and
the various other implementations. This commit gets rid of the
string-concatenate-and-split behaviour in favour of a proper object that
has predicates for testing all the different kinds of request that might
be made against S3.
Originally this metrics collector was just there to ignore API calls
that didn't make it all the way to S3, but (a) it doesn't really do that
because it also apparently ignores 4xx responses and (b) it also does a
bunch of other metrics collection too. `IgnoreNoResponseMetricsCollector`
is definitely the wrong name these days so this commit renames it to
something more general.
Extends the predicate in `AwsCredentialsUtils` to verify that we are
using a proper AWS v4 signature complete with the correct region and
service, rather than just looking for the access key as a substring.
* Refactor: findRequestingFrame
* INFO instead of WARN for NotEntitledException.
Some of these are expected, so an INFO seems more appropriate.
The stack trace tends to attract attention even when entitlements are not the
cause of a problem, so let's avoid the stack trace, but still include stack
frame info from the frame of interest.
* Use child loggers for Not Entitled logs
* Use warn, and include compoenent name
* Fix ALL_UNNAMED
* Mute entitlement warnings from repositories
* PR feedback
* Common out the Not Entitled prefix.
We're alerting on this, so let's not rely on every caller of notEntitled to remember it.
There's no need to have a `SnapshotInfo` consumer to run at the end of
finalization, we only pass it the value we already calculated earlier.
This replaces it with a bare `Runnable` instead.
Depending on whether a message is critical or warning a message should
indicate to check breaking changes documentation (critical level) or
deprecation changes documentation (warn level)
relates #79666
If all items fail to delete, the current warning log builds a huge
string to include all of them (up to 1000). This PR limits the string
length to first 10 entries.
Relates: #123630
This change adds a unit test to demonstrate a specific
behavior of the AWS SDKv1, which closes the InputStream
used to upload a blob only after the HTTP request has
been sent (this is to accomodate for retries). The SDK
then swallows any exception thrown when closing the
InputStream which has the effect to hide any potential
CorruptIndexException that could have been detected
at that time.
Relates ES-10931
When a CopmleteMultipartUpload request fails after the initial 200
response, the status code of the failure response use to be not set and
hence got translated to status code 0. With #116212, we handle this case
accordingly. Since AWS SDK 1.12.691, the status code is now set to 200
instead of 0. This PR changes our error handling code accordingly.
Relates: #122431 Relates: #116212Resolves: #122799
Relevant AWS SDK change
430899c217/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java (L3696-L3709)
The node environment has many paths. The accessors for these currently
use a "file" suffix, but they are always directories. This commit
renames the accessors to make it clear these paths are directories.
* Exhaustive testParseFractionalNumber
* Refactor: encapsulate ByteSizeUnit constructor
* Refactor: store size in bytes
* Support up to 2 decimals in parsed ByteSizeValue
* Fix test for rounding up with no warnings
* ByteSizeUnit transport changes
* Update docs/changelog/120142.yaml
* Changelog details and impact
* Fix change log breaking.area
* Address PR comments
This PR upgrades the minio docker image from
RELEASE.2021-03-01T04-20-55Z which is 3+ years old to the latest
RELEASE.2024-12-18T13-15-44Z.
Relates: #118548
This updates the gradle wrapper to 8.12
We addressed deprecation warnings due to the update that includes:
- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions
This method is never called against a general `BlobStore`, we only use
it in certain implementations for which a bulk delete at the `BlobStore`
level makes sense. This commit removes the unused interface method.
Today each relevant module defines the version of the AWS SDK that it
uses, which means there's a risk that we use different versions in
different modules. This commit pulls the version declarations to the top
level to make sure we keep everything in sync.
