This PR updates `bc-fips` and `bctls-fips` dependencies to the latest
minor versions.
(cherry picked from commit 6ea3e01958)
Co-authored-by: Slobodan Adamović <slobodanadamovic@users.noreply.github.com>
The option is only enabled for test clusters, and turns on features that are only present in tests
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Adding ChunkingSettings logic and enabling ChunkingSettings for OpenAI embedding endpoints
* Cleaning up naming in ChunkingSettings logic
* Incrementing InferenceIndex version
* Removing DefaultChunkingSettings, cleaning up chunking settings class and related tests, add chunking strategy to inference index
* Adding check for up to date index mappings when creating an inference endpoint
* Fixing transport version conflict
* Adding validation for invalid chunking settings inputs and improving error messaging
* Reverting SystemIndexMappingUpdateService changes and adding error messaging on mixed cluster exception
A freshly-started cluster processes a lot of setup tasks after it's
considered ready by the test-clusters framework. This commit makes it
also wait until the master queue is empty and there are no initializing
or relocating shards before it starts to run the test.
Closes#111395
Local test clusters have several methods allowing interaction with nodes
by ordinal number. However, there is currently no way to know how mnany
nodes were actually configured for the cluster. This commit adds an
accessor for the number of nodes the cluster handle knows about.
This ports our krb5kdc test fixture to test container and reworks hdfs handling to also be based on test containers.
The yaml rest tests that are using hdfs required introducing variable substitution in yamlresttestparser handling.
We have instances where BWC tests configure old ES version nodes with
the integTest distribution. This isn't a valid configuration, and while
we in reality resolve the default distribution artifact, we have other
configuration logic that behaves differently based on whether the
integTest distro was _requested_. Specifically, what to set ES_JAVA_HOME
to. This bug resulted in us attempting to run old nodes using the
current bundled JDK version, which may be incompatible with that older
version of Elasticsearch.
Closes#104858
Grants the SocketPermission "*", "connect" permission to the
bouncy castle JSSE implementation lib, when running ITs under
FIPS-configured JVMs.
Note that the plugin's permissions e.g.
x-pack/plugin/ml-package-loader/src/main/plugin-metadata/plugin-security.policy
are still checked and enforced.
This new permission is required because the BC lib hooks into
the JVM when opening any TLS connection, under the stacktrace
of the plugin's AccessController.doPrivileged (so the lib's permissions
are checked first, before the doPrivileged call on the stack).
Relates #100923Fixes#104414
https://github.com/elastic/elasticsearch/pull/103483 introduced a new
setting for FIPS only. Due to the way FIPS is configured with the elder
gradle test cluster framework this setting was getting applied to elder
clusters in BWC tests that did not have the settting causing test
failures.
The new test framework has better semantics for version specific
configuration. This commit updates applies the new setting via the new
framework with a version specific condition.
Adding this setting to the test clusters is a simple way to test the
setting (which will cause errors if the required providers are not found
in the cluster). The pseudo test does not care which framework is used
for configuration. Also, using the new framework allows to remove some
hacky configuration previously needed to handle some elder test cluster
configuration that used elder versions.
Fixes: https://github.com/elastic/elasticsearch/issues/104234
This PR adds the ability to update secure setting storage in internal
REST tests during test execution. This is useful when testing the
end-to-end flow of reloading secure settings via the [Nodes reload
secure settings
API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-reload-secure-settings.html).
Previously, re-applying secure settings was only possible with a full
test cluster restart.
- Replace docker-compose based s3-fixture usages to use plain java
- Support lazy evaluated system properties for local cluster definitions
- Make it work with configuration cache enabled
- Port Minio fixture usage to leverage test container testing library.
This commit upgrades the Bouncy Castle jars. Bouncy Castle is used for
some internal build concners as well as a comnand line application.
Most notably Bouncy Castle is also used as the FIPs certified JCE/JSEE provider
we use to test our ability to use a FIPs compliant crypto provider.
The following changes here are a result of the upgraded Bouncy Castle jars:
* TLSv1.3 is now supported when running in FIPs mode
* RSA PKCS#1 v1.5 is no longer allowed in FIPS mode
* Triple DES (3DES) is no longer allowed in FIPS mode
* Minor updates the security manager configuration used to test FIPs (to read permissions from the security provider)
* Minor adjustments to tests to accommodate the above changes.
* Minor adjustments to the gradle build to accommodate new dependencies
Note - update to the documentation will come in a later commit.
Adds the ability to configure a data stream to create a new kind of backing index called a failure store which will eventually be used to store error information when ingest pipelines fail to ingest a document or when a document fails to be parsed correctly by the configured mapping on the data stream.
This commit introduces the concept of shared tests clusters using the
new JUnit testing framework. Unlike normal test clusters which are
exclusive to each test suite (class), shared clusters persist across
test suites to be reused. This can be useful for test projects with a
large number of test suites that can all use a single cluster, and the
overhead of creating these clusters is the dominating factor.
* Avoid "this-escape" by making classes final
The "this-escape" compiler warning is intended to alert
developers to potential bugs in object initialization due to
subclassing. This class of bugs cannot occur when a class is
final. Here, we take cases where a class has no implementations
but generates a "this-escape" warning, and we make those
classes final rather than suppressing the compiler warning.
This makes the remaining suppressions more meaningful, since
they now indicate places where we may want to look for
initialization bugs.
In a few cases, making a class final meant changing some of its
protected fields and methods to private or default
accessibility.
Some classes with no implementations are mocked in testing.
Since making those classes final would involve non-trivial
rewrites of tests, I've left them alone.
* Spotless, remove redundant modifiers, clean up "protected" usage
* Revert a few more mocked classes
Another round of automated fixes to this, marking things that can be
made static as static. Saves some JIT cycles but also turns some lambdas
from capturing to non-capturing and makes the "utilityness" of some
classes visible.
Add support for passing `--debug-server-jvm` to tests that declare
multiple test clusters. This is common in tests for things like remote
cluster security, CCS, CCR, etc.
The convention for port numbers and debug settings is the same. The JVM
is expected to attach to an existing debugger in listen mode, starting
at port 5007 and incrementing from there.
Closes#94175
Adds @SuppressWarnings("this-escape") to all necessary places to that
Elasticsearch can compile with -Werror on JDK21
No investigation has been done to determine whether any of the cases
are a potential source of errors - we have simply suppressed all
existing occurrences.
Resolves: #99845
This commit updates the local cluster factory so that extra config files
(see `LocalSpecBuilder.configFile`) can be placed into subdirectories of
the config dir. This is needed when creating files for the
`FileSettingsService` - these k8s managed files are required to be in an
`"operator/"` directory within the general configuration directory.
This makes the data stream lifecycle generally available. This will allow
data streams to take advantage of a native simplified and resilient
lifecycle implementation.
