We have instances where BWC tests configure old ES version nodes with
the integTest distribution. This isn't a valid configuration, and while
we in reality resolve the default distribution artifact, we have other
configuration logic that behaves differently based on whether the
integTest distro was _requested_. Specifically, what to set ES_JAVA_HOME
to. This bug resulted in us attempting to run old nodes using the
current bundled JDK version, which may be incompatible with that older
version of Elasticsearch.
Closes#104858
Grants the SocketPermission "*", "connect" permission to the
bouncy castle JSSE implementation lib, when running ITs under
FIPS-configured JVMs.
Note that the plugin's permissions e.g.
x-pack/plugin/ml-package-loader/src/main/plugin-metadata/plugin-security.policy
are still checked and enforced.
This new permission is required because the BC lib hooks into
the JVM when opening any TLS connection, under the stacktrace
of the plugin's AccessController.doPrivileged (so the lib's permissions
are checked first, before the doPrivileged call on the stack).
Relates #100923Fixes#104414
https://github.com/elastic/elasticsearch/pull/103483 introduced a new
setting for FIPS only. Due to the way FIPS is configured with the elder
gradle test cluster framework this setting was getting applied to elder
clusters in BWC tests that did not have the settting causing test
failures.
The new test framework has better semantics for version specific
configuration. This commit updates applies the new setting via the new
framework with a version specific condition.
Adding this setting to the test clusters is a simple way to test the
setting (which will cause errors if the required providers are not found
in the cluster). The pseudo test does not care which framework is used
for configuration. Also, using the new framework allows to remove some
hacky configuration previously needed to handle some elder test cluster
configuration that used elder versions.
Fixes: https://github.com/elastic/elasticsearch/issues/104234
This PR adds the ability to update secure setting storage in internal
REST tests during test execution. This is useful when testing the
end-to-end flow of reloading secure settings via the [Nodes reload
secure settings
API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-reload-secure-settings.html).
Previously, re-applying secure settings was only possible with a full
test cluster restart.
- Replace docker-compose based s3-fixture usages to use plain java
- Support lazy evaluated system properties for local cluster definitions
- Make it work with configuration cache enabled
- Port Minio fixture usage to leverage test container testing library.
This commit upgrades the Bouncy Castle jars. Bouncy Castle is used for
some internal build concners as well as a comnand line application.
Most notably Bouncy Castle is also used as the FIPs certified JCE/JSEE provider
we use to test our ability to use a FIPs compliant crypto provider.
The following changes here are a result of the upgraded Bouncy Castle jars:
* TLSv1.3 is now supported when running in FIPs mode
* RSA PKCS#1 v1.5 is no longer allowed in FIPS mode
* Triple DES (3DES) is no longer allowed in FIPS mode
* Minor updates the security manager configuration used to test FIPs (to read permissions from the security provider)
* Minor adjustments to tests to accommodate the above changes.
* Minor adjustments to the gradle build to accommodate new dependencies
Note - update to the documentation will come in a later commit.
Adds the ability to configure a data stream to create a new kind of backing index called a failure store which will eventually be used to store error information when ingest pipelines fail to ingest a document or when a document fails to be parsed correctly by the configured mapping on the data stream.
This commit introduces the concept of shared tests clusters using the
new JUnit testing framework. Unlike normal test clusters which are
exclusive to each test suite (class), shared clusters persist across
test suites to be reused. This can be useful for test projects with a
large number of test suites that can all use a single cluster, and the
overhead of creating these clusters is the dominating factor.
* Avoid "this-escape" by making classes final
The "this-escape" compiler warning is intended to alert
developers to potential bugs in object initialization due to
subclassing. This class of bugs cannot occur when a class is
final. Here, we take cases where a class has no implementations
but generates a "this-escape" warning, and we make those
classes final rather than suppressing the compiler warning.
This makes the remaining suppressions more meaningful, since
they now indicate places where we may want to look for
initialization bugs.
In a few cases, making a class final meant changing some of its
protected fields and methods to private or default
accessibility.
Some classes with no implementations are mocked in testing.
Since making those classes final would involve non-trivial
rewrites of tests, I've left them alone.
* Spotless, remove redundant modifiers, clean up "protected" usage
* Revert a few more mocked classes
Another round of automated fixes to this, marking things that can be
made static as static. Saves some JIT cycles but also turns some lambdas
from capturing to non-capturing and makes the "utilityness" of some
classes visible.
Add support for passing `--debug-server-jvm` to tests that declare
multiple test clusters. This is common in tests for things like remote
cluster security, CCS, CCR, etc.
The convention for port numbers and debug settings is the same. The JVM
is expected to attach to an existing debugger in listen mode, starting
at port 5007 and incrementing from there.
Closes#94175
Adds @SuppressWarnings("this-escape") to all necessary places to that
Elasticsearch can compile with -Werror on JDK21
No investigation has been done to determine whether any of the cases
are a potential source of errors - we have simply suppressed all
existing occurrences.
Resolves: #99845
This commit updates the local cluster factory so that extra config files
(see `LocalSpecBuilder.configFile`) can be placed into subdirectories of
the config dir. This is needed when creating files for the
`FileSettingsService` - these k8s managed files are required to be in an
`"operator/"` directory within the general configuration directory.
This makes the data stream lifecycle generally available. This will allow
data streams to take advantage of a native simplified and resilient
lifecycle implementation.
This adds a new parameter to LocalClusterSpecBuilder.user(..) to
indicate whether the user should be an operator or not.
Users added with the simplified user(username, password) method are
operators (and have the "_es_test_root" role).
For RCS 2.0, we added a RemoteClusterNodesAction (#93893) for collecting
remote cluster server nodes. This action is used in two places: 1.
Collecting nodes that can be directly connected in sniff mode 2. In the
method RemoteClusterService#collectNodes
However, in the 2nd use case, we should collect all ndoes from the
remote cluster instead of just those nodes that have remote cluster
server enabled. This is because the method is used in the context of
search instead of building connection. A remote node can be *not*
directly accessible but still hosts searchable data. Search requests
should be sent to all the data nodes and *not* limited to just the
server nodes.
This PR fixes this issue by augmenting RemoteClusterNodesAction to
support retrieving either server nodes or all nodes. The caller can
decide which set of nodes are of interest depending on the context.
Fix for #97334 where incorrect feature name was provided.
Correct more instances of synonyms_feature_flag_enabled for synonyms_api_feature_flag_enabled
Closes#96641, #97177
Sometimes it is necessary for an integ test to poke at the Java process
of Elasticsearch. This commit makes a node's pid available through
ClusterHandle.
Somet special tests like die-with-dignity need to pass additional jvm
arguments (exit on OOM). This commit adds a new jvmArg to the cluster
spec for the new test clusters infra.
This PRs adds migration tests for transform and CCR jobs from using RCS
1.0 to 2.0 then back to 1.0 again. It is not intended to cover all
possible scenarios, but rather a proof to show migration is possible in
at least some cases.
Note the migration strategy here is to update remote cluster settings in
place, i.e. reuse the same cluster alias and change its definition.
Restart is required because a node needs to the cross-cluster API key
form the keystore.
