- This hasnt been maintained in a while and the vagrant gradle plugin also broke compatibiliy for gradle
--configuration-cache. Also this removes a lot maintenance burden.
- Rework DistroTestPlugin and simplify task dependencies
- Rename :qa:os to :qa:packaging
- Update testing doc recommending buildkite tools for debugging packaging tests
This commit adds the ability to initialize YAML rest test suites against
a subset of available test cases. Previously, the only way to do this is
via the `tests.rest.suite` system property, but that can only be set at
the test _task_ level. Configuring this at the test _class_ level means
that we can support having multiple test suite classes that execute
subsets of tests within a project. That allows for things like
parallelization, or having different test cluster setups for different
YAML tests within the same project.
For example:
```java
@ParametersFactory
public static Iterable<Object[]> parameters() throws Exception {
return ESClientYamlSuiteTestCase.createParameters(new String[] { "analysis-common", "indices.analyze" });
}
```
The above example would mean that only tests in the `analysis-common`
and `indices.analyze` directories would be included in this suite.
cc @jdconrad
Closes#95089
This commit adds TLS for the transport layer and optional support
for https for `./gradlew run` and `./gradlew run-ccs` tasks.
A new system property --https can be passed to expose https.
Transport layer TLS is always enabled which is always applicable
for `./gradlew run-css`, but only applicable for `./gradlew run` when multiple nodes are configured.
Additional certificates and instructions for use are also included in the commit.
This commit reverts the changes introduced via https://github.com/elastic/elasticsearch/pull/89442 / 20ed7e3fd9
and re-implements the same goals but with fewer changes. Also included in this commit
is the ability to change the proxyMode via system property and mention of this
new run task in the testing documentation.
This commit makes minor changes to support multiple nodes
via ./gradlew run. The specific desired use case, where there 2 nodes
in the cluster where one is a coordinating only node is used as
the example configuration.
The instructions in `TESTING.asciidoc` for how to disable assertions
when running `./gradlew run` didn't work. Fix the docs by describing
how to pass JVM args, using disabling assertions as an example.
We were depending on the BouncyCastle FIPS own mechanics to set
itself in approved only mode since we run with the Security
Manager enabled. The check during startup seems to happen before we
set our restrictive SecurityManager though in
org.elasticsearch.bootstrap.Elasticsearch , and this means that
BCFIPS would not be in approved only mode, unless explicitly
configured so.
This commit sets the appropriate JVM property to explicitly set
BCFIPS in approved only mode in CI and adds tests to ensure that we
will be running with BCFIPS in approved only mode when we expect to.
It also sets xpack.security.fips_mode.enabled to true for all test clusters
used in fips mode and sets the distribution to the default one. It adds a
password to the elasticsearch keystore for all test clusters that run in fips
mode.
Moreover, it changes a few unit tests where we would use bcrypt even in
FIPS 140 mode. These would still pass since we are bundling our own
bcrypt implementation, but are now changed to use FIPS 140 approved
algorithms instead for better coverage.
It also addresses a number of tests that would fail in approved only mode
Mainly:
Tests that use PBKDF2 with a password less than 112 bits (14char). We
elected to change the passwords used everywhere to be at least 14
characters long instead of mandating
the use of pbkdf2_stretch because both pbkdf2 and
pbkdf2_stretch are supported and allowed in fips mode and it makes sense
to test with both. We could possibly figure out the password algorithm used
for each test and adjust password length accordingly only for pbkdf2 but
there is little value in that. It's good practice to use strong passwords so if
our docs and tests use longer passwords, then it's for the best. The approach
is brittle as there is no guarantee that the next test that will be added won't
use a short password, so we add some testing documentation too.
This leaves us with a possible coverage gap since we do support passwords
as short as 6 characters but we only test with > 14 chars but the
validation itself was not tested even before. Tests can be added in a followup,
outside of fips related context.
Tests that use a PKCS12 keystore and were not already muted.
Tests that depend on running test clusters with a basic license or
using the OSS distribution as FIPS 140 support is not available in
neither of these.
Finally, it adds some information around FIPS 140 testing in our testing
documentation reference so that developers can hopefully keep in
mind fips 140 related intricacies when writing/changing docs.
It can be difficult to know how to run release builds. Certain changes require that tests run without a snapshot build (consider testing a feature behind a feature flag).
This adds a small doc line specifying what args to pass to run tests without a snapshot build.
In the past, the only way to run a local Elasticsearch build with a remote debugger was by extracting elasticsearch and passing ES_JAVA_OPTS. However, since switching to gradle, a convenience flag was added, `--debug-jvm` (which is documented elsewhere in the testings docs), when running a local elasticsearch build through gradle. This commit removes the old documentation.
This commit creates a new Gradle plugin to provide a separate task name
and source set for running YAML based REST tests. The only project
converted to use the new plugin in this PR is distribution/archives/integ-test-zip.
For which the testing has been moved to :rest-api-spec since it makes the most
sense and it avoids a small but awkward change to the distribution plugin.
The remaining cases in modules, plugins, and x-pack will be handled in followups.
This plugin is distinctly different from the plugin introduced in #55896 since
the YAML REST tests are intended to be black box tests over HTTP. As such they
should not (by default) have access to the classpath for that which they are testing.
The YAML based REST tests will be moved to separate source sets (yamlRestTest).
The which source is the target for the test resources is dependent on if this
new plugin is applied. If it is not applied, it will default to the test source
set.
Further, this introduces a breaking change for plugin developers that
use the YAML testing framework. They will now need to either use the new source set
and matching task, or configure the rest resources to use the old "test" source set that
matches the old integTest task. (The former should be preferred).
As part of this change (which is also breaking for plugin developers) the
rest resources plugin has been removed from the build plugin and now requires
either explicit application or application via the new YAML REST test plugin.
Plugin developers should be able to fix the breaking changes to the YAML tests
by adding apply plugin: 'elasticsearch.yaml-rest-test' and moving the YAML tests
under a yamlRestTest folder (instead of test)
This commit creates a new gradle plugin to provide a separate task name
and source set for running ESIntegTestCase tests. The only project
converted to use the new plugin in this PR is server, as an example. The
remaining cases in x-pack will be handled in followups.
This commit makes a number of improvements when importing the
Elasticsearch project into IntelliJ IDEA. Specifically:
- Contributing documentation has been updated to reflect that the
'idea' task should no long be used and Gradle project import is
instead the officially supported way of setting up the project.
- Attempts to run the 'idea' task will result in a failure with a
message directing folks to our CONTRIBUTING.md document.
- The project JDK is explicit set rather that using whatever JAVA_HOME
is.
- Gradle build operation delegation is disabled, and test execution is
configured to 'choose per test'.
- Gradle is configured to inherit the project JDK.
- Some code style conventions are automatically configured.
- File encoding is explicitly set to UTF-8.
- Parallel module compilation is enabled and deprecated feature
warnings are disabled.
- A remote debug run configuration using listen mode is created.
- JUnit runner is configured with required system properties.
- License headers are configured such that Apache 2 is the default
notice added to all source files with exception of source in /x-pack
which will use the Elastic license.
Update documentation for:
* restResources config (related #52114)
* call out YAML vs. Java based Rest tests
* update example to use newer syntax
* update example to target a test that is not skipped
* provide example for bwcRest test (related #52383)
If there are environmental reasons why docker-compose can't bring up
containers in the :distribution:docker project, it's not that clear from
the command line output. Add healthchecks to the compose file so that
the cluster's health is explicitly checked.
Also add a note about Docker memory requirements to the testing docs.
Before this change one needed to re-start debugging several times, as we
launched multiple JVMs in debug mode.
With this option the IDE has the option to re-launch and listen for
connections again leading for to a more pleasant experience.
* Update testing document for new packaging tests
The TESTING.asciidoc document had gotten out of date due to some new and
wonderful changes in our vagrant testing code. I've removed all of the
instructions that no longer work, and added working examples and descriptions
in their place.
This commit adds a variant for every official distribution that omits
the bundled jdk. The "no-jdk" naming is conveyed through the package
classifier, alongside the platform. Package tests are also added for
each new distribution.
