github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-06-28 17:34:17 -04:00
Code Issues Projects Releases Packages Wiki Activity

Update testing guidelines (#45617)

Browse source 
The testing guidelines didn't reflect current Gradle usage, so I've
updated them and added a link to the relevant part of the Gradle docs.
This commit is contained in:
Rory Hunter 2019-08-16 09:25:24 +01:00 • committed by GitHub
parent 32318b0c92
commit 915dd7afbc
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 12 additions and 19 deletions
Download patch file Download diff file Expand all files Collapse all files

31
TESTING.asciidoc
View file

@ -76,28 +76,21 @@ In order to set an Elasticsearch setting, provide a setting with the following p
=== Test case filtering. === Test case filtering.
- `tests.class` is a class-filtering shell-like glob pattern, You can run a single test, provided that you specify the Gradle project. See the documentation on
- `tests.method` is a method-filtering glob pattern. https://docs.gradle.org/current/userguide/userguide_single.html#simple_name_pattern[simple name pattern filtering].
Run a single test case (variants) Run a single test case in the `server` project:
---------------------------------------------------------- ----------------------------------------------------------
./gradlew test -Dtests.class=org.elasticsearch.package.ClassName ./gradlew :server:test --tests org.elasticsearch.package.ClassName
./gradlew test "-Dtests.class=*.ClassName"
---------------------------------------------------------- ----------------------------------------------------------
Run all tests in a package and its sub-packages Run all tests in a package and its sub-packages:
---------------------------------------------------- ----------------------------------------------------
./gradlew test "-Dtests.class=org.elasticsearch.package.*" ./gradlew :server:test --tests 'org.elasticsearch.package.*'
---------------------------------------------------- ----------------------------------------------------
Run any test methods that contain 'esi' (like: ...r*esi*ze...)
-------------------------------
./gradlew test "-Dtests.method=*esi*"
-------------------------------
Run all tests that are waiting for a bugfix (disabled by default) Run all tests that are waiting for a bugfix (disabled by default)
------------------------------------------------ ------------------------------------------------
@ -118,7 +111,7 @@ Every test repetition will have a different method seed
(derived from a single random master seed). (derived from a single random master seed).
-------------------------------------------------- --------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName ./gradlew :server:test -Dtests.iters=N --tests org.elasticsearch.package.ClassName
-------------------------------------------------- --------------------------------------------------
=== Repeats _all_ tests of ClassName N times. === Repeats _all_ tests of ClassName N times.
@ -127,7 +120,7 @@ Every test repetition will have exactly the same master (0xdead) and
method-level (0xbeef) seed. method-level (0xbeef) seed.
------------------------------------------------------------------------ ------------------------------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.seed=DEAD:BEEF ./gradlew :server:test -Dtests.iters=N -Dtests.seed=DEAD:BEEF --tests org.elasticsearch.package.ClassName
------------------------------------------------------------------------ ------------------------------------------------------------------------
=== Repeats a given test N times === Repeats a given test N times
@ -137,14 +130,14 @@ ie: testFoo[0], testFoo[1], etc... so using testmethod or tests.method
ending in a glob is necessary to ensure iterations are run). ending in a glob is necessary to ensure iterations are run).
------------------------------------------------------------------------- -------------------------------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.class=*.ClassName -Dtests.method=mytest* ./gradlew :server:test -Dtests.iters=N --tests org.elasticsearch.package.ClassName.methodName
------------------------------------------------------------------------- -------------------------------------------------------------------------
Repeats N times but skips any tests after the first failure or M initial failures. Repeats N times but skips any tests after the first failure or M initial failures.
------------------------------------------------------------- -------------------------------------------------------------
./gradlew test -Dtests.iters=N -Dtests.failfast=true -Dtestcase=... ./gradlew test -Dtests.iters=N -Dtests.failfast=true ...
./gradlew test -Dtests.iters=N -Dtests.maxfailures=M -Dtestcase=... ./gradlew test -Dtests.iters=N -Dtests.maxfailures=M ...
------------------------------------------------------------- -------------------------------------------------------------
=== Test groups. === Test groups.
@ -175,7 +168,7 @@ systemProp.tests.jvms=8
---------------------------- ----------------------------
Its difficult to pick the "right" number here. Hypercores don't count for CPU Its difficult to pick the "right" number here. Hypercores don't count for CPU
intensive tests and you should leave some slack for JVM-interal threads like intensive tests and you should leave some slack for JVM-internal threads like
the garbage collector. And you have to have enough RAM to handle each JVM. the garbage collector. And you have to have enough RAM to handle each JVM.
=== Test compatibility. === Test compatibility.