Commit graph

119 commits

Author SHA1 Message Date
Nhat Nguyen
86c1fa2a6c
Avoid convert to string when parse resp in heap attack (#105109)
We've seen cases of OOM errors in the test runner process, which occur 
when we convert a response to a JSON string and then parse it. We can
directly parse from its input stream to avoid these OOM errors.
2024-02-05 07:16:25 -08:00
Ryan Ernst
b67f5a6b57
Make cluster feature predicate available to plugins (#105022)
A predicate to check whether the cluster supports a feature is available
to rest handlers defined in server. This commit adds that predicate to
plugins defining rest handlers as well.
2024-02-01 09:11:18 -08:00
Moritz Mack
dbf59c5414
Update/Cleanup references to old tracing.apm.* legacy settings in favor of the telemetry.* settings (#104917) 2024-01-31 09:20:05 +01:00
David Turner
1116889819
Remove unused arg from ActionType ctor (#104650)
`ActionType` represents an action which runs on the local node, there's
no need for implementations to define a `Reader<Response>`. This commit
removes the unused constructor argument.
2024-01-25 03:28:32 -05:00
Bogdan Pintea
0aeb9beb7e
Lower the number of evaluation in testTooManyEval() (#104697)
This reverts the increase in #104521, which causes the parser to
sometimes stack overflow.

Fixes #104694.
2024-01-24 08:50:02 -05:00
Iraklis Psaroudakis
9aefad826e
Mute testTooManyEval (#104695)
Relates #104694
2024-01-24 06:46:38 -05:00
Nhat Nguyen
668ae505ca
Fix request timeout in HeapAttack tests (#104336)
I noticed we're using 5 minutes for both query timeout and triggering 
the out-of-memory action in heap attack tests. This means when we're
generating the heap dump, and some ESQL tasks might get canceled because
the connection was disconnected. This PR increases the query timeout to
6 minutes instead.
2024-01-22 11:27:39 -08:00
Bogdan Pintea
1e32c09cef
ESQL: minor heap-attack tests updates (#104521)
This updates two tests:  * it chunks the bulk loading of "many longs",
which can "choke" some test environments otherwise. * it increases the
number of the "too many evals", which otherwise can be served in some
environments and not trigger a breaker.
2024-01-18 13:58:13 -05:00
Przemyslaw Gomulka
e1fe7f48ba
Fix the assertion in MetricApmIT (#104420)
The test should print out the assertions that were not matched if it has not managed to finish within 30s
relates #103286
2024-01-18 11:49:13 +01:00
Simon Cooper
59e8aa887f
Rename transport version constant used for 8.10 releases (#104403) 2024-01-17 10:48:00 +00:00
Nhat Nguyen
aa1a5138fe
Fail fast on heap attack tests (#104328)
We can't use assume after a test fails.
2024-01-12 09:39:31 -08:00
Bogdan Pintea
08c9332350
ESQL: Update moved heap-attack QA suite's build group (#104312)
This updates ESQL's `heap-attack` QA suite's build group.
2024-01-12 17:23:40 +01:00
Nhat Nguyen
52bd9e895c
Unmute HeapAttack#testHugeManyConcat (#104241)
The heap attack recent failures are because of #104240. Analyzing 2000 
fields took more than 5 minutes in CI. With the fix in #104246, we can
enable this test now.

Relates #10424
2024-01-11 10:08:03 -08:00
Simon Cooper
a8d0e9c71d Update after merge conflict on #103277 2024-01-11 12:56:14 +00:00
Simon Cooper
016c778321
Remove NamedWriteableRegistry from NodeClient, pass it directly through to rest actions (#103277) 2024-01-11 12:42:22 +00:00
Nhat Nguyen
61ff4bb79a AwaitsFix #104241 2024-01-10 16:57:12 -08:00
Mark Vieira
142d4df606
Simplify handling of runtime java for tests clusters (#104232) 2024-01-10 14:37:38 -08:00
Nhat Nguyen
c14e300471
Skip heap attack tests in release builds (#104228)
We need to add a test plugin in the heap attack tests. However, we are 
not loading test plugins in the release builds; hence, we need to skip
this suite.
2024-01-10 12:22:02 -08:00
Nhat Nguyen
709c0f5d31
Re-enable HeapAttackIT (#104107)
This PR enables ESQL heap attack tests. I have run this suite over 500 
iterations with different hardware configurations for the last two days,
and all have been successful with the changes in #104159.

Additionally, this PR adds an action that can trigger OOM to generate a 
heap dump if a test takes more than 5 minutes. I've seen cases
(previously with our CI) where the test didn't result in OOM but was
taking too long. Having the ability to inspect the heap in such cases
would be beneficial.

Closes #103527
Closes #100678
2024-01-10 08:11:23 -08:00
Przemyslaw Gomulka
07780a8282
Add metric name validation (#103388)
This commit adds a minimum metric name validation which checks:

metric name starts with es. prefix
metric name is using . as a separator of elements
metric name is using characters from a white list
validate min number of elements = 3 elements ( prefix, group and the suffix name)
validate max number of elements and max characters per element
validate the suffix element in a metric name to be from the enumerated allow list

It also modifies existing metric names to adhere to those rules
2024-01-09 11:05:00 +01:00
Armin Braun
4cfeb4aa8d
Fix more SearchResponse leaks (#104069)
Some more obvious leaks in tests fixed.
2024-01-08 12:14:44 -05:00
Armin Braun
21386c784f
Fix all "lambda can be folded" warnings in the codebase (#104020)
Just some obvious cleanup.
2024-01-08 12:51:30 +01:00
Armin Braun
1a848eb52e
Fix more search response leaks in tests (#103957)
Some more fixes as part of https://github.com/elastic/elasticsearch/issues/102030
2024-01-05 11:14:04 +01:00
Simon Cooper
f35590f770
Collapse transport versions 8500040+ into 8500061 (#103448) 2023-12-19 09:12:47 +00:00
Przemyslaw Gomulka
9d27c2fc5a
Enable MetricsAPMIT and change the assertion that logs on failure (#103405)
this commit enables (temporarely) the test to gather some additional logging
relates #103286

Co-authored-by: Moritz Mack <moritz@mackmail.net>
2023-12-14 11:21:59 +01:00
David Turner
4455ef875f AwaitsFix for #103286 2023-12-11 15:43:16 +00:00
Yang Wang
b9c29807ec
Extract repositories metrics into its own class (#103034)
This PR is a follow up of
https://github.com/elastic/elasticsearch/pull/102505#discussion_r1402957598
that move the repositories metrics management into its own class which
is then passed around instead of relying on the raw meterRegistry and
string metric names.
2023-12-07 18:26:17 -05:00
ChrisHegarty
fa4057df82 Merge 'origin/main' into lucene_snapshot 2023-11-29 11:27:46 +00:00
Przemyslaw Gomulka
25dec2b70a
Fix wrong metric type in a test (#102729) 2023-11-29 10:04:48 +01:00
Przemyslaw Gomulka
bb6442579a
Add apm api for asynchronous counters (always increasing) (#102598)
asynchronous counters (available in otel) are instruments which allow to update the counter's value with the callback style api. The callback has to report always the latest value. Upon restart the value might go back to 0, but in the backend apm server will know that since the value is always increasing.
2023-11-28 20:08:05 +01:00
Przemyslaw Gomulka
2d75217d8f
Update APM agent version (#102691)
this commit upgrades APM agent version to 1.44 to include a fix to a max metric name being increased from 63 to 255
2023-11-28 13:18:39 +01:00
Luca Cavanna
1aed4e606f Merge branch 'main' into lucene_snapshot 2023-11-27 10:25:56 +01:00
Armin Braun
1b84ea7421
Delete all unused private methods (#98111)
Pretty straight forward dead-code cleanup I think. Just delete all
private methods or methods in private classes that aren't used.
2023-11-25 22:21:59 +01:00
Luca Cavanna
7c9e8356e6 Merge branch 'main' into lucene_snapshot 2023-11-24 09:57:22 +01:00
Armin Braun
cdc83ad29b
Add shorthand for prepareIndex to test infrastructure (#101187)
Same as #101175, shorten `client().prepareIndex(index)` and
`client().prepareIndex().setIndex(index)` via a test utility.
Saves lots of code now and sets up some follow-up simplifcations.
2023-11-23 15:47:36 +01:00
elasticsearchmachine
cfba82d7e0 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2023-11-09 11:01:20 +00:00
Przemyslaw Gomulka
60c7a661bf
Add Tracing APM integration test (#101887)
this commit adds an integration test where ES is run with
apm-java-agent and sending traces.
Traces are then received by a fake http server and could be asserted on.
2023-11-08 17:38:22 +01:00
elasticsearchmachine
2daa7ede78 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2023-11-07 11:01:07 +00:00
Przemyslaw Gomulka
461d004591
Fix parsing NDJson in RecordingApmServer (#101825)
depending on operating system a System.lineSeparator is different however when reading a stream always with UTF_8 it will always be '\n'

This commit fixes the parsing of the ndjson that was failing on windows

closes #101793
2023-11-06 16:18:04 +01:00
Luca Cavanna
8592eebaa9 Merge branch 'main' into lucene_snapshot 2023-11-03 14:16:26 +01:00
Przemyslaw Gomulka
ea8f78e732
Add Apm Integration test (#101379)
this commit adds an integration test for APM module. In these tests we want to be independent to metrics generated by server, to avoid accidental failures when a code in server is refactored. 
This test is implemented as a separate gradle module with a test plugin that is exposing a rest endpoint and executes metrics registration & reporting.
2023-11-03 10:07:41 +01:00
elasticsearchmachine
c1e4ff08e3 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2023-11-01 10:01:18 +00:00
Ryan Ernst
d0064250ad
Forbid non-named threads (#101631)
This commit forbids the use of Thread constructors that do not take a
name. In general nameless threads are more difficult to understand their
purpose when debugging. Note that this is only added to production
signatures. Tests are not forbidden here so as not to be pedantic (or
require a larger change since many tests create anonymous threads).

relates #101628
2023-10-31 15:18:32 -07:00
elasticsearchmachine
0b3ecc28af Merge remote-tracking branch 'origin/main' into lucene_snapshot 2023-10-27 10:01:10 +00:00
Simon Cooper
22381fd6a7
Refactor overrides of old Plugin.createComponents method to new services method (#101381) 2023-10-26 16:58:14 +01:00
Luca Cavanna
416edc5181 Merge branch 'main' into lucene_snapshot 2023-10-24 12:41:36 +02:00
David Turner
1eda6ac74b
Extract ESIntegTestCase#prepareSearch (#101175)
Relates #101172
2023-10-20 06:18:58 -04:00
elasticsearchmachine
351e725087 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2023-10-19 10:01:16 +00:00
David Turner
3ce905b754
Remove more unnecessary ActionType subclasses (#101053)
Relates #97721
2023-10-18 13:56:40 +01:00
elasticsearchmachine
5f0e20cf00 Merge remote-tracking branch 'origin/main' into lucene_snapshot 2023-10-17 10:01:24 +00:00