* Remove tech preview from agent driven LS monitoring pages.
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
(cherry picked from commit ae3b3ed17c)
Co-authored-by: Mashhur <99575341+mashhurs@users.noreply.github.com>
* Pin rubocop-ast development gem due to new dep on prism (#17407)
The rubocop-ast gem just introduced a new dependency on prism.
- https://rubygems.org/gems/rubocop-ast/versions/1.43.0
In our install default gem rake task we are seeing issues trying to build native
extensions. I see that in upstream jruby they are seeing a similar problem (at
least it is the same failure mode https://github.com/jruby/jruby/pull/8415
This commit pins rubocop-ast to 1.42.0 which is the last version that did not
have an explicit prism dependency.
(cherry picked from commit 6de59f2c02)
* Bump rubocop-ast
We are pinning rubocop-ast to the last version that did not require prism.
It is a development gem and should not change any behavior of LS.
---------
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
Updates only test code to be able to run a test that consumes big memory if:
- the physical memory is bigger than the requested Java heap
- JDK version is greater than or equal to 21.
The reason to limit the JDK version is that on 16GB machine the G1GC is more efficient than the one on previous JDKs and so let complete the test with 10GB heap, while in JDK 17 it consistently fails with OOM error.
(cherry picked from commit 075fdb4152)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
Replace removed Elasticsearch output SSL settings with the latest values
(cherry picked from commit f705a9de48)
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
Use long instead of int type to keep the length of the first token.
The size limit validation requires to sum two integers, one with the length of the accumulated chars till now plus the next fragment head part. If any of the two sizes is close to the max integer it generates an overflow and could successfully fail the test 9c0e50faac/logstash-core/src/main/java/org/logstash/common/BufferedTokenizerExt.java (L123).
To fall in this case it's required that sizeLimit is bigger then 2^32 bytes (2GB) and data fragments without any line delimiter is pushed to the tokenizer with a total size close to 2^32 bytes.
(cherry picked from commit afde43f918)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
* tests: make integration split quantity configurable
Refactors shared splitter bash function to take a list of files on stdin
and split into a configurable number of partitions, emitting only those from
the currently-selected partition to stdout.
Also refactors the only caller in the integration_tests launcher script to
accept an optional partition_count parameter (defaulting to `2` for backward-
compatibility), to provide the list of specs to the function's stdin, and to
output relevant information about the quantity of partition splits and which
was selected.
* ci: run integration tests in 3 parts
(cherry picked from commit 3e0f488df2)
Co-authored-by: Rye Biesemeyer <yaauie@users.noreply.github.com>
* ci: add shareable script for generating qualified version
* ci: use shared script to generate qualified version
(cherry picked from commit 10b5a84f84)
Co-authored-by: Rye Biesemeyer <yaauie@users.noreply.github.com>
* Upgrade elasticsearch-ruby client. (#17161)
* Fix Faraday removed basic auth option and apply the ES client module name change.
(cherry picked from commit e748488e4a)
* Update elasticsearch-ruby client in gemfile lock.
---------
Co-authored-by: Mashhur <99575341+mashhurs@users.noreply.github.com>
Co-authored-by: Mashhur <mashhur.sattorov@elastic.co>
The https://github.com/elastic/logstash/pull/17310 PR changed the rake task for
artifact creation to use shellwords from standard library. The acceptance tests
need to explitily load that library. This commit updates the rake file to handle
loading the required code.
(cherry picked from commit 964468f922)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
Previously when rake would shell out the output would be lost. This
made debugging CI logs difficult. This commit updates the stack with
improved message surfacing on error.
(cherry picked from commit 0d931a502a)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
A recent change to pqheck, attempted to address an issue where the
pqcheck would not on Windows mahcines when located in a folder containing
a space, such as "C:\program files\elastic\logstash". While this fixed an
issue with spaces in folders, it introduced a new issue related to Java options,
and the pqcheck was still unable to run on Windows.
This PR attempts to address the issue, by removing the quotes around the Java options,
which caused the option parsing to fail, and instead removes the explicit setting of
the classpath - the use of `set CLASSPATH=` in the `:concat` function is sufficient
to set the classpath, and should also fix the spaces issue
Fixes: #17209
(cherry picked from commit ba5f21576c)
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
migrate benchmark to the new artifacts-api
(cherry picked from commit d61a83abbe)
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
* Additional cleanify changes to ls2ls integ tests: replace heartbeat-input with reload option, set queue drain to get consistent result.
(cherry picked from commit 1e06eea86e)
Co-authored-by: Mashhur <99575341+mashhurs@users.noreply.github.com>
* tests: ls2ls delay checking until events have been processed
* Make sure upstream sends expected number of events before checking the expectation with downstream. Remove unnecessary or duplicated logics from the spec.
* Add exception handling in `wait_for_rest_api` to make wait for LS REST API retriable.
---------
Co-authored-by: Mashhur <mashhur.sattorov@elastic.co>
Co-authored-by: Mashhur <99575341+mashhurs@users.noreply.github.com>
(cherry picked from commit 73ffa243bf)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
This commit refactors the `PersistedQueueConfigValidator` class to provide a
more detailed, accurate and actionable warning when pipeline's PQ configs are at
risk of running out of disk space. See
https://github.com/elastic/logstash/issues/14839 for design considerations. The
highlights of the changes include accurately determining the free resources on a
filesystem disk and then providing a breakdown of the usage for each of the
paths configured for a queue.
(cherry picked from commit 062154494a)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
This commit migrates gradle task to the new artifacts-api
- remove dependency on staging artifacts
- all builds use snapshot artifacts
- resolve version from current branch, major.x, previous minor,
with priority given in that order.
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
(cherry picked from commit 0a745686f6)
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
Fixed an issue where the `/_node/stats` API displayed empty pipeline metrics
when X-Pack monitoring was enabled
(cherry picked from commit 86785815bd)
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
* build full docker image from dockerfiles during docker acceptance tests
(cherry picked from commit fecfc7c602)
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
* qa: don't bypass plugin manger tests on linux
* add gradle task to build gem fixtures for integration tests
(cherry picked from commit 0f81816311)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
This commit adds Windows 2025 to the Windows JDK matrix and exhaustive tests pipelines.
(cherry picked from commit 4d52b7258d)
Co-authored-by: Dimitrios Liappis <dimitrios.liappis@gmail.com>
In https://github.com/elastic/logstash/pull/17125 jvm setup was redirected to
stderr to avoid polluting stdout. This test was actually having to do some
additional processing to parse that information. Now that we have split the
destinations the tests can be simplified to look for the data they are trying to
validate on the appropriate stream.
(cherry picked from commit 227c0d8150)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
routes output from setup-related functions to stderr, so that stdout can
include only the output of the actual program.
(cherry picked from commit 91258c3f98)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
This commit updates the acceptance tests to expect messages in the updated
format for removing plugins. See https://github.com/elastic/logstash/pull/17030
for change.
(cherry picked from commit e8e24a0397)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
