* Update patch plugin versions in gemfile lock
* pull in minors from ES input and filter
* remove addition of java 17 platform
already covered by `java` platform
* pull in minor from http input to get fixed netty thread names
---------
Co-authored-by: logstashmachine <43502315+logstashmachine@users.noreply.github.com>
Co-authored-by: Ry Biesemeyer <ry.biesemeyer@elastic.co>
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Backport PR #16968 to 8.16 branch, original message:
----
Permit to use effectively the tokenizer also in context where a line is bigger than a limit.
Fixes an issues related to token size limit error, when the offending token was bigger than the input fragment in happened that the tokenzer wasn't unable to recover the token stream from the first delimiter after the offending token but messed things, loosing part of tokens.
## How solve the problem
This is a second take to fix the processing of tokens from the tokenizer after a buffer full error. The first try #16482 was rollbacked to the encoding error #16694.
The first try failed on returning the tokens in the same encoding of the input.
This PR does a couple of things:
- accumulates the tokens, so that after a full condition can resume with the next tokens after the offending one.
- respect the encoding of the input string. Use `concat` method instead of `addAll`, which avoid to convert RubyString to String and back to RubyString. When return the head `StringBuilder` it enforce the encoding with the input charset.
(cherry picked from commit 1c8cf546c2)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
This commit removed irrelevant warning for internal pipeline, such as monitoring pipeline.
Monitoring pipeline is expected to be one worker. The warning is not useful
Fixes: #13298
(cherry picked from commit 3f41828ebb)
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
Non clean backport of #16959 from 8.x to 8.16 (which is a non clean backport of #16576 to 8.x)
Changes respect to 8.x:
- in 8.x the `with_deprecated_alias` receives also a parameter specifying the version where the setting is removed. In this case we removed the extra parameter to the method a0378c05cb/logstash-core/lib/logstash/environment.rb (L79)
- fixed the test removing the check for the remove version a0378c05cb/logstash-core/spec/logstash/settings/setting_with_deprecated_alias_spec.rb (L157)
----
Reimplements `LogStash::Setting::String` Ruby setting class into the `org.logstash.settings.SettingString` and exposes it through `java_import` as `LogStash::Setting::SettingString`.
Updates the rspec tests in two ways:
- logging mock is now converted to real Log4J appender that spy log line that are later verified
- verifies `java.lang.IllegalArgumentException` instead of `ArgumentError` is thrown because the kind of exception thrown by Java code, during verification.
* Fixed the rename of NullableString to SettingNullableString
* Fixed runner test to use real spy logger from Java Settings instead of mock test double
Backport of #16907 cherry picked from 9385cfa
This commit uses the new --qualifier parameter in the release manager
for publishing dra artifacts. Additionally, simplifies the expected
variables to rely on a simple `VERSION_QUALIFIER`.
Snapshot builds are skipped when VERSION_QUALIFIER is set.
Finally, for helping to test DRA PRs, we also allow passing the `DRA_BRANCH` option/env var
to override BUILDKITE_BRANCH.
Closes https://github.com/elastic/ingest-dev/issues/4856
handle maven output that can carry "garbage" information after the jar's name. this patch deletes that extra information, also pins psych to 5.2.2 until jruby ships with snakeyaml-engine 2.9 and jar-dependencies 0.5.2
Related to: https://github.com/jruby/jruby/issues/8579
(cherry picked from commit 52b7fb0ae6)
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
When Logstash 8.12.0 added increased Jackson stream read constraints to
jvm.options, assumptions about the existence of that file's contents
were invalidated. This led to issues like #16683.
This change ensures Logstash applies defaults from config at runtime:
- MAX_STRING_LENGTH: 200_000_000
- MAX_NUMBER_LENGTH: 10_000
- MAX_NESTING_DEPTH: 1_000
These match the jvm.options defaults and are applied even when config
is missing. Config values still override these defaults when present.
(cherry picked from commit cc608eb88b)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
Because a `break` escapes a `begin`...`end` block, we must not use a `break` in order to ensure that the explicitly set value gets memoized to avoid lock contention.
> ~~~ ruby
> def fake_sync(&block)
> puts "FAKE_SYNC:enter"
> val = yield
> puts "FAKE_SYNC:return(#{val})"
> return val
> ensure
> puts "FAKE_SYNC:ensure"
> end
>
> fake_sync do
> @ivar = begin
> puts("BE:begin")
> break :break
>
> val = :ret
> puts("BE:return(#{val})")
> val
> ensure
> puts("BE:ensure")
> end
> end
> ~~~
Note: no `FAKE_SYNC:return`:
> ~~~
> ╭─{ rye@perhaps:~/src/elastic/logstash (main ✔) }
> ╰─● ruby break-esc.rb
> FAKE_SYNC:enter
> BE:begin
> BE:ensure
> FAKE_SYNC:ensure
> [success]
> ~~~
(cherry picked from commit 01c8e8bb55)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Moves the application of jackson defaults overrides into pure java, and
applies them statically _before_ the `org.logstash.ObjectMappers` has a chance
to start initializing object mappers that rely on the defaults.
We replace the runner's invocation (which was too late to be fully applied) with
a _verification_ that the configured defaults have been applied.
(cherry picked from commit 202d07cbbf)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Also update license checker with new logger dependency (#16695) (#16700)
A new transative dependency on the `logger` gem has been added through sinatra 4.1.0. Update the
license checker to ensure this is accounted for.
(cherry picked from commit e0ed994ab1)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
---------
Co-authored-by: logstashmachine <43502315+logstashmachine@users.noreply.github.com>
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
* pipeline bus: add deadlock test for unlisten/unregisterSender
* pipeline bus: eliminate deadlock
Moves the sync-to-notify out of the `AddressStateMapping#mutate`'s effective
synchronous block to eliminate a race condition where unlistening to an address
and unregistering a sender could deadlock.
It is safe to notify an AddressState's attached input without exclusive access
to the AddressState, because notifying an input that has since been disconnected
is net zero harm.
(cherry picked from commit 8af6343a26)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Fixes the behaviour of the tokenizer to be able to work properly when buffer full conditions are met.
Updates BufferedTokenizerExt so that can accumulate token fragments coming from different data segments. When a "buffer full" condition is matched, it record this state in a local field so that on next data segment it can consume all the token fragments till the next token delimiter.
Updated the accumulation variable from RubyArray containing strings to a StringBuilder which contains the head token, plus the remaining token fragments are stored in the input array.
Furthermore it translates the `buftok_spec` tests into JUnit tests.
(cherry picked from commit 85493ce864)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
