JvmOptionsParser adds support for ${VAR:default} syntax when parsing jvm.options
- allow dynamic resolution of environment variables in the jvm.options file
- enables fallback to default value when the environment variable is not set
(cherry picked from commit ef36df6b81)
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
When Logstash 8.12.0 added increased Jackson stream read constraints to
jvm.options, assumptions about the existence of that file's contents
were invalidated. This led to issues like #16683.
This change ensures Logstash applies defaults from config at runtime:
- MAX_STRING_LENGTH: 200_000_000
- MAX_NUMBER_LENGTH: 10_000
- MAX_NESTING_DEPTH: 1_000
These match the jvm.options defaults and are applied even when config
is missing. Config values still override these defaults when present.
(cherry picked from commit cc608eb88b)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
Because a `break` escapes a `begin`...`end` block, we must not use a `break` in order to ensure that the explicitly set value gets memoized to avoid lock contention.
> ~~~ ruby
> def fake_sync(&block)
> puts "FAKE_SYNC:enter"
> val = yield
> puts "FAKE_SYNC:return(#{val})"
> return val
> ensure
> puts "FAKE_SYNC:ensure"
> end
>
> fake_sync do
> @ivar = begin
> puts("BE:begin")
> break :break
>
> val = :ret
> puts("BE:return(#{val})")
> val
> ensure
> puts("BE:ensure")
> end
> end
> ~~~
Note: no `FAKE_SYNC:return`:
> ~~~
> ╭─{ rye@perhaps:~/src/elastic/logstash (main ✔) }
> ╰─● ruby break-esc.rb
> FAKE_SYNC:enter
> BE:begin
> BE:ensure
> FAKE_SYNC:ensure
> [success]
> ~~~
(cherry picked from commit 01c8e8bb55)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Moves the application of jackson defaults overrides into pure java, and
applies them statically _before_ the `org.logstash.ObjectMappers` has a chance
to start initializing object mappers that rely on the defaults.
We replace the runner's invocation (which was too late to be fully applied) with
a _verification_ that the configured defaults have been applied.
(cherry picked from commit 202d07cbbf)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
A new transative dependency on the `logger` gem has been added through sinatra 4.1.0. Update the
license checker to ensure this is accounted for.
(cherry picked from commit e0ed994ab1)
Co-authored-by: Cas Donoghue <cas.donoghue@gmail.com>
* pipeline bus: add deadlock test for unlisten/unregisterSender
* pipeline bus: eliminate deadlock
Moves the sync-to-notify out of the `AddressStateMapping#mutate`'s effective
synchronous block to eliminate a race condition where unlistening to an address
and unregistering a sender could deadlock.
It is safe to notify an AddressState's attached input without exclusive access
to the AddressState, because notifying an input that has since been disconnected
is net zero harm.
Updates release notes for `8.14.x` to call for an update to a subsequent minor for fix performance regression in JSON decoding.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
(cherry picked from commit 52836f8caf)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
This fixes the offline install problem of the logstash-input-java_filter_example off-line install.
(cherry picked from commit 113585d4a5)
Co-authored-by: Nicole Albee <2642763+a03nikki@users.noreply.github.com>
The DRA build failed because the required jars were missing, as they had been removed during the Docker build process.
(cherry picked from commit 00da72378b)
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
there's no need to build jdk-less and windows tarballs for docker images
so this change simplifies the build process.
It should reduce the time spent needed to build docker images.
(cherry picked from commit 9eced9a106)
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
Switch execution agent of JDK availability check pipeline from vm-agent to container-agent.
Moves the schedule definition from the `Logstash Pipeline Scheduler` pipeline into the pipeline definition, adding a schedule also for `8.x` branch.
(cherry picked from commit c602b851bf)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
Updates the existing `createElasticCatalogDownloadUrl` method to use the precise version retrieved `versions.yml` to download the JDK instead of using the latest of major version. This makes the build reproducible again.
Defines a new Gradle `checkNewJdkVersion` task to check if there is a new JDK version available from JVM catalog matching the same major of the current branch.
Creates a new Buildkite pipeline to execute a `bash` script to run the Gradle task; plus it also update the `catalog-info.yaml` with the new pipeline and a trigger to execute every week.
(cherry picked from commit ed5874bc27)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
The current max inflight error message focuses on a single pipeline and on a maximum amount of 10k events regardless of the heap size.
The new warning will take into account all loaded pipelines and also consider the heap size, giving a warning if the total number of events consumes 10% or more of the total heap.
For the purpose of the warning events are assumed to be 2KB as it a normal size for a small log entry.
(cherry picked from commit ca19f0029e)
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
- refactor deprecated alias to support obsoleted version
- add deprecation log for http.* config
(cherry picked from commit 3f0ad12d06)
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
Fixes the behaviour of the tokenizer to be able to work properly when buffer full conditions are met.
Updates BufferedTokenizerExt so that can accumulate token fragments coming from different data segments. When a "buffer full" condition is matched, it record this state in a local field so that on next data segment it can consume all the token fragments till the next token delimiter.
Updated the accumulation variable from RubyArray containing strings to a StringBuilder which contains the head token, plus the remaining token fragments are stored in the input array.
Furthermore it translates the `buftok_spec` tests into JUnit tests.
(cherry picked from commit 85493ce864)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
Adds a JMH benchmark to measure the peformances of BufferedTokenizerExt.
Update also Gradle build script to remove CMS GC flags and fix deprecations for Gradle 9.0.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
(cherry picked from commit b6f16c8b81)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
