a 50ms delay helps creating more full batches without practical added latency see #8707
a 64mb page helps PQ perfmance related to a large page size see #8702#8707
A regression was introduced in the new LSCL/LIR implementation in 6.0, in
which configurations that had nested hashes produced valid treetop-ASTs that
could not produce a valid LIR; the values in an `LSCL::AST::Hash` were
assumed to implement `ValueExpression`, but `LSCL::AST::Hash#expr` produced
an unwrapped `RubyHash`.
By making `LSCL::AST::Hash#expr` emit a `ValueExpression` _representing_ a
`RubyHash`, and relying on the existing code that deals with `ValueExpression`
appropriately, we can re-enable deeply-nested hashes in configs.
Resolves: elastic/logstash#8701
Resolves: logstash-plugins/logstash-output-http#77
Resolves: sw-jung/logstash-filter-memoize#1
Fixes#8750
As more java code is being used in logstash, it's likely that
java.lang.Threads which aren't set as daemon may be running when
`exit(1)` is called.
Because System.exit only raises a SystemExit exception, it's possible
that threads will continue to execute, and if they're not set as daemon
then the JVM won't terminate.
By calling `System.exit!` instead we ensure jruby calls the underlying
java.lang.System.exit and logstash terminates immediately.
Fixes#8729
This is support of a adding additional command line entry points that also need to settings (logstash.yml) file. Since there is a distinct before/read/after life-cycle this utility provides some direction for additional command line entry points without duplicating code and logic.
Part of #8353Fixes#8700
This is in support of a future change to also support substitution from the secret store in addition to the environment variables.
Part of #8353Fixes#8699
A [build failure][] of the grok plugin when run through LIR/lscl indicates that
there is an expectation for multiple Attributes of the same name to be merged
together.
This commit ports the failing spec in the plugin to an abstraction that can be
tested within logstash-core, and adds a caveat to the LSCL LIR-builder to
ensure that we merge the hashes in a way that is compatible with legacy
behaviour.
NOTE: when multiple Attributes of the same name are used in a single config,
it's possible to create configurations that circumvent `AST::Hash`'s ability to
report duplicate keys.
[build failure]: 293778268Fixes#8597
Adds a new INFO-level log message at Logstash startup indicating that
Logstash is starting, including a structured-log key `logstash.version`
populated with the current logstash version.
This message will only be emitted if the runner gets far enough to actually
start up (e.g., if it is not passed a flag that causes it to bail early)
Resolves elastic/logstash#8512
Fixes#8521
If modules configuration has elasticsearch credentials set, but not
kibana, attempt to use elasticsearch credentials for Kibana login.
Fixes#8422Fixes#8482
When X-Pack is installed an internal (hidden) pipeline named .monitoring-logstash is also running. This means that for X-Pack is there N +1 pipelines where N = user defined pipelines. The code currently only supports monitoring 1 pipeline, and it is possible that the internal pipeline gets picked up instead of the intended pipeline.
When multiple user pipelines are enabled (6.0+), the code still only supports monitoring 1 pipeline.
The change here is to ignore the system pipelines, and monitor as many user defined pipelines that may exist.
Fixes: #8382Fixes#8472
final (I hope) fixes for consolidated versioning.
After testing with rake artifact:zip, need to try both files.
Add desc to rake test:install-* tasks, tired of
not seeing them in rake -vT
changes requested via review
Fixes#8373
Password settings for Modules supplied via a yml file were being wrapped inside a
LogStash::Utils::Password object twice during startup - once in from_yaml, and once
in validate. Disallow this from happening by skipping the wrap if the password is
already wrapped
This should complete the fix for #8224Fixes#8258
