* A logic to handle non-unicode payload in Logstash.
* Well tested and code organized version of the logic.
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
* Upgrade jrjackson to 0.4.20
* Code review: simplify the logic with a standard String#encode interface with replace option.
Co-authored-by: Ry Biesemeyer <ry.biesemeyer@elastic.co>
---------
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Co-authored-by: Ry Biesemeyer <ry.biesemeyer@elastic.co>
Adds section to describe the intended usage of and impact on memory sizing.
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
In the grammar definitions for hashes, `whitespace` was replaced with `cs` to allow either whitespace _or_ comments.
Additionally, the grammar definition for comments was previously required to end with a newline, now it can end with a newline _or_ EOF, using the "not anything" treetop rule `!.`.
Co-authored-by: Jonas Lundholm Bertelsen <jonas.lundholm.bertelsen@beumer.com>
Adds log.format.json.fix_duplicate_message_fields feature flag to rename the clashing fields when json logging format (log.format) is selected.
In case two message fields clashes on structured log message, then the second is renamed attaching _1 suffix to the field name.
By default the feature is disabled and requires user to explicitly enable the behaviour.
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
Bugfix to parse correctly Java options when the environment variable LS_JAVA_OPTS contains multiple definitions separated by space character.
Adapt the parsing of `LS_JAVA_OPTS` environment variable to split by space various definitions it can contains.
* Wipe out comment lines if config comment contains.
* Remove substitution var process when loading the YAML, instead align on the generic approach which LSCL happens during the pipeline compile.
* Update logstash-core/src/main/java/org/logstash/config/ir/PipelineConfig.java
Put the logging config back as it is being used with composed configs.
Introduce a new setting named `pipeline.buffer.type` which could be valued direct or heap to enable the allocation on Java heap.
The processing of the setting is done in `LogStash::Runner#execute` and sets the Java properties considered by Netty to disable the direct allocation: `io.netty.noPreferDirect`.
However, if that system property is already configured explicitly by the user (because set in `jvm.options`or `LS_JAVA_OPTS`) the setting doesn't take place and warning log is reported, respecting the user's will.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
* Replace 'git show-ref' with 'git rev-parse' to fix the issue where show-ref is not working as expected.
* Use git checkout instead 'git rev-parse'.
* Apply prune dependencies recommended for big projects (like we have multi gradle projects) by Snyk.
* Apply prune repeated dependency option directly to snyk monitor.
* Avoid the exit, continue scanning to the end.
* Remove the debugging.
This commit adds logic to copy the appropriate env2yaml file to the Docker image
* Clean up env2yaml folder
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
Change trigger from pull_request_target to pull_request, as the former uses the base branch instead of the PR source code.
This allows simplification of the checkout action (also took the opportunity to bump from v2 to v4).
* Save name came through ENV vars to let Logstash decide using either keystore or ENV value.
* Apply suggestions from code review to simplify array declaration.
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
---------
Co-authored-by: kaisecheng <69120390+kaisecheng@users.noreply.github.com>
Given that JRuby comes with ruby-maven-libs 3.3.9 this commit upgrades the gem to 3.8.9 and ensures files from 3.3.9 are not included in the distribution.
