Updates release notes for `8.14.x` to call for an update to a subsequent minor for fix performance regression in JSON decoding.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
make a note of the fix that went to 8.13.1: #16026
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
(cherry picked from commit 03841cace3)
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
* pq: avoid blocking writer when queue is precisely full
A PQ is considered full (and therefore needs to block before releasing the
writer) when its persisted size on disk _exceeds_ its `queue.max_bytes`
capacity.
This removes an edge-case preemptive block when the persisted size after
writing an event _meets_ its `queue.max_bytes` precisely AND its current
head page has insufficient room to also accept a hypothetical future event.
Fixes: elastic/logstash#16172
* docs: PQ `queue.max_bytes` cannot be less than `queue.page_capacity`
(cherry picked from commit ea930861ef)
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
As the module is not maintained since 2018 and it was community supported, I would like to remove it from the documentation.
(cherry picked from commit 53d9480176)
Co-authored-by: Luca Belluccini <luca.belluccini@elastic.co>
Adds log.format.json.fix_duplicate_message_fields feature flag to rename the clashing fields when json logging format (log.format) is selected.
In case two message fields clashes on structured log message, then the second is renamed attaching _1 suffix to the field name.
By default the feature is disabled and requires user to explicitly enable the behaviour.
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
(cherry picked from commit 830733d758)
Co-authored-by: Andrea Selva <selva.andre@gmail.com>
Introduce a new setting named `pipeline.buffer.type` which could be valued direct or heap to enable the allocation on Java heap.
The processing of the setting is done in `LogStash::Runner#execute` and sets the Java properties considered by Netty to disable the direct allocation: `io.netty.noPreferDirect`.
However, if that system property is already configured explicitly by the user (because set in `jvm.options`or `LS_JAVA_OPTS`) the setting doesn't take place and warning log is reported, respecting the user's will.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
* Describe how to troubleshoot connectivity problems at HTTP level with Azure Blob Storage
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
This commit added support to add and remove multiple keystore keys in a single operation. It also fixed the empty value validation for editing existing key values and added ASCII validation for values.
* Update the Logstash to Logstash Native doc to reflect the multiple hosts usage.
* Logstash to Logstash comm page, adding LS-to-LS native HA support.
* Apply suggestions from code review
Refining the context.
Co-authored-by: Andres Rodriguez <andreserl@gmail.com>
---------
Co-authored-by: Andres Rodriguez <andreserl@gmail.com>
* geoip: extract database manager to stand-alone feature
Introduces an Elastic-licensed GeoipDatabaseManagement tool that can be used
by ANY plugin running on Elastic-licensed Logstash to retrieve a subscription
to a GeoIP database that ensures EULA-compliance and frequent updates, and
migrates the previous Elastic-licensed code-in-Logstash-core extension to
the Geoip Filter to use this new tool, requiring ZERO changes to in-the-wild
versions of the plugin.
The implementation of the new tool follows the previous implementation as
closely as possible, but presents a new interface that ensures that a
consumer can ATOMICALLY subscribe to a database path without risk that the
subscriber will receive an update or expiry before it is finished applying
the initial value:
~~~ ruby
geoip_manager = LogStash::GeoipDatabaseManagement::Manager.instance
subscription = geoip_manager.subscribe('City')
subscription.observe(construct: ->(initial_dbinfo){ },
on_update: ->(updated_dbinfo){ },
on_expire: ->( _ ){ })
subscription.release!
~~~
* docs: link in geoip database manager docs
* docs: reorganize pending 'geoip database management' feature
* docs: link to geoip pages from feature index
* geoip: add SubscriptionObserver "interface"
simplifies using Subscription#observe from Java
* geoip: fixup SubscriptionObserver after rename
* geoip: quacking like a SubscriptionObserver is enough
* geoip: simplify constants of legacy geoip filter extension
* geoip: bump logging level to debug for non-actionable log
* geoip: refine log message to omit non-actionable info
* re-enable invokedynamic (was disabled to avoid upstream bug)
* geoip: resolve testing fall-out from filter extension's "private" constants removal
* geoip: consistently use `DataPath#resolve` internally, too
* Add known imap and email plugin issues section to Logstash 8.10+ versions.
Co-authored-by: Mashhur <mashhur.sattorov@elastic.co>
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
(cherry picked from commit a88f82e77f)
