Breaking changes for 9.0 (#17380) (#17480)

(cherry picked from commit 3c6cbbf35b)

Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
This commit is contained in:
mergify[bot] 2025-04-02 13:33:15 -04:00 committed by GitHub
parent 6c1fcdb304
commit 0d162a533b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -196,6 +196,79 @@ Weve removed deprecated SSL settings in some {{ls}} plugins, and have replace
::::
### Pipeline buffer type defaults to `heap`[pipeline-buffer-type]
We've improved memory configuration for certain {{ls}} plugins.
Input plugins such as `elastic_agent`, `beats`, `tcp`, and `http` allocate buffers in Java memory to read events from the network.
The default allocation method is `direct` memory rather than `heap` memory to simplify configuration, and to help facilitate debugging memory usage problems through the analysis of heap dumps.
If you need to re-enable the previous behavior, change the `pipeline.buffer.type` setting in [logstash.yml](/reference/logstash-settings-file.md).
Check out [off-heap-buffers-allocation](/reference/jvm-settings.md#off-heap-buffers-allocation) for details. [#16500](https://github.com/elastic/logstash/pull/16500)
### {{ls}} modules removed [removed-modules]
We have removed the {{ls}} modules framework, and encourage users to try Elastic Integrations
This includes the netflow, azure and arcsight modules, and the modules framework as a whole. [#16794](https://github.com/elastic/logstash/pull/16794)
### Deprecated configuration settings removed [removed-params]
We have removed support for previously deprecated configuration settings:
#### `http.*` prefixed settings for the {{ls}} API
When configuring the {{ls}} API, settings prefixed by `http.*` have been replaced by the equivalent settings prefixed with `api.*`. [#16552](https://github.com/elastic/logstash/pull/16552)
#### `event_api.tags.illegal`
Any events that include field named tags automatically rename the field _tags to avoid any clash
with the reserved {{ls}} tags field.
Instead, {{ls}} generates `_tagsparsefailure` in the event `tags` and the illegal value is written to the `_tags` field. [#16461](https://github.com/elastic/logstash/pull/16461)
### Ingest converter removed [removed-ingest-converter]
The ingest converter, which was used to convert ingest pipelines to {{ls}} pipelines, has been removed. [#16453](https://github.com/elastic/logstash/pull/16453)
The `logstash-filter-elastic_integration` plugin offers similar functionality, and can help you use [Logstash to extend Elastic integrations](/reference/using-logstash-with-elastic-integrations.md).
### JDK11 not supported [jdk-11-support-drop]
JDK17 is the minimum version of the JDK required to run Logstash.
For the best experience, we still recommend running {{ls}} using the bundled-jdk.
See [Logstash JVM requirements](/reference/getting-started-with-logstash.md#ls-jvm)
for details. [#16443](https://github.com/elastic/logstash/pull/16443)
### Docker base image now UBI9 based [docker-base-image-change]
The base image for {{ls}} on Docker has been changed from Ubuntu to UBI9.
If you create a Docker image based on the {{ls}} image and rely on it being Ubuntu based, you need to change your derived image to take account of this change.
For example, if your derived docker image installs additional packages using a package manager, UBI9 uses `microdnf`, rather than `apt`.
[#16599](https://github.com/elastic/logstash/pull/16599)
### Cannot run {{ls}} as `superuser` by default [disallow-superuser]
We've changed the default behavior to prevent users from accidentally running {{ls}} as a superuser.
If you try to run {{ls}} as a superuser, it logs an error and fails to start, ensuring that users cannot run Logstash with elevated privileges by accident.
You can change the value of the `allow_superuser` setting to `true` in [logstash.yml](/reference/logstash-settings-file.md) if you want to restore the previous behavior and allow {{ls}} to run with superuser privileges. [#16558(https://github.com/elastic/logstash/pull/16558)
### New setting required to continue using legacy internal monitoring [allow-legacy-monitoring]
To continue using deprecated internal collection to monitor {{ls}}, set `xpack.monitoring.allow_legacy_collection` to `true` in [logstash.yml](/reference/logstash-settings-file.md).
We encourage you to move to [agent-driven monitoring](/reference/monitoring-logstash-with-elastic-agent.md), the latest, supported way to monitor Logstash [#16586](https://github.com/elastic/logstash/pull/16586)
### Avoiding JSON log lines collision [avoid-collision-on-json-fields]
We've improved the way we deal with duplicate `message` fields in `json` documents.
Some code paths that log in `json` produce log events that include multiple instances of the `message` field. (The JSON codec plugin is one example.)
While this approach produces JSON that is technically valid, many clients do not parse this data correctly, and either crash or discard one of the fields.
We recently introduced the option to fix duplicates, and made it the default behavior for `9.0` and later.
To re-enable the previous behavior, change the `log.format.json.fix_duplicate_message_fields` setting in [logstash.yml](/reference/logstash-settings-file.md) to `false`.
Check out [Logging in json format can write duplicate message fields](docs-content://troubleshoot/ingest/logstash.md) for more details about the issue. [#16578](https://github.com/elastic/logstash/pull/16578)
### Enterprise_search integration plugin is deprecated [enterprise_search-deprecated-9.0]
Weve deprecated the {{ls}} Enterprise_search integration plugin, and its component App Search and Workplace Search plugins. These plugins will receive only security updates and critical fixes moving forward.