Removes the usage of JAVA_HOME completely which is not anymore used for JDK path resolution.
Updated all the Logstash launching scripts to use only LS_JAVA_HOME as environment variable to
determine the JDK to use to launch Logstash. JAVA_HOME is abandoned for this job.
This commit modifies the launch scripts to take care of the LS_JAVA_HOME giving precedence over the JAVA_HOME, which is still used it the first is not found.
This PR enables the upgrade of bundler to the latest version.
Prior to this PR, the ability to do so was blocked by bundler.setup in versions of bundler > `2.23` making runtime changes to `Gemfile.lock` (unless the lock file was `frozen`) based on the specific platform the application was being run on, overriding any platforms (including generic `java` platform) set during build time. This was in conflict with changes made in #12782, which prevented the logstash user writing to files in `/usr/share/logstash`.
This PR will freeze the lockfile when logstash is run, and unfreeze it when manipulating plugins (install, update, remove, install from offline pack) to allow new plugins to be added. While unfrozen, changes are also made to ensure that the platform list remains as the generic `java` platform, and not changed to the specific platform for the runtime JVM.
This PR also introduces a new runtime flag, `--enable-local-plugin-development`. This flag is intended for use by Logstash developers only, and enables a mode of operation where a Gemfile can be manipulated, eg
```
gem "logstash-integration-kafka", :path => '/users/developer/code/plugins/logstash-integration-kafka'
```
to facilitate quick and simple plugin testing.
This PR also sets the `silence_root_warning` flag to avoid bundler printing out alarming looking warning messages when `sudo` is used. This warning message was concerning for users - it would be printed out during normal operation of `bin/logstash-plugin install/update/remove` when run under `sudo`, which is the expected mode of operation when logstash is installed to run as a service via rpm/deb packages.
This PR also updates the vagrant based integration tests to ensure that Logstash still runs after plugin update/install/remove operations, fixes up some regular expressions that would cause test failures, and removes some dead code from tests.
## Release notes
* Updated Bundler to latest version
* Ensured that `Gemfile.lock` are appropriately frozen
* Added new developer-only flag to facilitate local plugin development to allow unfrozen lockfile in a development environment
As we close in on the availability of 8.0.0 alphas, we are reassessing which
breaking changes are _necessary_, and which are merely _desired_. And while
we would love to be in a world where ECS was on by default, and have put
substantial effort into designing an upgrade path that would be as simple as
possible, we have determined that the time may not be right to change the
default value out of under our users.
This change restores the default value for `pipeline.ecs_compatibility` to
`disabled`, ensuring pipelines will continue running in Logstash 8 as they have
in Logstash 7 without modification. We will still encourage our users to be
explicit about which behaviour they desire, and will revisit making ECS on by
default at a later date.
* noop: avoid declaring default value in config file
* docs: ecs compatibility from 7.x perspective
Co-authored-by: Karen Metts <karen.metts@elastic.co>
* ecs: on by default
We know that ECS version 8 will release along-side Logstash 8.0, but its scope
is still coming into focus. In this changeset, we change the default value
of `pipeline.ecs_compatibility` from `disabled` to `v1`, which is a
significantly closer approximation to what will eventually ship in Logstash
8.0.0.
* docs: ecs from 8.x perspective
Co-authored-by: Karen Metts <karen.metts@elastic.co>
Co-authored-by: Karen Metts <karen.metts@elastic.co>
In the List a Plugin docs, we suggest users to create an issue to request we list community plugins. This commit improves that page by adding a template to create an issue.
- moved parsing of jvm.options file into Java code
- chnaged the parsing code to consider conditional notation to bind the applicability of certain JVM flags to specific JVM versions
- changed the launch scripts (.sh and .bat) to use the options string composition
- binded CMS flags to JVM specifications 8-14
* plugin: adds `:validate => :field_reference`
Provide plugins a way of validating that an input is a literal field-reference.
This is useful for input plugins that implement a `target` or other
non-interpolated directive, and allows these plugins to reject invalid
configuration before start-up instead of at run-time.
Plugins should not use this named validator directly, as doing so would cause
validation to fail with "Unknown validator" when the plugin is run on older
releases of Logstash. Instead, plugins should use the `validator_support`
adapter mixin that provides back-ports when necessary.
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
Restructures troubleshooting docs in preparation for expanding content.
Adds info for plugin tracing to help users track down plugins that might be causing problems.
Co-authored-by: João Duarte <jsvd@users.noreply.github.com>
Fixes: #12228
Moves Cloud info to Configuration section to make it more obvious and easier to find
Expands content for using cloud id and cloud auth outside of modules
Moves module-specific info into modules section
