This commit adds a rake task `rake artifact:dockerfile_ironbank` to generate ironbank docker build context for automatic release.
The output can be found in build/logstash-ironbank-$VERSION-docker-build-context.tar.gz
Co-authored-by: Rob Bavey <rob.bavey@elastic.co>
Remove an useful dynamic creation of appender's log file which leverages the `log.format` property
also when it's explicit by the appender itself.
Log4j configuration leverages the placeholder `${sys:ls.log.format}` to compose the name of the log file.
This generates some not evident conflicts in log4j internals, these conflicts became evident when enabling the `pipeline.separate_logs` feature is enabled and the log4j appender definitions contains both json and plain format.
The problem is that under those circumstances the rollover of the log file doesn't happen.
This commit also add a test against the production log4j configuration, to avoid future regressions.
Remove hard coded alias definitions in favor of yaml descriptor file.
Introduce a single point of aliases definition (logstash-core/src/main/resources/org/logstash/plugins/AliasRegistry.yml), checksum and copy it around to be used by Logstash and by Logstash's plugin management tool.
The descriptor yml file contains a checksum to verify it's not changed accidentally in a deployment of Logstash, if the verification phase fail Logstash avoid to start and plugin management tool avoid to operate.
The signing and copying around is managed by a specific Gradle task invoked during the build.
Co-authored-by: Ry Biesemeyer <yaauie@users.noreply.github.com>
Fixes#12831
introduces two rake tasks: `rake artifact:docker_oss` and `rake artifact:docker`, which will create the docker images of the OSS and non OSS packages. These tasks depend on the tar artifacts being built.
Also `rake artifact:all` has been modified to also call these two tasks.
most code was moved from https://github.com/elastic/logstash-docker/
Fixes#8657
Part 1: API and JavaKeyStore implementation (#8657)
Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.
Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (#8659)
* Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support
* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.
Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (#8745)
* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy
Part 4: Integrate secret store with Logstash core (#8905)
This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.
Part 5: Hardening and test fixes (this PR)
Fixes#8935
final (I hope) fixes for consolidated versioning.
After testing with rake artifact:zip, need to try both files.
Add desc to rake test:install-* tasks, tired of
not seeing them in rake -vT
changes requested via review
Fixes#8373
Work done by @guyboertje and @ph
Since JRuby 1.7.25 is now EOL we are migrating Logstash to use JRuby 9k and JDK8 only,
Not much needed updating to make this work, its was mostly a drop in replacement from the previous version.
The major point was the change in the implementation of Time in JRuby, JRuby now use `java.time`
instead of joda time, this allow JRuby to have nanoseconds precision on time object.
This PR is based on the proposal from #6846, it does the following
- Change the format of the plugin help page
- Create an external file for each plugin type that will get included by the plugin
- Add a script to generate the PR on each plugin repo with the static doc
- Add a script `ci/ci_docs.sh` that will be used to generate the list of plugins that can be installed in logstash from all the available plugins from the community organization
This PR is the building block for change in https://github.com/elastic/docs-tools/issues/3Fixes#6941
A pack in this context is a *bundle* of plugins that can be distributed outside of rubygems; it is similar to what ES and kibana are doing, and
the user interface is modeled after them. See https://www.elastic.co/downloads/x-pack
**Do not mix it with the `bin/logstash-plugin pack/unpack` command.**
- it contains one or more plugins that need to be installed
- it is self-contains with the gems and the needed jars
- it is distributed as a zip file
- the file structure needs to follow some rules.
- As a reserved name name on elastic.co download http server
- `bin/plugin install logstash-mypack` will check on the download server if a pack for the current specific logstash version exist and it will be downloaded, if it doesn't exist we fallback on rubygems.
- The file on the server will follow this convention `logstash-mypack-{LOGSTASH_VERSION}.zip`
- As a fully qualified url
- `bin/plugin install http://test.abc/logstash-mypack.zip`, if it exists it will be downloaded and installed if it does not we raise an error.
- As a local file
- `bin/plugin install file:///tmp/logstash-mypack.zip`, if it exists it will be installed
Fixes#6168
* add RubyJavaObject and Valuefier classes and use, junit and rspec pass
* add benchmark
* update benchmark
Deep fetch 6.015M (± 4.4%) i/s - 60.086M
* change warmup to 30 seconds
* support for rubyValue = RubyNil, shortcut null when asking for javaValue
* prevent Lists and Maps from being converted to RubyJavaValue too
* camel case: is_ruby_nil
* rename method, use renamed method, make serializable and add junit test
* encapsulate refs to RubyJavaObject to setableValue, use StringBuilder
* remove unused newObj
* some fixes after PR review
* wip
* ruby and java tests pass with BiValues.
* Benchmarks are still good
* Fixes for spec failures in normal event specs
* integrate JavaProxy support into BiValues
* edit a TODO
* rebase master and updates from PR review.
* revert the use of JavaUtil.convertJavaToUsableRubyObject
* allow initTimestamp to handle NullBiValue
* remove TimeBiValue, convert Joda DateTime and Ruby Time to Timestamp beforehand
This resolves a few inconsistencies and issues in the generated packages
for Debian/Ubuntu:
* Creates /etc/logstash/conf.d
* Installs a basic defaults file in /etc/default/logstash (which also sets HOME)
* Stops creating /var/run/logstash (unused)
* Uses /var/lib/logstash as the home directory of the logstash user instead of /home/logstash
* Defaults package files to be owned by root, fixes ownership in post-install script
* Properly stops service when removing the package on Debian
The end result is that after installing, LogStash will pretty much "just
work" with only 2 changes:
1) Enable START=yes in /etc/default/logstash
2) Provide a config for the LogStash agent
