With the addition of the qa acceptance test suite we have decide that we
don't need to maintain this suite also since the acceptance test are
running really similar test.
Fixes: #5744Fixes#5760
Purpose:
* manage releases through a minimum number of rake tasks
* simplify building of snapshot builds
* create staged artifacts, candidates for releases, that required no changes to become releases
* this means the snapshot release process will not involve publishing gems, therefore:
* the gem artifacts should only be published to rubygems as a final artifact, at the time of GA
Changes:
* release artifacts no longer depend on gems of core components
* all core components are used locally AS-IS to minimize code changes between snapshot, RC and GA
* `versions.yml` describes the versions of all logstash parts and package
* `rake version:set[version]` manage the yaml file and push the changes to the gemspecs/version.rb files
* `rake version:set_plugin_api[version]` manage the yaml file and push the changes to the gemspecs/version.rb files
* `rake artifact:all` generates SNAPSHOT artifacts: tar.gz, zip, rpm, deb
* `RELEASE=1 rake artifact:all` creates release candidate artifacts + 4 gems: logstash-core, logstash-core-event, logstash-core-event-java and logstash-core-plugin-api
implements #5416 and #5414Fixes#5460
This change introduce a new command ci/ci_acceptance which is
responsable of building the packages, bootstraping the acceptance test
environment, launching the VMs and running the tests.
You can use the following command to target specific platform.
```
ci/ci_acceptance.sh all
ci/ci_acceptance.sh centos
ci/ci_acceptance.sh debian
```
This PR also add a new rake task to build all the artifacts in a single
run called rake artifact:all it make sure we only install the plugin
only once and make the build process faster.
Fixes#5350
Update after-install scripts
Use /usr/lib/logstash instead of /opt/logstash
More revisions.
Using `/usr/share/logstash` instead of `/usr/lib/logstash`
Track these new files
More work in progress.
Proper arg parsing obtained
WIP: Use jvm.options like ES
Remove facter dependency
Due to an update from @jordansissel on pleaserun, facter is no longer necessary
Revert Gemfile to proper version
Latest work in progress.
I added a simple test to logstash.lib.sh to catch empty strings in java options.
Added `startup.options`, and touched up `jvm.options`
Edited the package scripts to make things play nice with new paths.
Tweaked `system-install` to allow users to provide their own `startup.options` file.
Patch up merge failure
Changes to work with the new logstash.yml
Add fpm dependency (build fails without it)
Fix missing gems in Gemfile
Don't know how these got lost
Use /etc/logstash for configuration in packages
Update the instructions/documentation for...
...startup.options
Merging proposed changes
Remove mustache and clamp
They're covered by pleaserun anyway
Prune config dir from /usr/share/logstash...
...but only when using RPM/DEB packages
Fixes#5341
* by default lives in LOGSTASH_HOME/config/logstash.yml
* location can be changed by $LS_SETTINGS_DIR or --path.settings
* overrides defaults of LogStash::Environment, BUT
* CLI flags override the yaml file
* several breaking changes to cli flag names (see config/logstash.yml)
* setting values are now type checked (see Setting and Settings classes)
Fixes#5313
5.0 is a chance for us to review and remove any plugins that we don't have to bundle by default. These plugins are still supported and maintained, but the default package should only contain most popular, most used plugins.
Removed plugins:
logstash-filter-anonymize: Replaced with newer logstash-filter-fingerprint
logstash-input-zeromq/output: Not a lot of users. Most users seem to prefer rabbitmq, redis, kafka as broker
Please comment if you'd like to remove more/less
Fixes#5254
Logstash uses by default in the shell script 1 gig of memory for the
JVM. but when we use rake it uses the default settings of JRUBY and
start a vm 500megs of memory. The recent changes in JRUBY and
JRuby-OpenSSL have increased the base memory and make the installing of
plugins with a OOM issues. This PR make sure the CI scripts and the rake
jruby launch uses the default of 1gig.
Fixes: #5179Fixes#5198
This PR introduce a unified naming scheme for all our artifacts and will
produce the following files:
logstash-5.0.0-alpha1_all.deb
logstash-5.0.0-alpha1.tar.gz
logstash-5.0.0-alpha1.zip
logstash-5.0.0-alpha1.noarch.rpm
Fix#5100Fixes#5125
ruby-maven ships a hidden directory called .mvn that
needs to be included in the package.
Currently the list of files to be packaged is generated using
Rake::FileList[globs] which excludes hidden files by default
(same happens with Dir.glob).
The fix is to ensure the contents of that directory are included
by explicitly referencing it.
For example, the file "extensions.xml", included in this directory,
is responsible for loading the ruby DSL for POMs. Not carrying this
file in the logstash package breaks any plugin update that uses
jar-dependencies.
Fixes#4845
format test properly for the node_spec api
Introduce a more proper organization of the dummy/mock runner to avoid
having wrong state propagaged across the different tests. This was
discovered by the unnecessary need to have the rubydebug codec as it was
due to wrong pipeline startup.
make the node stats test sync with last changes in data formats
improve web api service logic from reviews
remove the requirement of using output-null as only with output {} you get the same effect
Fixes#4782
Test improvements:
* Cleanup non used fixture.
* Improve node_stats tests by adding omre context and format checks.
* Add a new node_specs endpoint test that checks the expected data is
returned for the hot_threads resource.
Fixes#4782
cleaned dependencies, removed Guava code
default to java event
also compile java event
use json-unit for json result assertions
remove Java 8 specific constructs to be able to compile on Java 7
use jar-dependencies
better jar file loading
copy jar into lib after build
add post-build copy and clean of gem jar and re-indented file
fix jar destination
include generated jar in gem
remove useless ArrayList constructions
add tests for KeyNode.join method
remove useless shadow and disable unchecked warnings
update gradle dependency to 2.8
fix deprecated asserEquals usage
simplify convoluted list contruction
fix array fields nil values handling
moar nil value tests
simplify for loop
* move cli argument handling from agent to runner
* add a short-help message
* add interactive shell option
* log fatal messages to terminal when logging to file
* change docs:generate task to use bundle exec
Fixes#3872
including dependencies, and reuse them in an offline installation by
providing an package for it. It adds two important commands to the
plugin manager, the pack and upack, to handle package creation and
installation and adds the --local flag to install and update to pull
plugins from the installed local packages.
Former commits:
add a task to create a bundle, under vendor/cache of the installed gems + their dependencies, this can be used later on to be installed offline
add an option to pass the --local flag to the bin/plugin update task, so it fetch plugins from a local cache, by default under vendor/cache
rename package:build to package:build-cache as is more meaningfull
add a --local flag to bin/plugin install to users can install plugins from the local cache, under the default directory of vendor/cache
add a plugin manager command to build the local cache of installed plugins using bundler package alike command
It adds code for compressing and extracting in zip and tar formats to
the utils module of logstash. The zip module is only jruby dependant as
it uses functions provided by java.
There is also code add in the plugin manager package command to handle
compression of plugins dumping.
Cleanup the custom java code to compress and extract zip files as it has
been known that using rubyzip is good and it has a more ruby like
features so the code is more clean and portable.
set of smallish improvement requested during review
added several options to handle situation when the package of plugins we want to generate is already there
clean up old code
applyed feedback from review, mostly changed in documentating behaviour plus better wording for messages
relocate the Environment.windows? check to the bootstrap side as it's used also in the plugin manager
add an unpack bin/plugin command so users can install packages of plugins throw the plugin manager
document override behaviour in the compress helpers (for zip and tar) plus add a fix for the tar extract when reading entries
made the unpack filename parameter a required one
add a force option to the bin/plugin unpack command
add a notice to that if using a local cache the --local flag should be passed with
Code cleanup and refactor introduced during review
add two wording suggestions comming from review
ammend more wording
skip the major version validation as in situation where there is no internet connection like when using the local vendor/cache to do the update
move compress to the bootstrap environment as being used in the plugin manager means not being loaded with logstash-core
Bring pack cached gems in the vendor directory so they can be used for bundler when packaging dependencies
Revert "Bring pack cached gems in the vendor directory so they can be used for bundler when packaging dependencies"
This reverts commit a9d7f46649932b06efaafebdd0eed2b4c63c2235.
patch the Bundler::Source::Rubygems to fetch only gems from a remote source
small changes in several parts of the plugin manager and the creation of a common pack command with shared code
change compress to read in chuncks
fix wrong var name in the bootstrap compress utils module
fix namespacing conflicts
add basic test for the compress utility module
ammend more namespace issues
add a comment to the rubygems mockey patch to rebuild the gem cache
apply cosmetic changes
make the compress module raise CompressError
remove vendor_path and pattern_path form environment as they where mix up during rebase
change the bin/pack force option to be force-delete-cache
rename force_delete_cache to overwrite
change force for overwrite in tha pack command
remove the override option for bin/plugin unpack
revert Gemfile and Genfile.lock wrong committed
