The new way to build is using 'rake' and not 'make'.
For some some attempt at compatibility, any 'make' invocation will
invoke rake. `make foo` will invoke `rake foo`.
This is for #1640
Major changes:
* rake is used instead of make. This allows us to more easily improve,
debug, and test the build tooling.
* The build process can be bootstrapped from any Ruby that has
Rake available. Bootstrapping rake is automatic will download JRuby and
re-execute itself under JRuby.
* rpm and deb packages no longer list any java as dependencies. This
is the result of user discomfort and discussions which concluded
that there is no good way to "depend" on Java via rpm or deb.
Internal changes:
* Vendoring paths are project-based. Each vendor directory is now
specific as vendor/<project>/... For example, JRuby is dumped into
vendor/jruby/...
Other changes:
* We don't use jruby-complete anymore. We now use bin/jruby as the entry point.
This improves the startup time of Logstash on my laptop by approximately
1.5 seconds (~25% speedup of startup time). Testing `bin/logstash
version` speed; 5 runs each:
* Before patch: 7.27 7.55 7.26 7.42 7.47
* After patch: 5.54 5.97 5.67 5.58 5.54
Smaller changes:
- removed 'env' subcommand
- Remove 'deps' subcommand. Replaced by 'rake vendor:all'
- Add gem_home method for this platform's GEM_HOME
- .travis.yml removed because we haven't used Travis in a long time.
- bin/logstash exits nonzero with a useful message if JRuby can't be found.
Notes:
Invoke bundler via system() call instead of Bundler::CLI.
After some discussion in #bundler on freenode (very helpful!) on how
to do what we need to do with bundler, I was encouraged to not use the
bundler api from ruby, so system() it is.
Patch slightly modified to fit the new model of packaging (ruby instead
of shell invocation), but original author is obazoud in #1008.
Richard Pijnenburg helped with some of the code and was lovely in
helping test it.
Fixes#1640
deprecated USE_JRUBY environment var
added USE_RUBY and USE_DRIP environment vars
centralized gem path handling in LogStash::Environment
use consistent discovery strategy
This fixes the following broken in #1189:
% bin/logstash
=> Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME.
This helps ensure we provide only our own gem directory for rubygems.
Helps improve LOGSTASH-1918. Before this patch, my local ruby install
was accidentally providing 'rack' so I never noticed this problem.
This was missing a / between the directory and the filename, so it was trying to
uncompress: /opt/logstash/logstash-1.4.0.beta1/vendor/logstashlogstash-contrib-1.4.0.beta1.tar.gz
Changed to uncompress ${TARGET}, which is defined as: ${TARGETDIR}/${FILENAME}
Drip allows for faster startup time by pre-starting JVMs and keeping a
pool of them around for use on-demand. More details here:
https://github.com/flatland/drip
Tested with this config:
* 'input { generator { count => 1 } } output { null { } }'
Without drip:
% time bin/logstash -e '...'
13.19s user 2.94s system 170% cpu 9.458 total
With drip (install drip, then set 'JAVACMD=drip' in env):
% time JAVACMD=drip bin/logstash -e '...'
0.03s user 0.46s system 6% cpu 7.008 total
2.5 seconds faster (27% faster)
'bundle install' when already installed should be faster than the
previous gembag. More importantly, it is also more accurate in terms of
dependencies.
Bonus points that we can do 'clean' which will purge any unknown gems
and keep our gems directory nice and tidy.
The main purpose is to speed up repetitive build invocations. Bundler's
dependency resolver is better than that of the previous gembag.rb, and
its 'clean' feature lets bundler manage the gems listing - even between
builds. This lets not have to 'rm -rf vendor' between builds on jenkins
to ensure quality testing :)
Other changes:
- Hide the Gemfile and Gemfile.lock in the 'tools' directory. This
should hopefully prevent users from accidentally updating these files
and thus avoiding merge conflicts later.
- Minor patch to use RbConfig::CONFIG['ruby_version'] instead of
RUBY_VERSION. Confusing, I know. The former is the 'ruby abi' version,
it seems, and the latter is the actual ruby implementation version.
Example, on MRI 1.9.3, the former is 1.9.1 and latter is 1.9.3. On
JRuby, the former is 1.9 and latter is 1.9.3. COMPUTERS. HELP.
- Have bin/logstash fetch jruby if ruby is not found.
- note redis 2.6 requirement for new batch_count on redis input
- randomize batch size
- only build-jruby if it's not there. This avoids the 'make: Nothing to
be done for ...' message
