Commit graph

32 commits

Author SHA1 Message Date
Armin
dff953d881 JAVA EntryPoint
* Created `org.logstash.Logstash` as entrypoint
* Safely handle `Ruby` runtime (which sadly is still a singleton, moving away from that will require a few iterations on top of this)
* Adjusted `bat` and `sh` entry point wrappers
* Verified manually that performance is unchanged (i.e. all Java opts are still loaded properly)
* Flattened `.jar` path to make it a little less bothersome to build the `-cp` string
* Retained ability to load jars from Ruby via the global `$LS_JARS_LOADED` variable hack, to keep plugin specs that load LS as a `.gem` functional (like e.g. the ITs in LS itself)
* No need for the gem jars magic anymore, the downloading and moving into place of jars is now all handled by Gradle

Fixes #8161
2018-01-03 19:13:28 +00:00
Armin
f002542a22 MINOR: Prevent enviornment GEM_DIR and GEM_PATH from leaking into build
Fixes #8896
2017-12-29 18:21:38 +00:00
Colin Surprenant
6dabcb4ad9 refactor linux launch scripts per #8261
remove dripmain.rb
2017-09-19 16:45:41 -04:00
Wainer dos Santos Moschetta
d4e7705b14 Quote shell script variables handling paths.
Some variables that hold path in Linux shell scripts
are not properly quoted, thus undefined behaviors may
show up (e.g. a link path with space).

This change enclose variables with quotes in bin/logstash,
bin/logstash.lib.sh and, bin/system-install scripts.

Fixes #6596

Fixes #6877
2017-04-04 15:00:48 -04:00
Josh Soref
a06dc211ff Spelling fixes (#6806)
Spelling fixes across the board.
2017-03-29 19:33:26 -07:00
Aaron Mildenstein
60b6a9f72e Have logstash.lib.sh find jvm.options
This will work if `--path.settings` is properly used on the command line

Remove unused count variable

It isn't strictly necessary

Add double-quotes around $@

This should protect against strange spaces

fixes #6379

Fixes #6382
2016-12-09 12:19:22 -05:00
Bartłomiej Palmowski
d5d556989d Don't pass jruby opts explicitly
they are already passed via JAVA_OPTS environment variable.

Fixes #5830
2016-12-06 13:24:59 -05:00
汪啸
1f1bbb1f89 fix wrong logic when no JAVA_OPTS set
Fixes #6034
2016-10-14 17:55:43 -04:00
Aaron Mildenstein
4baf29dc5f Add SINCEDB_DIR env var to logstash.lib.sh
This is to fix the file input

Fixes #5385
2016-05-27 22:08:58 +00:00
Aaron Mildenstein
e40ecbb6b8 Update existing packaging to use /usr/lib/logstash
Update after-install scripts

Use /usr/lib/logstash instead of /opt/logstash

More revisions.

Using `/usr/share/logstash` instead of `/usr/lib/logstash`

Track these new files

More work in progress.

Proper arg parsing obtained

WIP: Use jvm.options like ES

Remove facter dependency

Due to an update from @jordansissel on pleaserun, facter is no longer necessary

Revert Gemfile to proper version

Latest work in progress.

I added a simple test to logstash.lib.sh to catch empty strings in java options.

Added `startup.options`, and touched up `jvm.options`

Edited the package scripts to make things play nice with new paths.

Tweaked `system-install` to allow users to provide their own `startup.options` file.

Patch up merge failure

Changes to work with the new logstash.yml

Add fpm dependency (build fails without it)

Fix missing gems in Gemfile

Don't know how these got lost

Use /etc/logstash for configuration in packages

Update the instructions/documentation for...

...startup.options

Merging proposed changes

Remove mustache and clamp

They're covered by pleaserun anyway

Prune config dir from /usr/share/logstash...

...but only when using RPM/DEB packages

Fixes #5341
2016-05-25 17:25:26 +00:00
Siddhartha Sahu
02f5b54c46 Add ability to configure custom gc log file
This adds ability to add gc log file to the custom $LS_LOG_DIR path

Fixes #4718
2016-02-26 19:30:11 +00:00
Aaron Mildenstein
68bdde886a Allow Logstash to be launched from a symlink
These changes allow Logstash to be launched from a symlink _only_ when the `readlink`, or suitable version of `stat` binaries are present. An error message will result if Logstash is launched from a symlink and the path cannot be discovered by one of these two methods.  The message will advise the user to use the full path instead of a symlink to launch Logstash.

fixes #4291
2015-12-04 15:25:54 -08:00
Joao Duarte
44edf5bd00 allow logstash to run in a dir with spaces
because HeapDumpPath is set, if it contains spaces the jruby_opts
function will create two strings appended by -J
even if the list of jruby opts is constructed using replacement
shell script will not escape the path correct to the heap dump path
option needs to be added as late as possible

this is a *nix fix only, windows is not affected

Fixes #4303
2015-12-04 21:27:57 +00:00
Suyog Rao
f1775d0b7e Update comment about location
Fixes #4205
2015-11-17 15:53:41 +00:00
Suyog Rao
d69cae23e6 Add option for JVM to dump its heap on OutOfMemory
Fixes #4192

Fixes #4205
2015-11-17 15:53:41 +00:00
Pier-Hugues Pellerin
1045ec8602 Change how we set the max heap size
This PR change the name of the variable to set to change the maximum
heap size on window to `LS_HEAP_SIZE` this replace the LS_MIN_MEM and
LS_MAX_MEM option.

Also We change the default size of the heap from 500m to 1g

Fixes #4085
2015-10-30 19:50:51 +00:00
Colin Surprenant
48bcc5b74d cleanup dependencies and rely on logstash-core gem
remove unused load path

use either local core lib dir or logstash-core gem

include spec in logstash-core

do not include lib, spec and locales dirs in package root

environment bootstrapping in lib/bootstrap.rb

added comments

move pluginmanager out of logstash-core

kafla installation solved by pluginmanager refactor

refactor bootstrap code

refactor shell scripts to simplify and DRY, indirect rspec command to our ruby launcher

add bin/rspec

cut by half the bin/plugin and bin/rspec startup time

fix drip context

use printf instead of tr

updated Windows bin scripts

missing gemspec in gemspec

use gem instead of gemspec so our plugin manager can correctly install logstash-core

generate packages including locally built logstash-core gem

move jenkins developmnent dependencies into Gemfile, they do not belong in logstash-core

path leftover

clean help agent help usage message and remore rspec in available command

comments cosmetics

update Bundler dependency, all recent testing have been made with 1.9.4

updated .lock file with regard to updated Gemfile

cleanup gemspec, Gemfile and regen .lock file

added progress output

avoid dual require on version

closes #3096
2015-05-04 18:17:22 -04:00
Pere Urbon-Bayes
193765ced2 adapt the lib.sh file to the use case proposed LS_JAVA_OPTS add to defualts while JAVA_OPTS override
updated the packaged to enable JAVA_OPTS to override the defaults

ammended JAVA_OPTS being exported

ammend warning wording

ammend the last commit

fix wording again

Fixes #2942
2015-04-17 13:49:19 +00:00
Pere Urbon-Bayes
cbaadb16f2 make the LS_JAVA_OPTS environment variable available to the client side, so the user is not force to use the default ones and they can rewrite it
remove stale code

cleanup the bash conditions for when a variable is defined or not

make sure LS_JAVA_OPTS override JAVA_OPTS when need

ammend unused var

rollback to previous use case of appending vars to JAVA_OPTS in the case of having LS_JAVA_OPTS

Fixes #2942
2015-04-17 13:49:19 +00:00
Pere Urbon-Bayes
05a097f853 Make sure the vendorized jruby only have 1.9 libs, so no extra space is used. Also make sure the jruby is run in 1.9 mode.
Fixes #2285
2015-01-05 15:50:15 +00:00
Jordan Sissel
920a9e9362 Refactor build tooling to use Ruby
The new way to build is using 'rake' and not 'make'.

For some some attempt at compatibility, any 'make' invocation will
invoke rake. `make foo` will invoke `rake foo`.

This is for #1640

Major changes:
* rake is used instead of make. This allows us to more easily improve,
  debug, and test the build tooling.
* The build process can be bootstrapped from any Ruby that has
  Rake available. Bootstrapping rake is automatic will download JRuby and
  re-execute itself under JRuby.
* rpm and deb packages no longer list any java as dependencies. This
  is the result of user discomfort and discussions which concluded
  that there is no good way to "depend" on Java via rpm or deb.

Internal changes:
* Vendoring paths are project-based. Each vendor directory is now
  specific as vendor/<project>/... For example, JRuby is dumped into
  vendor/jruby/...

Other changes:

* We don't use jruby-complete anymore. We now use bin/jruby as the entry point.
  This improves the startup time of Logstash on my laptop by approximately
  1.5 seconds (~25% speedup of startup time). Testing `bin/logstash
  version` speed; 5 runs each:

  * Before patch: 7.27 7.55 7.26 7.42 7.47
  * After patch: 5.54 5.97 5.67 5.58 5.54

Smaller changes:
- removed 'env' subcommand
- Remove 'deps' subcommand. Replaced by 'rake vendor:all'
- Add gem_home method for this platform's GEM_HOME
- .travis.yml removed because we haven't used Travis in a long time.
- bin/logstash exits nonzero with a useful message if JRuby can't be found.

Notes:

Invoke bundler via system() call instead of Bundler::CLI.
After some discussion in #bundler on freenode (very helpful!) on how
to do what we need to do with bundler, I was encouraged to not use the
bundler api from ruby, so system() it is.

Patch slightly modified to fit the new model of packaging (ruby instead
of shell invocation), but original author is obazoud in #1008.

Richard Pijnenburg helped with some of the code and was lovely in
helping test it.

Fixes #1640
2014-10-22 17:04:55 +00:00
Colin Surprenant
0d3df38cb9 logstash boot sequence cleanup & drip support fix
deprecated USE_JRUBY environment var
added USE_RUBY and USE_DRIP environment vars
centralized gem path handling in LogStash::Environment

use consistent discovery strategy
2014-07-13 20:02:42 -04:00
Colin Surprenant
a2373502c3 removed dependency on bash 2014-05-02 21:46:50 -04:00
Colin Surprenant
fff8fae97b refactor commands and arguments processing to allow spaces in paths 2014-04-29 17:48:46 -04:00
Jordan Sissel
c4712971ca Permit relative path or $PATH searches for the java command.
This fixes the following broken in #1189:

    % bin/logstash
    => Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME.
2014-04-16 12:35:35 -07:00
Marc Chadwick
843797405c Handle $JAVACMD properly while still respecting $JAVA_HOME 2014-04-02 07:45:49 -04:00
Marc Chadwick
5187857b4f Fix setup_java so $JAVA_HOME can be used. 2014-03-20 12:37:00 -04:00
Jordan Sissel
83ab14983a Actually clear GEM_PATH
This helps ensure we provide only our own gem directory for rubygems.

Helps improve LOGSTASH-1918. Before this patch, my local ruby install
was accidentally providing 'rack' so I never noticed this problem.
2014-03-12 14:00:23 -07:00
Matthew Buckett
55f8488695 Correctly set RUBYLIB for JRuby. 2014-01-15 14:28:53 +00:00
Jordan Sissel
b15852521b Add support for drip.
Drip allows for faster startup time by pre-starting JVMs and keeping a
pool of them around for use on-demand. More details here:
https://github.com/flatland/drip

Tested with this config:

* 'input { generator { count => 1 } } output { null { } }'

Without drip:

    % time bin/logstash -e '...'
    13.19s user 2.94s system 170% cpu 9.458 total

With drip (install drip, then set 'JAVACMD=drip' in env):

    % time JAVACMD=drip bin/logstash -e '...'
    0.03s user 0.46s system 6% cpu 7.008 total

2.5 seconds faster (27% faster)
2014-01-03 21:30:49 -08:00
Jordan Sissel
45c7c5ca48 - remove stacksize setting; use the default 2013-12-10 17:32:44 -08:00
Jordan Sissel
b43600f823 Factor out functions into bin/logstash.lib.sh to keep bin/logstash a bit
cleaner.
2013-12-07 17:18:03 -08:00