Commit graph

93 commits

Author SHA1 Message Date
Suyog Rao
32e4958832 Release 2.4.1 (#6207)
* Release 2.4.1

* Add release notes. Explicitly constraint docker and ansicolor
2016-11-07 12:57:03 -08:00
Joao Duarte
b564382017 avoid logging exception objects in agent.rb
Fixes #6011
2016-10-12 04:53:31 -04:00
Joao Duarte
2aa445a690 2.4.0 release
Fixes #5842
2016-08-30 09:38:06 -04:00
Suyog Rao
36f3d4162d Release snapshot4 (#5839) 2016-08-26 20:48:01 +00:00
Joao Duarte
9a6d76b22c prevent exception if logstash is interrupted during startup
during startup signal trapping is set up to gracefully shutdown logstash
this signal handling relies on Stud.stop? which depends on @thread being set

Currently we set that variable after setting up the trapping, which
creates a window where an interruption will trigger Stud.stop and find
@thread to be nil.

This patch moves the setting of @thread to be done before trapping, this
fixing the issue.

Fixes #5829
2016-08-25 06:59:24 -04:00
Suyog Rao
f920c4f30d Downgrade FPM and clamp versions to 2.3 levels (#5809) 2016-08-22 22:16:57 +00:00
Suyog Rao
fc4c37f954 release 2.4.0.snapshot2 (#5807) 2016-08-22 19:40:03 +00:00
Andrew Cholakian
a69d2f5632 Add new Codec#encode_sync, Codec#multi_encode and Output#multi_receive_encoded methods. (#5805)
[2.4] Add new `Codec#encode_sync`, `Codec#multi_encode` and `Output#multi_r…

Fails due to stale gemfile
2016-08-22 13:30:58 -05:00
Andrew Cholakian
48e421e7e6 Backport of Outputs#base.concurrency to 2.x branch
This is a backport of https://github.com/elastic/logstash/pull/5752 to
the 2.x branch targetting the 2.4 release.

The changes here are more minimal since the pipeline code is simpler and we need to support
some other legacy constructs such as the instance level version of Outputs::Base#workers_not_supported.
2016-08-22 13:05:58 -05:00
Suyog Rao
f569d707a5 Release 2.4.0.snapshot1
This PR moves from 2.4.0.dev1 to snapshot1. I also published
required gems to RubyGems

Fixes #5782
2016-08-19 13:52:26 -04:00
Joao Duarte
cc309cf893 remove duplcate config_loader recreation when -t is enabled
Fixes #5765
2016-08-18 12:41:05 -04:00
Joao Duarte
5ef5b983af update fpm and clamp to remove .lock conflits
Fixes #5669
2016-07-21 03:49:20 -04:00
Andrew Cholakian
9f8169fe57 Add support for hostname:port syntax to SafeURI class
This is a deviation from what the ruby URI class normally allows.
With this patch users can use "myhost:123" as an option and have it do the
right thing, as opposed to before where they'd get an error and have to
use "//myhost:123".

Fixes #5618
2016-07-12 12:29:32 -04:00
Joao Duarte
a8e7e136a2 bump core to 2.4.0.dev and make plugin_api to 1.21
* bump logstash-core/logstash-core-event/logstash-core-event-java to 2.4.0.dev
* bump logstash-core-plugin-api to 1.21.0
  * provides both get/set and []/[]= events apis by allowing to be installed in logstash >= 2.4 and < 3
* minor bumps on gem dependencies

Fixes #5601
2016-07-11 06:49:41 -04:00
Pere Urbon-Bayes
92fc45d7be ammend specs
Fixes #5587
2016-07-07 14:47:14 -04:00
Pere Urbon-Bayes
9fcae840cc ammend debug code
Fixes #5587
2016-07-07 14:47:14 -04:00
Pere Urbon-Bayes
df2fa74df6 make sure errors are reported also to terminal if show_startup_errors is true
Fixes #5587
2016-07-07 14:47:14 -04:00
Pier-Hugues Pellerin
d717c718cf added a test to make sure NullMetric#time return the block result
Fixes #5544
2016-06-23 19:34:35 -04:00
Pier-Hugues Pellerin
3cd5b4b5a6 Implement a NullMetric Object in the 2.X series of Logstash
This PR, backport the `NullMetric` class from the master branch into the
2.X series and allow to write plugin that can be run on both Logstash 5
and Logstash 2.X without changing the code and without having to check
if the metric object is set before capturing a metric.

Fixes: #5539

Fixes #5544
2016-06-23 19:34:34 -04:00
Andrew Cholakian
1c6dff1b68 Fix a regression where untyped fields would return null values
This patch fixes a bug where config fields declared without a type
would just return null.

Fixes #5512
2016-06-16 15:37:23 -04:00
Andrew Cholakian
d77895b7fa Add new :list property to configuration parameters.
If set to try this will allow the user to specify one or more values.
This generally replaces the :array type, which had fewer type checks.

The array type is still needed for lists of complex objects, e.g. hashes.

Fixes #5499
2016-06-15 14:43:42 -04:00
Andrew Cholakian
21af4cbeb9 Add URI config validator/type
Often times plugins (like the Elasticsearch output) can naturally use URIs for their configuration.
Unfortunately using the :string type here means that the password portion of the URI can easily be leaked.

This wraps the URI class in a new LogStash::Util::SafeURI class that proxies all regular URI methods but masks
the password when `#to_s` and `#inspect` are invoked.

Fixes #5470
2016-06-09 09:42:34 -04:00
Joao Duarte
81138c9914 ensure plugins are correctly compiled outside of same plugin section
Fixes #5459
2016-06-07 16:17:05 -04:00
Joao Duarte
3106f3cf91 reduce overhead of pipepile verification/compilation
* avoid class variable during pipeline compilation
* avoid creating pipeline if fetched config is the same as previous
* reduce overhead of create_pipeline if config is known
* use closure table instead of ivars
* avoid dynamic use of singleton methods in pipeline compilation
* adapt config_ast test to support lack of ivars
2016-05-17 08:43:37 +01:00
Joao Duarte
2427768fa2 better error message for reload incompatible pipelines 2016-05-11 08:22:53 +01:00
Jordan Sissel
4e1f8258ae Log a string instead of an exception instance to avoid serialization problems.
Fixes #5277
2016-05-10 19:21:21 +00:00
Jordan Sissel
39e0ef5783 Allow Logstash to write its logs in JSON format
This is made available by a --log-in-json flag. Default is false.
When false, the old behavior [1] is used.

When true, JSON logs are emitted.

[1] The old behavior is realy two things. First, using Object#inspect to
serialize. Second, to color the output if the IO is a tty.

For #1569

This is a manual backport of PR #4820 into the 2.x branch.

Fixes #5277
2016-05-10 19:21:21 +00:00
Pier-Hugues Pellerin
4b32d4d956 Fix missing translation on debug_config and other keys from the agent
Fix the issues found when running the i18n static analysis.

Fix #5216

Fixes #5219
2016-04-28 13:28:03 +00:00
Joao Duarte
b964c01ee1 converted static I18n test to rspec
Fixes #5158
2016-04-27 14:49:42 +00:00
Colin Surprenant
9538458ac8 upgrade to JRuby 1.7.25 and required jruby-openssl 0.9.16 to match JRuby 2016-04-20 14:13:03 -04:00
Pier-Hugues Pellerin
657a27ae33 Missing translation when running Logstash with a wrong path for -f
Fixes #5118

Fixes #5119
2016-04-14 16:01:11 +00:00
Colin Surprenant
cbfc93eb8d revert to logstash-core-event 2016-04-06 14:53:47 -04:00
Andrew Cholakian
08628fa2df Prevent config leak on output delegator close
On shutdown logstash may leak config information for outputs including
plaintext passwords

Fixes #5001
2016-04-05 20:56:26 +00:00
Tal Levy
7d86a6195f add experimental flag for environment variables in config
Fixes #5003
2016-04-04 22:25:27 +00:00
Andrew Cholakian
893ecc2dcc Properly set @debug_config flag in override
Fixes #4987
2016-04-04 18:54:29 +00:00
Joao Duarte
3c254afd22 fix loader tests for debug_config
Fixes #4987
2016-04-04 18:54:29 +00:00
Andrew Cholakian
7748630922 Hide sensitive AST/config data unless --debug-config argument passed
We now hide this because displaying it is dangerous. Generated code
and other AST data may contain plaintext copies of 'password' type fields.
Users can force this to appear with the --debug-config flag.

Fixes https://github.com/elastic/logstash/issues/4964

Fixes #4987
2016-04-04 18:54:29 +00:00
Joao Duarte
4686b7a431 ensure Plugin#original_params hides plaintext passwords
fixes #4952
2016-04-04 17:53:20 +01:00
Joao Duarte
2d24bf772a test identity of generated conditional functions
Fixes #4970
2016-04-01 17:22:02 +00:00
Joao Duarte
b5be117fa1 ensure cond_func methods are singletons
Fixes #4970
2016-04-01 17:22:02 +00:00
Joao Duarte
e11464861b remove Pipeline.validate_config
Fixes #4944
2016-03-31 21:00:39 +00:00
Joao Duarte
e2870cac07 fix --config-test and add tests for it
fixes #4933

Fixes #4944
2016-03-31 21:00:39 +00:00
Joao Duarte
51874b88d1 fix race condition in agent spec
Fixes #4912
2016-03-29 15:52:15 +01:00
Andrew Cholakian
a8d5177f8b Revert to use only a single worker by default on outputs
The move to auto-scale output workers was great in theory, but a lot of outputs
just weren't built to support it well, they often used too many resources or had
logical errors.

Fixes #4905
2016-03-28 18:34:20 +00:00
Joao Duarte
6cf8846686 remove node_name cli argument
this argument is not used and was introduced by mistake in
the backport of the automatic configuration reloading

Fixes #4850
2016-03-24 21:13:29 +00:00
Andrew Cholakian
a088de13a2 Fix threadsafe workers in output delegators to not instantiate extras Cherry pick of db9e67802fb1f6cd097bcac0a7aa59a3a453ceb1 from master Originally from https://github.com/elastic/logstash/pull/4763
Fixes #4894
2016-03-24 21:11:53 +00:00
Joao Duarte
69813c0f29 prevent reloading of configurations containing non reloadable plugins
Fixes #4893
2016-03-24 20:55:52 +00:00
Joao Duarte
62720bcd22 fix reload_state spec to ensure pipeline has started
Fixes #4855
2016-03-21 16:19:03 +00:00
Joao Duarte
0d63d2c47d add cli description of reload_interval
Fixes #4832
2016-03-18 20:53:20 +00:00
fbaligand
e343d86ad0 add complementary tests to 'environment variable evaluation'
Fixes #3944
2016-03-07 11:16:57 -08:00