Commit graph

3725 commits

Author SHA1 Message Date
Jordan Sissel
4c86e3342f config grammar: Allow spaces in [field references][like this]
Tests included.

Note about grammar compilation:
  I had to use treetop 1.4.15 (1.5.3 generates ruby code that fails to
  parse, haven't spent any time debugging)

Fixes #1513
2014-08-01 17:35:48 +00:00
wiibaa
0f790649bc use non-capturing group in float regex
Closes #1253
2014-07-30 10:55:46 -07:00
wiibaa
2e21df8e9e stronger regex and code-style change 2014-07-30 10:55:04 -07:00
Wiibaa
95abfdebe8 reject invalid UNIX timestamp 2014-07-30 10:55:04 -07:00
Suyog Rao
925ce925ec Removed version info in LS command
Closes #1362
2014-07-25 17:38:13 -07:00
Alejandro E Brito Monedero
e1485422e3 Update metrics.rb example
Update the example because output doesn't have the message parameter, instead it should use a a codec.
Also update the way how logstash is executed.
2014-07-25 17:36:01 -07:00
Gaurav Arora
18c4ec2b35 correct documentation of some config elements
Closes #1380
2014-07-25 17:29:21 -07:00
Jordan Sissel
5daa328987 Update ES version docs
ES version 1.0 or greater is now supported by all protocols.

The Elasticsearch eng team has tests to ensure client nodes from all ES versions >=1.0 work with all ES clusters version >=1.0
2014-07-24 07:35:31 -07:00
Jordan Sissel
a466581dc6 Merge pull request #1532 from colinsurprenant/fix/gem_path
use bundler compatible ruby version string
2014-07-23 14:40:43 -07:00
Suyog Rao
0d987cac9e Add tests for issue to reproduce behavior
Fix break_on_match issue while evaluating multiple patterns

Closes #1547

Make better exception message

Add more specs for inputs with arrays

Closes #1547
2014-07-23 13:38:06 -07:00
Suyog Rao
6e3d377bba Evaluate dynamic fields in event when used with gsub
Closes #1529
2014-07-22 16:43:50 -07:00
Ehtesh Choudhury
4fa053dff8 Use the newer { "fieldname" => "pattern" } for grok filters.
Replace the older grok style:

    { match => [ "fieldname", "pattern" ] }

With the newer one:

    { match => { "fieldname" => "pattern" } }

Also see https://github.com/elasticsearch/logstash/issues/1499
for more examples.

Closes #1516
2014-07-18 16:50:12 -07:00
bonky42
973f899bdc Fix a missing coma in doc for remove_field
Closes #1490
2014-07-18 10:01:43 -07:00
Colin Surprenant
cc2155919f use bundler compatible ruby version string 2014-07-14 17:41:06 -04:00
Jordan Sissel
ea350aaf23 inputs/base: Copy the codec's charset when switching.
Includes tests
2014-07-14 14:32:15 +00:00
Colin Surprenant
0d3df38cb9 logstash boot sequence cleanup & drip support fix
deprecated USE_JRUBY environment var
added USE_RUBY and USE_DRIP environment vars
centralized gem path handling in LogStash::Environment

use consistent discovery strategy
2014-07-13 20:02:42 -04:00
Colin Surprenant
7b6ab95124 fix connection threads tracking leak
better mutex name, loop with while true

event per connection & thread cleanups specs

remove leftover

closes #1509
2014-07-11 16:02:23 -04:00
Suyog Rao
393a961805 Fix for #1451
Line codec was using an undefined variable which casued it to throw and exception.

Fixed and added tests for line codes

Closes #1451
2014-06-30 11:18:02 -07:00
Jordan Sissel
65af91d015 Fix typo 2014-06-24 13:08:18 +00:00
Jordan Sissel
3dfe8e9d2c Fix missing require 2014-06-24 13:08:13 +00:00
Colin Surprenant
dfd26239f2 fix external command invocation 2014-06-20 16:24:07 -04:00
Colin Surprenant
e03b67dc7d replace json parsers with JrJackson and Oj
refactored timestamps with new Timestamp class
closes #1434
2014-06-12 17:53:30 -04:00
Philippe Weber
ccb453d557 initial spec for pipe input
removed logging

fixed encoding header

closes 1385
2014-05-22 15:55:24 -04:00
Colin Surprenant
ee3e684feb fix stdin message handling & specs
closes #1386
2014-05-22 14:23:04 -04:00
Jordan Sissel
4d64691d0a Automatically choose most correct codec for bytestream inputs.
Now, if you try to use `plain` or `json` codecs with either `stdin` or
`tcp` inputs, it will automatically select `line` or `json_lines` codec
respectively.

Right now, many users try to use json codec with tcp input. This
*should* work, but due to technical limitations[1], it does not.
To compensate, we have always provided a json_lines codec to help
parse "streams of line-delimited json" which is what most folks
seem to be handling these days.

[1] I haven't found a JSON library capable of processing a byte stream.
Jackson has one, but it requires an input stream model which logstash
does not currently expose. Future work will make this possible.

Related: #1391
2014-05-20 04:52:58 +00:00
Aaron Mildenstein
ab9985c26a Fixing relative path issues left over from flatjar era
Use plugin_path and pattern_path

closes 1377
2014-05-16 08:53:29 -05:00
Aaron Mildenstein
ed2cdbdf22 Add NaN support back in to collectd codec
* Add spec tests for NaN support
* Catch warning on NaN with nan_handling set to warn
* Make lambdas ivars and initialize so they can reference other ivars
* Fixed reference to point to ivar
* Streamline ivar declarations in one method
* Improve tests to cover more cases
* Make lambdas not be ivars
* Change idiomatic calling of init_lambdas to reflect lack of return value

closes 1363
2014-05-14 14:32:21 -05:00
Colin Surprenant
b710536ba8 missing require
cosmetic

closes #1345
2014-05-08 17:54:20 -04:00
Colin Surprenant
c97845a91c bumped version to 1.4.1 2014-05-06 15:13:27 -04:00
Richard Pijnenburg
e8d297136a Merge pull request #1332 from coolacid/LOGSTASH-2166
Fix Date#Timestamp link
2014-05-06 16:14:37 +01:00
Aaron Mildenstein
1a011db2fc Merge pull request #1101 from untergeek/collectd-untergeek
New collectd codec…
Better performance and raft of other fixes along the way!
2014-05-02 14:21:48 -05:00
Colin Surprenant
af3a952dd7 Merge pull request #1319 from coolacid/LOGSTASH-2165
[DOCS] Update irc input docs for password channels
2014-05-02 15:02:12 -04:00
Aaron Mildenstein
5b793bb4f6 Fix the same array path bug in the collectd input plugin (soon to be deprecated). 2014-05-02 13:46:49 -05:00
Aaron Mildenstein
f1397c37f8 Merge branch 'master' into collectd-untergeek
* master:
  Fix typo I somehow forgot to commit in the last PR
  Refacter known file lookups to a single method in LogStash::Environment
  Purge jar-specific handling that is not needed anymore.
2014-05-02 13:42:38 -05:00
Aaron Mildenstein
296851674c Refactored to use new vendor_path method. Fix array paths issue with streamlined types.db discovery 2014-05-02 13:40:10 -05:00
Jordan Sissel
ed0faf7c2e Fix typo I somehow forgot to commit in the last PR 2014-05-02 11:24:05 -07:00
Jordan Sissel
0d2f0b9a11 Refacter known file lookups to a single method in LogStash::Environment 2014-05-02 11:09:22 -07:00
Jordan Sissel
98f28ad376 Purge jar-specific handling that is not needed anymore.
This removes all monkeypatches for JRUBY-6970, woohoo!
I also tried to find any references to jar, jar:, or file: and
remove the special handling there.

`make test` says it's cool, so I'm happy enough.

    Finished in 19.7 seconds
    502 examples, 0 failures
2014-05-02 10:23:27 -07:00
Aaron Mildenstein
686fe37b23 Fixed to use path relative to collectd.rb codec. 2014-05-02 11:54:01 -05:00
CoolAcid
38b275b4dc Fix Date#Timestamp link 2014-04-30 12:10:35 -04:00
CoolAcid
50c61bf1f4 Update docs for password channels 2014-04-30 11:26:19 -04:00
Aaron Mildenstein
613b5711ea Need to rescue on HeaderError too 2014-04-25 11:09:34 -05:00
Aaron Mildenstein
9fccbd7280 Patches some suggestions from @pieterlexis
- Pruned unnecessary variable `remove_header` #L196
- Fixed indent #L358
- Removed break, replaced with raise exception #363-365
2014-04-25 09:38:30 -05:00
Aaron Mildenstein
25033496cd Latest info:
* Hash + Closures. Now replacing case/when statements with closures and hashes to improve performance
* Raise exceptions now instead of using `break`
* Zipped lines 163-164 per @colinsurprenant
* In line 280, this change is a regression.  The authfile is tiny, so this should still be okay, and it's only read once.  Tests failed without this change.
* Use `has_key?` instead of `include?()` in multiple places.
* In the spec/test, I fixed a typo that was bugging me.

With this commit, all tests pass!

```
$ bin/logstash rspec -fd spec/codecs/collectd.rb
Using Accessor#strict_set for specs
Run options: exclude {:redis=>true, :socket=>true, :performance=>true, :elasticsearch=>true, :broken=>true}

LogStash::Codecs::Collectd
  None
    should parse a normal packet
  Sign
    should parse a correctly signed packet
    should not parse an incorrectly signed packet
  Encrypt
    should parse an encrypted packet
    should not parse unencrypted packets when encrypt is configured

Finished in 2.1 seconds
5 examples, 0 failures
```
2014-04-24 17:39:31 -05:00
Aaron Mildenstein
6bf38771c1 It turns out that it would fail every time if…
Consistent failure if I didn't put `next if typename.nil?` before the rest of the statement.  This compromise puts 2 `next if` statements, then an implied `else` since those were the two conditions I needed to check 😄
2014-04-23 16:23:30 -05:00
Aaron Mildenstein
56b7f1b274 Patches per @colinsurprenant 2014-04-23 15:46:17 -05:00
Aaron Mildenstein
f16582b1ba Prune commented-out old one-liner 2014-04-22 14:42:08 -05:00
Aaron Mildenstein
a7f3003124 Forgot to prune out the old jar code. 2014-04-22 14:38:26 -05:00
Aaron Mildenstein
c782d83074 Patched the recommendations per @jordansissel 2014-04-22 14:34:52 -05:00
Aaron Mildenstein
1a02dbfdd8 Merge branch 'master' into collectd-untergeek
* master: (208 commits)
  Fix for LOGSTASH-2093
  Fix for LOGSTASH-2129
  Bump Elasticsearch to 1.1.1 and Kibana to 3.0.1
  Permit relative path or $PATH searches for the java command.
  Add LS_LOG_DIR option
  Improve package acceptance tests
  Fix minor issues with acceptance test
  Fix minor bug in init script found after acceptance test
  Add gemfile
  Initial commit of package acceptance testing
  Typo
  Copy-paste typo fix
  Make embedded => true respect `bind_host` setting.
  Start ES embedded server before creating a client.
  fix and regression tests for github issue #1258
  fix for undefined method + for nil:NilClass for invalid paramters
  reenabled multiline filter specs, refactored concurent streams generator, cleaups
  flush filters at end of test input
  flush support in multiline filter
  Adding new sysv init style scripts based on pleaserun
  ...
2014-04-22 13:42:11 -05:00