Commit graph

57 commits

Author SHA1 Message Date
Joao Duarte
cd85dd081d test output plugin vs worker params
Fixes #2876
2015-03-24 21:06:12 +00:00
Pere Urbon-Bayes
e08cad94c0 enabled coverage report again throw installing missing dependencies when the user run rake test:prep, user should use COVERAGE=true as before
Fixes #2412
2015-01-27 12:15:32 +00:00
wiibaa
232cf92178 harmonized event filtering by type/tags in filter and output + correct logging
Fixes #2339
2015-01-12 18:17:45 +00:00
Richard Pijnenburg
fec0be423a Cleanup plugins to move to plugin system
Plugins have moved to https://github.com/logstash-plugins/... into
individual repositories. One plugin per repo.

* Add plugin install task and add required plugins for testing
* Adding a plugin task to install plugins
* Added required plugins to a prep task for the test part.

Fixes #1984

Conflicts:
	lib/logstash/filters/multiline.rb
	lib/logstash/outputs/hipchat.rb
	logstash.gemspec
2014-11-04 23:30:56 +00:00
Jordan Sissel
d4ce452a7e Refactor build tooling to use Ruby
The new way to build is using 'rake' and not 'make'.

For some some attempt at compatibility, any 'make' invocation will
invoke rake. `make foo` will invoke `rake foo`.

This is for #1640

Major changes:
* rake is used instead of make. This allows us to more easily improve,
  debug, and test the build tooling.
* The build process can be bootstrapped from any Ruby that has
  Rake available. Bootstrapping rake is automatic will download JRuby and
  re-execute itself under JRuby.
* rpm and deb packages no longer list any java as dependencies. This
  is the result of user discomfort and discussions which concluded
  that there is no good way to "depend" on Java via rpm or deb.

Internal changes:
* Vendoring paths are project-based. Each vendor directory is now
  specific as vendor/<project>/... For example, JRuby is dumped into
  vendor/jruby/...

Other changes:

* We don't use jruby-complete anymore. We now use bin/jruby as the entry point.
  This improves the startup time of Logstash on my laptop by approximately
  1.5 seconds (~25% speedup of startup time). Testing `bin/logstash
  version` speed; 5 runs each:

  * Before patch: 7.27 7.55 7.26 7.42 7.47
  * After patch: 5.54 5.97 5.67 5.58 5.54

Smaller changes:
- removed 'env' subcommand
- Remove 'deps' subcommand. Replaced by 'rake vendor:all'
- Add gem_home method for this platform's GEM_HOME
- .travis.yml removed because we haven't used Travis in a long time.
- bin/logstash exits nonzero with a useful message if JRuby can't be found.

Notes:

Invoke bundler via system() call instead of Bundler::CLI.
After some discussion in #bundler on freenode (very helpful!) on how
to do what we need to do with bundler, I was encouraged to not use the
bundler api from ruby, so system() it is.

Patch slightly modified to fit the new model of packaging (ruby instead
of shell invocation), but original author is obazoud in #1008.

Richard Pijnenburg helped with some of the code and was lovely in
helping test it.

Fixes #1640
2014-10-22 17:04:58 +00:00
Suyog Rao
f523f0a063 Remove the rescue so we can see what errors we get
Closes #1931
2014-10-21 18:24:55 -07:00
Tal Levy
475afda874 Include stud/try module to elasticsearch_spec test
Previously, a few elasticsearch integration tests relied on this module,
but the module could not be found.

Fixes #1929
2014-10-21 20:20:00 +00:00
Colin Surprenant
b7f33f6f70 add elasticsearch spec tag to exclude on external ES dependency
Fixes #1920
2014-10-20 18:01:14 +00:00
Chen,Hao
462216d1c9 ElasticSearch output plugin to support multiple hosts and enhance stability. (by Hao Chen)
Closes #1791
2014-10-09 17:26:24 -07:00
Suyog Rao
8b6c8e035c remove kafka source since they moved to separate plugin
Closes #1801
2014-09-30 14:42:00 -07:00
Richard Pijnenburg
e7d41e3c80 [TESTING] Fix Kafka output test
When doing randomized testing, one of the tests failed because kafka was not initialized yet

Fixes #1758
2014-09-30 16:06:51 +00:00
Richard Pijnenburg
88a7ae3283 [RSPEC] Refactoring step 1
- Move helper functions in own modules and extend Rspec
- Refactor files into correct naming and paths
- Modify files to use new spec_helper and helpers
- Pin rspec to 2.14.x

Fixes #1758
2014-09-30 16:06:50 +00:00
Joao Duarte
277a890451 Prevents event.sprintf from converting floats to sci notation
This allows sprintf to correctly print floats up to 15 decimal
places, which should be enough for most. Without this, very big
or very small floats will be converted to scientific notation.

closes #1670
2014-09-08 10:57:16 +01:00
Suyog Rao
cdf6c85588 Merge @joekiller's Kafka plugin to logstash-core
Add documentation and tests
Use scala version 2.9.2 and Kafka 0.8.1.1
Closes #1472
2014-09-03 15:07:34 -07:00
Colin Surprenant
84d938ebdc replace json parsers with JrJackson and Oj
refactored timestamps with new Timestamp class
closes #1434
2014-06-12 18:09:22 -04:00
Colin Surprenant
6271cbf8fb refactor jar loading 2014-04-08 23:32:45 -04:00
Colin Surprenant
efbf2f5606 tag :socket for exclusion 2014-03-14 11:12:23 -04:00
Colin Surprenant
1e48802b8c tag :broken for exclusion 2014-03-14 11:12:23 -04:00
Colin Surprenant
74a214e39c tag :elasticsearch for exclusion 2014-03-14 11:12:23 -04:00
Colin Surprenant
a633afb67c tag :redis for exclusion 2014-03-14 11:12:23 -04:00
Jordan Sissel
2e7198b8f0 Fix typos and test failures 2014-02-21 12:06:40 -08:00
Jordan Sissel
13b11c8ecd Add tests for templates and all 3 protocol settings 2014-02-21 11:48:05 -08:00
Jordan Sissel
06899c3940 Fix 'cleanup' step to properly delete all data.
In elasticsearch 1.0, the "DELETE /" request is now rejected by default.
So instead, we will "DELETE /*"
2014-01-21 11:59:38 -08:00
Jordan Sissel
830bc8fc33 Use refresh, not flush, silly me. 2013-12-23 15:49:57 -08:00
Jordan Sissel
ffbe9a752b In which I join a long list of folks who have typoed this: it's "lon"
not "long"
2013-12-16 15:47:13 -08:00
Jordan Sissel
c5729ecbd5 - Blow away templates first 2013-12-16 15:47:04 -08:00
Jordan Sissel
beb8512f46 Add stopwords test to make sure stopwords (like 'at') are *not* removed
...from the terms list.

Also, minor fix, call _refresh, not _flush since _refresh is what we
meant.
2013-12-16 15:22:28 -08:00
Jordan Sissel
9a1b17efcd Add tests to ensure correct behavior of elasticsearch with our new
template settings.

Some of these tests fail at this time because our template is incorrect.
The template will be fixed shortly.
2013-12-16 14:54:05 -08:00
Matt Gray
d65cb0b889 output : csv - Create a CSV file output
Allows writing of events to CSV or other delimited files

Based on existing file output, using ruby CSV lib
2013-12-06 15:46:07 +00:00
Matt Gray
870e4def19 Fix file output tests (broken by removal of Event::from_json) 2013-11-13 11:04:51 +00:00
Jordan Sissel
faf6023dea - copy tests from elasticsearch_http to test type settings 2013-10-03 11:57:05 -07:00
Jordan Sissel
1729db4f89 - Add more tests to verify document index and index type settings 2013-10-03 11:27:54 -07:00
Wiibaa
edded01ab8 Repair outputs/file test + add use of zcat command to cover LOGSTASH-997 2013-09-11 14:39:14 +02:00
Jordan Sissel
0ec869bca0 - fix redis tests 2013-08-20 22:59:04 -07:00
Jordan Sissel
85be32b5ca - use real tcp sockets 2013-07-03 22:48:25 -07:00
Jordan Sissel
e48f05e607 Remove '@' from '@message' 2013-06-19 10:03:38 -07:00
Jordan Sissel
86dc8ba4cc Merge pull request #480 from wiibaa/output-mails
Allow @message, @source etc in match for email output
2013-05-28 23:59:03 -07:00
Wiibaa
8d644ba281 Rebase of previous pull to execute spec tests and integrate ibrahima comment:
* Allow @message @source in match for email output
* Restore lost plugin documentation from some time ago
  6c9b36b227
* Integrate ibrahima comments: use respond_to? :each
2013-05-14 08:20:16 +02:00
Wiibaa
dc4aff493d initial spec test for output/file 2013-05-14 07:20:49 +02:00
Jordan Sissel
bc6965d521 - Add coverage for LOGSTASH-968 2013-05-01 13:02:09 -07:00
Jordan Sissel
aef2d5cff0 - make these proper tests that run in a way that rspec understands. 2013-04-28 13:27:17 -07:00
Rene Lengwinat
4888c4ec07 Replaced placeholder %{metric} with '*' as aligned with jordansissel 2013-03-24 14:58:03 +01:00
Rene Lengwinat
24eb0b2223 Minor changes on graphite plugin.
- Added metrics_format option to define the resulting format of the
  metric string that will be sent to graphite.

  example:

    graphite {
      host => "graphite.local"
      port => 2003
      fields_are_metrics => true
      include_metrics => ["foo"]
      metrics_format => "my.system.%{metric}"
    }

- added test suite for graphite plugin
2013-03-09 11:44:24 +01:00
Wiibaa
c84196ba96 Fix and test for LOGSTASH-559 authenticationType=>nil 2013-02-28 07:20:32 +01:00
Wiibaa
f2feaa3c31 Fixes for LOGSTASH-827 and LOGSTASH-841
Adding a solution for testing output/email using rumbster (https://github.com/aesterline/rumbster)
2013-02-28 07:20:32 +01:00
Alex Dean
ab7f03cded fix race condition which causes redis output to lose some messages in batch mode
Scenario:
  - @flush_thread starts writing to redis (@redis.rpush)
  - main thread adds a new message to @pending
  - @flush_thread deletes this message (@pending.delete)
2013-02-04 08:36:06 -06:00
Jordan Sissel
a5ece58282 - add test converting US-ASCII to UTF-8 from generator input to redis
output
2012-11-22 09:05:00 +00:00
Laust Rud Jacobsen
60fc1693d4 Logstash::Outputs::Redis: JSON-encode message outside of Redis reconnect
Fixes LOGSTASH-712
2012-11-19 11:17:57 +01:00
Jordan Sissel
83218266a4 Give up for now; will probably do vagrant in the future. For now just requiring a live redis server externally for redis tests 2012-10-21 00:17:04 -07:00
Jordan Sissel
8e17cc292b - Add 'redis' test service (download/build/run redis)
- Make redis input/output specs use a real redis server
- Fix test_utils bugs to make around(:each) work.
2012-10-20 23:28:10 -07:00