Commit graph

8588 commits

Author SHA1 Message Date
Pier-Hugues Pellerin
b94b24a7a1 Remove unecessary dependency file-dependencies
To understand why we can now remove this gem we have to go back at the
history of LS 1.5 and the choice we made back them. In the begining
plugins depending on external files like the `user-agent` or the `filter-geoip`
were not bundling theses files in the gem.

The `file-dependencies` was providing a new hook when plugins were
installed to trigger the file download.

In the context of making the plugins work in an offline environment we
need to make sure that every plugins external resources are present in
the gem.

We have currently a few plugins that require external files and they all
bundle it. So I think its safe to remove that feature, also the hook was
not triggered correctly. So it was a NOOP.

Fixes #8298
2017-09-18 18:56:58 +00:00
Pier-Hugues Pellerin
7b09be4ead Bootstrap JRuby using gradle
Instead of using rake to bootstrap we are slowly moving part of our
infra to gradle this is the first step taken, I didn't change the calls,
I just made the `rake bootstrap` call the `gradle bootstrap`

Closes: #8234

Fixes #8238
2017-09-18 14:32:10 +00:00
Pier-Hugues Pellerin
0b0395622c Remove tests dependency on filter multiline
Some of the test were still depending on the multiline filter, this
filter is now shipped anymore with Logstash. I've keep the same logic
but created a simple dummy filter.

Fixes #8284
2017-09-18 13:24:40 +00:00
Armin
bc4cdcb0a4 #8149 Fix Logstash::Agent to not leak its webserver thread
Fixes #8151
2017-09-18 13:09:19 +00:00
Armin
91b5128338 #4640 make Event#clone clone metadata as well
Fixes #8291
2017-09-18 12:54:39 +00:00
Armin
ad57ae8136 MINOR: Remove Docker from Travis
Fixes #8289
2017-09-18 12:45:04 +00:00
Armin
6ea7802d50 PERFORMANCE: Remove needless timestamp reference in Event
Fixes #8294
2017-09-18 11:04:35 +00:00
Armin
912a84c9fe MINOR: Fix dlq reader not closing its watchservice
Fixes #8276
2017-09-18 00:20:14 +00:00
Colin Surprenant
d9ba3463b3 update plugin manager script for setup.bat changes 2017-09-15 18:05:14 -04:00
Armin
1ee3b5c5d4 #8131 Fix BigInteger and BigDecimal serialization by removing BigDecimalBivalue and BigIntegerBivalue
Fixes #8205
2017-09-15 19:03:42 +00:00
Armin
c3d5332d7d MINOR: Faster batch clocks
Fixes #8081
2017-09-15 16:16:01 +00:00
Jake Landis
5d35a8f315 Revert "Add jvm option ExitOnOutOfMemoryError"
This reverts commit a14fbfc3e2.

Fixes #8268
2017-09-15 13:55:02 +00:00
Armin
5fcf028deb #8271 fix one excess '/' added to log config file uri on *nix in runner.rb
Fixes #8272
2017-09-15 13:15:27 +00:00
Armin
b23edffb63 Purging headpages needs to reflect in the currentByteSize of the Queue
Fixes #8186
2017-09-15 06:01:21 +00:00
Rob Bavey
ee1a2825f8 Fix Modules doc incorrectly doc referring to Kibana
var.elasticsearch.ssl.certificate_authority should not refer to Kibana in the
description of the value.

Fixes #8264
2017-09-15 02:02:04 +00:00
Rob Bavey
822e281429 Fix double password wrap
Password settings for Modules supplied via a yml file were being wrapped inside a
LogStash::Utils::Password object twice during startup - once in from_yaml, and once
in validate. Disallow this from happening by skipping the wrap if the password is
already wrapped

This should complete the fix for #8224

Fixes #8258
2017-09-14 22:12:45 +00:00
Colin Surprenant
79f6618647 reword windows launch script
fixes

fix jvm.options parsing

windows compatible URI
2017-09-14 16:25:56 -04:00
Jake Landis
7391be9ea2 Remove CHANGELOG from Dockerfile
Fixes #8262
2017-09-14 20:20:11 +00:00
Colin Surprenant
be20242d6f bubble up java stacktrace from wrapped exception
move exception builder in RubyUtil

cosmetic

cosmetic
2017-09-14 14:59:12 -04:00
Jake Landis
1340f2eee0 Introduce docker for CI builds.
This commit includes:
* A base Dockerfile and script to push to a Docker repo
* A per-build Dockerfile (derived from the base)
* Updates to the test scripts to allow for more parallel builds
* Docker wrappers for the tests scripts
* Update for the integration test readme to manually run the tests
* Clean up the output of the Java tests
* Remove offline tag for tests (no longer needed that we don't use docker dependent services)

This commit does NOT include:
* Changes needed for the CI system to use Docker

Fixes #8223
2017-09-14 18:55:41 +00:00
Rob Bavey
2d6eef7f5e Remove CHANGELOG.md from list of packaged files
CHANGELOG.md has been removed, so it should no longer be packaged

Fixes #8255

Fixes #8256
2017-09-14 17:46:57 +00:00
DeDe Morton
e5b1e0a693 Clarify use of the dead letter queue path
Fixes #8243
2017-09-14 16:34:22 +00:00
liketic
b46f8d207f Remove deprecated CHANGELOGs
Fixes #8231
2017-09-14 16:02:17 +00:00
Colin Surprenant
192e857c81 remove unsued windows code 2017-09-14 10:53:02 -04:00
DeDe Morton
76c7975962 Fix heading level in docker docs
Fixes #8222
2017-09-14 03:36:03 +00:00
Rob Bavey
ef431bd8f7 Fix failing scaffold test
Fix failing scaffold test by wrapping module settings with
ModuleSettingsArray, as happens when running Logstash

Fixes #8226
2017-09-13 17:57:11 +00:00
Rob Bavey
1e4e0dbb8b Removed unnecessary Password type check
Fixes #8226
2017-09-13 17:57:11 +00:00
Rob Bavey
70844f02d3 Fix password support for Modules
Module settings should use value field for setting password in
Elasticsearch output config, not the implicit, obfuscated to_s value.

Fixes #8226
2017-09-13 17:57:11 +00:00
Jordan Sissel
acc6d50ab8 Log an error if the require fails while loading a plugin.
Fixes #6834

Fixes #8147
2017-09-13 17:50:41 +00:00
Armin
9acc61e751 BENCHMARK: Add Workercount and Batch size as configurable settings in benchmark
Fixes #8217
2017-09-13 13:27:16 +00:00
Armin
d1b053cf9b MINOR: Clean up some dead code around PQ + PQ Benchmarks
Fixes #8232
2017-09-13 13:19:19 +00:00
Rob Bavey
462e9c368f Remove wait for client sleep
Fixes #8175
2017-09-13 12:53:34 +00:00
Rob Bavey
1675eb3a0f Handle Elasticsearch errors better during DLQ integration tests
Give more time for Elasticsearch to start up, and retry after
503 responses

Fixes #8175
2017-09-13 12:53:34 +00:00
Armin
0ce82557d7 MINOR: Some DLQ Stability Fixes
Fixes #8171
2017-09-12 21:57:31 +00:00
DeDe Morton
19c04b080a Clarify that host setting requires port number
Fixes #8206
2017-09-12 21:49:52 +00:00
DeDe Morton
d20d837cdd Improve link to content about setting up config management
Fixes #8209
2017-09-12 21:44:44 +00:00
DeDe Morton
bb06af1989 Clarify description of pipeline.batch.delay
Fixes #8192
2017-09-12 21:39:51 +00:00
Armin
32b162c168 #8177 break cyclic dependency on old Logstash core in ITs and properly load it before service setup
Fixes #8181
2017-09-12 21:24:25 +00:00
Jake Landis
d2cde417d9 Revert "MINOR: Adjust Ruby Version File"
This reverts commit 3b545b261d.
The build system still needs to use .12 due some issues we found with clean environments and rake bootstrap and JRuby 9.1.13.0

Fixes #8219
2017-09-12 17:55:54 +00:00
Jake Landis
ef02b16cff RATS: Remove Docker from integration tests
This is in prep for a Docker based test run approach, and by removing the dependent Docker containers we will avoid Docker in Docker requirements.

Fixes #8211
2017-09-12 13:51:04 +00:00
Armin
fbeb49cf39 MINOR: Remove redundant throws in PipelineIR path
Fixes #8212
2017-09-12 13:42:05 +00:00
Armin
b66562342d MINOR: Cleanup JAVAC warnings around our Jruby Extensions
Fixes #8201
2017-09-12 08:05:11 +00:00
Armin
290ea7963f MINOR: Adjust Ruby Version File
Fixes #8165
2017-09-11 20:38:20 +00:00
Armin
cfddcf61bc #8128 Fix timestamp not getting properly deserialized under custom key
Fixes #8130
2017-09-11 17:14:03 +00:00
Suyog Rao
6cf0091ce5 Add info about Smart Connector
Fixes #8203
2017-09-11 16:01:47 +00:00
DeDe Morton
8d048f8678 Add --setup option to modules docs
Fixes #8191
2017-09-11 15:32:50 +00:00
Armin Braun
7083ad015e MINOR: Gitignore compiled c code in ITs
Fixes #8200
2017-09-11 14:14:21 +00:00
Armin
89af438309 PERFORMANCE: Flatten logic for write batch instantiation
Fixes #8163
2017-09-11 09:48:43 +00:00
Armin
baf1c8f03d PERFORMANCE: Remove SymbolBiValue
Fixes #8189
2017-09-11 08:15:47 +00:00
Armin
a252ec6178 #8172 Move timestamp handling for metrics to java.lang.System.nano_time
Fixes #8180
2017-09-08 14:02:16 +00:00