Commit graph

135 commits

Author SHA1 Message Date
Suyog Rao
32e4958832 Release 2.4.1 (#6207)
* Release 2.4.1

* Add release notes. Explicitly constraint docker and ansicolor
2016-11-07 12:57:03 -08:00
Joao Duarte
2aa445a690 2.4.0 release
Fixes #5842
2016-08-30 09:38:06 -04:00
Suyog Rao
36f3d4162d Release snapshot4 (#5839) 2016-08-26 20:48:01 +00:00
Suyog Rao
2ddfd848dd Add new beats input (#5824) 2016-08-24 19:18:08 +00:00
Suyog Rao
f920c4f30d Downgrade FPM and clamp versions to 2.3 levels (#5809) 2016-08-22 22:16:57 +00:00
Suyog Rao
fc4c37f954 release 2.4.0.snapshot2 (#5807) 2016-08-22 19:40:03 +00:00
Suyog Rao
f569d707a5 Release 2.4.0.snapshot1
This PR moves from 2.4.0.dev1 to snapshot1. I also published
required gems to RubyGems

Fixes #5782
2016-08-19 13:52:26 -04:00
Joao Duarte
5ef5b983af update fpm and clamp to remove .lock conflits
Fixes #5669
2016-07-21 03:49:20 -04:00
Joao Duarte
6a63375986 update plugin api to be 1.60 and not 1.21
Fixes #5601
2016-07-11 06:49:41 -04:00
Joao Duarte
a8e7e136a2 bump core to 2.4.0.dev and make plugin_api to 1.21
* bump logstash-core/logstash-core-event/logstash-core-event-java to 2.4.0.dev
* bump logstash-core-plugin-api to 1.21.0
  * provides both get/set and []/[]= events apis by allowing to be installed in logstash >= 2.4 and < 3
* minor bumps on gem dependencies

Fixes #5601
2016-07-11 06:49:41 -04:00
Colin Surprenant
cbfc93eb8d revert to logstash-core-event 2016-04-06 14:53:47 -04:00
Pier-Hugues Pellerin
219d213d1a Create a virtual gem to handler the plugin dependency on core
We have decided to create a gem that the plugins can depend on to make
sure they use the correct core, with that in mind this will reduce the
mass update required when releasing a major release of logstash if the
contract between logstash and the plugin didnt change. The first version
of this gem act as a virtual gem, future release will include the actual
plugins code of the contract.

This gem should follow the same convention as semver, a breaking change
in the api will mean a bump in the major version.  Plugins author are
encouraged to test their plugins with a specific version of the contract
and define their dependency as the following:

```
  s.add_runtime_dependency "logstash-core-plugin-api", "~> 2.0"
```

This PR also introduce rake task to help managing this gem, since every
release of logstash will produce a new version of this gem with the
minor version bump. A **logstash-core-plugin-api** will declare a strict
dependency on a specific **logstash-core** version, like this:

```
  s.add_runtime_dependency "logstash-core", "2.3.0"
```

On the other hand each release of logtash will also declare a strict
dependency on a specific version of the `logstash-core-plugin-api` gem.

ref #4829

Fixes #4881
2016-03-23 20:23:54 +00:00
Colin Surprenant
b985abebfc code cleanups and integration work for defaulting to Java Event implementation
cleaned dependencies, removed Guava code

default to java event

also compile java event

use json-unit for json result assertions

remove Java 8 specific constructs to be able to compile on Java 7

use jar-dependencies

better jar file loading

copy jar into lib after build

add post-build copy and clean of gem jar and re-indented file

fix jar destination

include generated jar in gem

remove useless ArrayList constructions

add tests for KeyNode.join method

remove useless  shadow and disable unchecked warnings

update gradle dependency to 2.8

fix deprecated asserEquals usage

simplify convoluted list contruction

fix array fields nil values handling

moar nil value tests

simplify for loop
2016-01-26 15:07:06 -05:00
Colin Surprenant
12a5982da2 logstash-core & logstash-core-event extraction to support logstash-core-event-java impl, relates to #4191
fixed timezone issue

extracted logstash-core and reorganized specs

extracted logstash-core-event

extract java Event into logstash-core-event-java in a proper gem

remove obsolete jruby_event bootstrapping

fix require path

add java code bootstrap

use logstash-core-event/logstash-core-event.rb

remove obsolete files

basic instructions

LogStash::Json need to be initialized from event

update jruby and gradle versions

update compile:logstash-core-event-java rake task

WIP tasks refactor

fix gem.files

skip test if class is not defined

fix gem related tasks for new structure

add gem spec dirs in core tests

bootstrap java implementation when requiring timestamp

new Cloner class and Event clone impl

fix array fields assignments, see #4140

don't rely on json implementation ordering

fix skipped last iterpolation char

remove implementation specific unnecessary check

also require ruby classes

define error class in ruby

raise exception on invalid format

remove implementation specific tests and extract and put logger related test in pending

missing bits for having all core timestamp specs pass

run all core specs

remove leftover

comment regex

missing encoding header

revert to logstash-core-event by default

finished proper gemification

useless require

dynamically pick specs depending on logstash-core-event-* implementation

logstash root package version

missing file for proper gemification

do not build java event by default

always check for root logstash lib dir

fix concurrent-ruby version confict

fix rebase conflict

re-enable specs

user vars instead of constants

move non core code in bootstrap

document version files

move version file

remove useless code

use version in logstash-core

fix gem files list

put back concurrent-ruby version constrain as in master

add dependency on logstash-core-event

remove dependency on logstash-core to avoid circular dependency

fix rebase conflict

remove circular dependency

fix specs

update README
2015-11-16 17:16:10 -05:00
Pier-Hugues Pellerin
2e2b96f867 Fix tins to 1.6
The 1.6 releases is the latest release to support ruby 1.9
Fixes: #4163

Fixes #4164
2015-11-11 20:03:52 +00:00
Pier-Hugues Pellerin
90b6191ec2 Change master version for 3.0.0
Fixes #3999
2015-10-08 13:57:08 +00:00
Pere Urbon-Bayes
e4c198113a Revert "merged from upstream"
This reverts commit a6ab581979.

Fixes #3951
2015-09-25 18:24:57 +00:00
guyboertje
a6ab581979 merged from upstream 2015-09-03 12:08:54 +01:00
Jordan Sissel
bb0de88583 Update devutils to 0.0.15 or newer
This ensures we have a 'try' helper method in rspec

Fixes #3836
2015-09-01 23:07:03 +00:00
Andrew Cholakian
5bb9eb9fb3 Bump stud to latest version
Fixes #3699
2015-08-04 21:26:49 +00:00
Pier-Hugues Pellerin
cbfd1eebf5 Do not force a verify_mode with a our stronger ssl settings
We have discovered that in some cases and some plaftorms
configuring a default `verify_mode` when creating a SSL/TCPServer
could make the certificate verification fail. Ruby default behavior is
to use `NIL` when creating a new ssl context, this revert that change.

keep in mind that all TCP clients using SSL **must** use `VERIFY_PEER`
as their verify mode to prevent man in the middle attack.

Fix: https://github.com/elastic/logstash/issues/3657
2015-07-31 20:56:11 -04:00
Andrew Cholakian
13ed78790f Add unicode trimmer utility function
Fixes #3505
2015-06-26 20:13:59 +00:00
Pier-Hugues Pellerin
fd4824c23f Add a micro benchmark runner
Fixes #3457
Add `rake benchmark:run` task to run micro benchmark on specific part of
the code, it's uses the benchmark-ips gem to generate gem.
See https://github.com/evanphx/benchmark-ips for usage
2015-06-26 11:11:52 -07:00
Pier-Hugues Pellerin
d85fdcb86d remove benchmark from this PR
Fixes #3425
2015-06-18 17:59:49 +00:00
Pier-Hugues Pellerin
fcf8d783fa Add a caching mechanism for the #sprintf calls
This PR allow the Event class to compile a `sprintf` the first time and
reuse the template for the other calls. This strategy speeds up considerably
calls that uses date formatting or use fieldref its a bit slower if you
dont do any string manipulation.

Fixes #3425
2015-06-18 17:59:47 +00:00
Pere Urbon-Bayes
e2450f1f78 Make the necessary changes to use rspec3 as a default test runner for
logstash, including adapting the test suites that were still using 2.x
syntax.
2015-06-08 12:19:11 +02:00
Pere Urbon-Bayes
7675a19032 Few minor improvements and pending cleanups:
* Clean up pending development dependencies from the logstash-core gemspec
* Add gems as a build dependency in the Gemfile, is used when doing an
  install-all task, and failed in the ci env (like octokit)
2015-05-11 16:01:04 +02:00
Pere Urbon-Bayes
286dac435e make rubyzip a dependency installed via the Gemfile, so the artifact:zip task works as expected 2015-05-06 15:46:10 +02:00
Colin Surprenant
7a1ec64c23 use :build dependencies
fix octokit dep

fixes #3171
2015-05-06 08:27:57 -04:00
Colin Surprenant
48bcc5b74d cleanup dependencies and rely on logstash-core gem
remove unused load path

use either local core lib dir or logstash-core gem

include spec in logstash-core

do not include lib, spec and locales dirs in package root

environment bootstrapping in lib/bootstrap.rb

added comments

move pluginmanager out of logstash-core

kafla installation solved by pluginmanager refactor

refactor bootstrap code

refactor shell scripts to simplify and DRY, indirect rspec command to our ruby launcher

add bin/rspec

cut by half the bin/plugin and bin/rspec startup time

fix drip context

use printf instead of tr

updated Windows bin scripts

missing gemspec in gemspec

use gem instead of gemspec so our plugin manager can correctly install logstash-core

generate packages including locally built logstash-core gem

move jenkins developmnent dependencies into Gemfile, they do not belong in logstash-core

path leftover

clean help agent help usage message and remore rspec in available command

comments cosmetics

update Bundler dependency, all recent testing have been made with 1.9.4

updated .lock file with regard to updated Gemfile

cleanup gemspec, Gemfile and regen .lock file

added progress output

avoid dual require on version

closes #3096
2015-05-04 18:17:22 -04:00
Pere Urbon-Bayes
92655105b4 Create a logstash-core gem and made the necessary adoptions to logstash so test can he also run
fix the logstash-core gemspec gem.name

Fixes #2677
2015-02-26 14:26:51 +00:00
Pier-Hugues Pellerin
9e7d6204a5 plugins/bundler refactor
squashed 113 commits by @ph, @colinsurprenant, @jordansissel, @purbon, @jsvd

cleaning the environment.rb

initial work

remove multiples polyglot

force to load rubygems

notice

point to one config

update lock

bootstrap will package the standard application until we have the plugin manager on top of the gemfile

add the gem path to find bundler

fixing dependencies

add gemfiles and bundle

comment platform requiremments in the gemfile

Set `Gem.paths = ...` to the GEM_HOME

Without this, `bin/logstash` will error:

    LoadError: no such file to load -- bundler
         require at org/jruby/RubyKernel.java:1071
         require at C:/Users/jls/Documents/GitHub/logstash/vendor/jruby/lib/ruby/shared/rubygems/core_ext/kernel_require
.rb:55
         set_gem_paths! at C:/Users/jls/Documents/GitHub/logstash/lib/logstash/environment.rb:64
         (root) at C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:8

Patch JRuby's jar_installer.rb with a newer version

jar-dependencies is shipped by default with JRuby. However, the version
it ships with has a bug, and despite upgrading jar-dependencies to
0.1.7, which has the fix, JRuby continues to load the "shipped" version
which has a bug.

This patch will patch JRuby by copying in our own jar_installer.rb from
jar-dependencies master, This patching occurs while unpacking JRuby just
after `vendor:jruby`.

The jar_installer.rb is MIT-licensed and no changes were made.

Add `test:plugins` to test all installed plugins

Tell rspec to do the wildcards instead of trying to make the shell do it.

This fixes test runs on Windows (which doesn't do wildcards like this).
The solution was to tell RSpec to look for all files matching a certain path pattern
within the GEM_HOME.

do not run bundle install twice for test:plugins

Gemfile manipulation classes

Gemfile now in root

install POC with new Gemfile classes

specs for the Gemfile classes

header, some bug fixes

spec header

WIP pluginmanager refactor

generated format

updated .lock

unused file

removed --proxy option

fail elegantly

initial support for update

refactor invoke_bundler, debug output

removed old .lock file

no plugin Gemfile

use clamp error handling

reenable list, some cleanups

support for more finders

refactored using new finders

refactor using is_installed_plugin? and some cleanups

verify valid plugin and update all now only update plugins not all gems

support multiple plugins argument, --force option to bypass validation

list only explicitly installed plugins by default, --all option to include dependencies

support multiple plugins argument

renamed all_installed_plugins to all_installed_plugins_gem_specs and return specs, added comments

remove then

update_gem bug, comments

moved gems as development dependencies

updated .lock file

support for :without

--development option, better output feedback, cleanups

abort when no plugins

cosmetic

param fix, cleanups

removed deprecated .lock files

remove deprecated bundler_install_command and struggling with set_gem_paths

refactored all bundler stuff into LogStash::Bundler

rakelib refactor

refactored bundler code

Bundler.real_fuckin_reset

real_fuckin_reset is now reset_settings, removed unnecessary resetings, cleanups

duplicate plugins requirements merging

cleanups

added vendor:rake, cleanups

broke test tasks into 2 steps, first invoke install then test

rake task name fix

proper error exit code

Freezing the jar-dependencies, ruby-maven, tools

set gem paths on error

renamed TEST_PLUGINS to CORE_PLUGINS, added comments

replace test by core, cleanups

DRYied test tasks

missing gemspec

fixed output

correct image build for artifacts

removed deprecated Gemfiles

fix update with multiple plugins arguments

fix update with multiple plugins arguments

remove deprecated code

implicit begin, explicit block

spec for LogStash::Bundler

remove explit require of rubygems

renamed Environment.set_gem_paths! to bundler_setup!

rename Dsl to DSL

respect to_s contract and return empty string not nil

rename DSL.parse gemfile argument to gemfile_content

better error message

renamed --force to --[no-]verify

refactored exit error handling

use explicit array items in iterators

fix --development switch validation, cleaned bundler_options hash init

renamed is_xxx? methods to xxx?

use direct method call on collection

removed double negative, typo

move maven tools monkey patch into maven_tools_patch.rb

renamed build-image to prepare

fix forgotten argument name change

namespace cleanups

spec files glob pattern

cleanup

cleanup

cleanup

Fix the file-dependencies version to 0.1.5 to fix path issues on windows

monkeypatch Rubygems to silence ffi warning

list all plugin gems by default, new options

adding rake task to get faster feedback loop on windows

refactor bat scripts

Use an unreleased filedependencies gem and skip installing drupal

bump file-dependencies

use the released gem of file-dependencies

Add a set of scripts that simplify:   * The necessary setup for the CI test to be executed.   * The actuall CI test execution, having a different scripts for each     different workflow. This makes the test execution simpler and standard, so changes in internal task don't need big changes in the CI config. It also documents and makes easy for developers to run the tests.

Fixes #2640

refactored test to not depend on STDOUT

create a .rspec file to pass some options used while the code is in the testing environment

add coveralls and simplecov as a development dependencies so we can run coverage tests

remove unnecessary .bundle directory from the artifact

In a release, the .bundle/config has a BUNDLE_PATH setting that can
interfere by misleading the plugin manager into not seeing the
vendor/bundle path.

Fixes #2647

Fixes #2475
2015-02-20 17:31:43 +00:00
Joao Duarte
68c03cf19e move plugins gemfile to tools
Fixes #2171
2014-12-04 17:09:56 +00:00
Joao Duarte
eff219a003 install default plugins via bundler
Fixes #2107
2014-12-01 19:37:57 +00:00
Jordan Sissel
bd8caf9874 Use bundler.
'bundle install' when already installed should be faster than the
previous gembag. More importantly, it is also more accurate in terms of
dependencies.

Bonus points that we can do 'clean' which will purge any unknown gems
and keep our gems directory nice and tidy.

The main purpose is to speed up repetitive build invocations. Bundler's
dependency resolver is better than that of the previous gembag.rb, and
its 'clean' feature lets bundler manage the gems listing - even between
builds. This lets not have to 'rm -rf vendor' between builds on jenkins
to ensure quality testing :)

Other changes:
- Hide the Gemfile and Gemfile.lock in the 'tools' directory. This
  should hopefully prevent users from accidentally updating these files
  and thus avoiding merge conflicts later.
- Minor patch to use RbConfig::CONFIG['ruby_version'] instead of
  RUBY_VERSION. Confusing, I know. The former is the 'ruby abi' version,
  it seems, and the latter is the actual ruby implementation version.
  Example, on MRI 1.9.3, the former is 1.9.1 and latter is 1.9.3. On
  JRuby, the former is 1.9 and latter is 1.9.3. COMPUTERS. HELP.
2013-12-02 23:59:19 -08:00
Jordan Sissel
5923d76a42 fiddling with guard 2013-08-15 13:21:59 -07:00
Jordan Sissel
087b66fd43 - clean out gemfile (Should use only the gemspec now) 2012-08-29 16:44:03 -07:00
Jordan Sissel
3b45a4f45a - remove exec bit 2012-08-29 16:44:03 -07:00
Jordan Sissel
2049634624 set gemspec name now that there are multiple 2012-08-16 11:53:24 -07:00
Grant Rodgers
25303ceec6 add gemspec and use it in Gemfile 2012-07-12 15:22:04 -07:00
Pete Fritchman
db17c68fd0 support setting ES node name 2012-07-05 21:03:02 -04:00
Jordan Sissel
67d3c79e4e Merge pull request #166 from nickethier/feature/irc
IRC Input/Output
2012-07-02 17:02:19 -07:00
John E. Vincent
409611a729 updating riak output with better support 2012-07-01 03:47:43 -04:00
Nick Ethier
6223d6a595 update Gemfile for support 2012-06-30 14:35:25 -06:00
John E. Vincent
7f6f004b4b experimental riak output 2012-06-25 23:34:50 -04:00
Pete Fritchman
33efbe1469 latest jls-grok to fix LOGSTASH-300 2012-06-24 18:13:14 -04:00
Pete Fritchman
05a225a27b add pry command, too 2012-06-20 14:27:19 -04:00
Harlan Barnes
cd2677ddac xml.rb - XML Filter
Supports parsing whole XML into destination field and/or parsing
for specific strings using XPath.
2012-06-18 03:35:42 -04:00
Jordan Sissel
41662ffff5 - update fpm to 0.0.19; fixes a header bug where both content-length and
transfer-encoding would be incorrectly specified (LOGSTASH-487, and
  LOGSTASH-483)
2012-06-06 14:01:24 -07:00
Jordan Sissel
5cb6d8fef2 Upgrade jruby-elasticsearch 2012-06-01 09:55:50 -07:00