Commit graph

27 commits

Author SHA1 Message Date
Colin Surprenant
ab05082a89 merge feature/java_persistence into 5.x 2016-10-31 17:03:40 -04:00
Pier-Hugues Pellerin
23e87bc44f Remove spec/integration
With the addition of the qa acceptance test suite we have decide that we
don't need to maintain this suite also since the acceptance test are
running really similar test.

Fixes: #5744

Fixes #5760
2016-08-14 15:09:29 -04:00
Pere Urbon-Bayes
834d52a9aa refactor integration test to be inside the specs directory and not in the main level
Fixes #5268
2016-05-12 06:56:25 +00:00
Colin Surprenant
d74d41cb30 logstash-core & logstash-core-event extraction to support logstash-core-event-java impl, relates to #4191
fixed timezone issue

extracted logstash-core and reorganized specs

extracted logstash-core-event

extract java Event into logstash-core-event-java in a proper gem

remove obsolete jruby_event bootstrapping

fix require path

add java code bootstrap

use logstash-core-event/logstash-core-event.rb

remove obsolete files

basic instructions

LogStash::Json need to be initialized from event

update jruby and gradle versions

update compile:logstash-core-event-java rake task

WIP tasks refactor

fix gem.files

skip test if class is not defined

fix gem related tasks for new structure

add gem spec dirs in core tests

bootstrap java implementation when requiring timestamp

new Cloner class and Event clone impl

fix array fields assignments, see #4140

don't rely on json implementation ordering

fix skipped last iterpolation char

remove implementation specific unnecessary check

also require ruby classes

define error class in ruby

raise exception on invalid format

remove implementation specific tests and extract and put logger related test in pending

missing bits for having all core timestamp specs pass

run all core specs

remove leftover

comment regex

missing encoding header

revert to logstash-core-event by default

finished proper gemification

useless require

dynamically pick specs depending on logstash-core-event-* implementation

logstash root package version

missing file for proper gemification

do not build java event by default

always check for root logstash lib dir

fix concurrent-ruby version confict

fix rebase conflict

re-enable specs

user vars instead of constants

move non core code in bootstrap

document version files

move version file

remove useless code

use version in logstash-core

fix gem files list

put back concurrent-ruby version constrain as in master

add dependency on logstash-core-event

remove dependency on logstash-core to avoid circular dependency

fix rebase conflict

remove circular dependency

fix specs

update README
2015-11-16 16:40:19 -05:00
Pier-Hugues Pellerin
a30b9a905e Add the possibility to run local integration test
Fixes #4166
2015-11-13 17:16:49 +00:00
Gabriel Moskovicz
abd97aa98b Add powershell script for windows testing
- Release acceptance test, which tests ports and run Logstash.
- Default plugins, which runs the same plugin installation and
environment for windows
- Simple integration, which integrates Logstash with Elasticsearch and a
simple configuration

Fixes #3994
2015-11-06 10:29:57 -08:00
guyboertje
7bb94d0692 add test for compact and compact! in java_integrations
Fixes #3772
2015-09-03 19:43:10 +00:00
Pere Urbon-Bayes
cf10890d63 fix the coverage analysis throw simplecov to take care of all files
make the eager loading patterns consistent in the spec_helper.rb file

make the setup-simplecov task not a dependency, but an explicit task only executed when ENV['COVERAGE'] is defined

refactor eager loading code plus add some documentation to the setup-simplecov task

Added more comments to the test:setup task

Fixes #3465
2015-06-25 15:36:57 +00:00
Pere Urbon-Bayes
e2450f1f78 Make the necessary changes to use rspec3 as a default test runner for
logstash, including adapting the test suites that were still using 2.x
syntax.
2015-06-08 12:19:11 +02:00
Colin Surprenant
7a1ec64c23 use :build dependencies
fix octokit dep

fixes #3171
2015-05-06 08:27:57 -04:00
Colin Surprenant
48bcc5b74d cleanup dependencies and rely on logstash-core gem
remove unused load path

use either local core lib dir or logstash-core gem

include spec in logstash-core

do not include lib, spec and locales dirs in package root

environment bootstrapping in lib/bootstrap.rb

added comments

move pluginmanager out of logstash-core

kafla installation solved by pluginmanager refactor

refactor bootstrap code

refactor shell scripts to simplify and DRY, indirect rspec command to our ruby launcher

add bin/rspec

cut by half the bin/plugin and bin/rspec startup time

fix drip context

use printf instead of tr

updated Windows bin scripts

missing gemspec in gemspec

use gem instead of gemspec so our plugin manager can correctly install logstash-core

generate packages including locally built logstash-core gem

move jenkins developmnent dependencies into Gemfile, they do not belong in logstash-core

path leftover

clean help agent help usage message and remore rspec in available command

comments cosmetics

update Bundler dependency, all recent testing have been made with 1.9.4

updated .lock file with regard to updated Gemfile

cleanup gemspec, Gemfile and regen .lock file

added progress output

avoid dual require on version

closes #3096
2015-05-04 18:17:22 -04:00
Colin Surprenant
ae17b4160d refactored all gem/bundler code into logstash/bundler.rb and moved patches into logstash/patches
require bundler/cli to expose expectations classes
2015-04-16 17:34:51 +02:00
Colin Surprenant
380ea33708 refactor test tasks, test:plugins now correctly run specs of installed plugins, including local ones
robustness in shutdown handling and other thread safeties
2015-04-08 08:43:49 -04:00
Pere Urbon-Bayes
fb88eef749 add a check for the exit code, so if the exit code is not zero we make the rake task to fail otherwise it pass always causing false positive
move to use exit insted of fail as it aborts the test execution

add a comment to clarify why we set the exit code in the rake task

Fixes #2920
2015-03-31 08:57:10 +00:00
Pere Urbon-Bayes
d44b58bcdc add an option to bundler_setup! to handle testing environment where development dependencies should not be excluded in order to run proper tests
as a request during review, I made the signature of bundler_setup use the same as invoke_bundler

remove empty line in bundler_setup

ammend last commit

Fixes #2726
2015-03-02 16:05:54 +00:00
Pier-Hugues Pellerin
9e7d6204a5 plugins/bundler refactor
squashed 113 commits by @ph, @colinsurprenant, @jordansissel, @purbon, @jsvd

cleaning the environment.rb

initial work

remove multiples polyglot

force to load rubygems

notice

point to one config

update lock

bootstrap will package the standard application until we have the plugin manager on top of the gemfile

add the gem path to find bundler

fixing dependencies

add gemfiles and bundle

comment platform requiremments in the gemfile

Set `Gem.paths = ...` to the GEM_HOME

Without this, `bin/logstash` will error:

    LoadError: no such file to load -- bundler
         require at org/jruby/RubyKernel.java:1071
         require at C:/Users/jls/Documents/GitHub/logstash/vendor/jruby/lib/ruby/shared/rubygems/core_ext/kernel_require
.rb:55
         set_gem_paths! at C:/Users/jls/Documents/GitHub/logstash/lib/logstash/environment.rb:64
         (root) at C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:8

Patch JRuby's jar_installer.rb with a newer version

jar-dependencies is shipped by default with JRuby. However, the version
it ships with has a bug, and despite upgrading jar-dependencies to
0.1.7, which has the fix, JRuby continues to load the "shipped" version
which has a bug.

This patch will patch JRuby by copying in our own jar_installer.rb from
jar-dependencies master, This patching occurs while unpacking JRuby just
after `vendor:jruby`.

The jar_installer.rb is MIT-licensed and no changes were made.

Add `test:plugins` to test all installed plugins

Tell rspec to do the wildcards instead of trying to make the shell do it.

This fixes test runs on Windows (which doesn't do wildcards like this).
The solution was to tell RSpec to look for all files matching a certain path pattern
within the GEM_HOME.

do not run bundle install twice for test:plugins

Gemfile manipulation classes

Gemfile now in root

install POC with new Gemfile classes

specs for the Gemfile classes

header, some bug fixes

spec header

WIP pluginmanager refactor

generated format

updated .lock

unused file

removed --proxy option

fail elegantly

initial support for update

refactor invoke_bundler, debug output

removed old .lock file

no plugin Gemfile

use clamp error handling

reenable list, some cleanups

support for more finders

refactored using new finders

refactor using is_installed_plugin? and some cleanups

verify valid plugin and update all now only update plugins not all gems

support multiple plugins argument, --force option to bypass validation

list only explicitly installed plugins by default, --all option to include dependencies

support multiple plugins argument

renamed all_installed_plugins to all_installed_plugins_gem_specs and return specs, added comments

remove then

update_gem bug, comments

moved gems as development dependencies

updated .lock file

support for :without

--development option, better output feedback, cleanups

abort when no plugins

cosmetic

param fix, cleanups

removed deprecated .lock files

remove deprecated bundler_install_command and struggling with set_gem_paths

refactored all bundler stuff into LogStash::Bundler

rakelib refactor

refactored bundler code

Bundler.real_fuckin_reset

real_fuckin_reset is now reset_settings, removed unnecessary resetings, cleanups

duplicate plugins requirements merging

cleanups

added vendor:rake, cleanups

broke test tasks into 2 steps, first invoke install then test

rake task name fix

proper error exit code

Freezing the jar-dependencies, ruby-maven, tools

set gem paths on error

renamed TEST_PLUGINS to CORE_PLUGINS, added comments

replace test by core, cleanups

DRYied test tasks

missing gemspec

fixed output

correct image build for artifacts

removed deprecated Gemfiles

fix update with multiple plugins arguments

fix update with multiple plugins arguments

remove deprecated code

implicit begin, explicit block

spec for LogStash::Bundler

remove explit require of rubygems

renamed Environment.set_gem_paths! to bundler_setup!

rename Dsl to DSL

respect to_s contract and return empty string not nil

rename DSL.parse gemfile argument to gemfile_content

better error message

renamed --force to --[no-]verify

refactored exit error handling

use explicit array items in iterators

fix --development switch validation, cleaned bundler_options hash init

renamed is_xxx? methods to xxx?

use direct method call on collection

removed double negative, typo

move maven tools monkey patch into maven_tools_patch.rb

renamed build-image to prepare

fix forgotten argument name change

namespace cleanups

spec files glob pattern

cleanup

cleanup

cleanup

Fix the file-dependencies version to 0.1.5 to fix path issues on windows

monkeypatch Rubygems to silence ffi warning

list all plugin gems by default, new options

adding rake task to get faster feedback loop on windows

refactor bat scripts

Use an unreleased filedependencies gem and skip installing drupal

bump file-dependencies

use the released gem of file-dependencies

Add a set of scripts that simplify:   * The necessary setup for the CI test to be executed.   * The actuall CI test execution, having a different scripts for each     different workflow. This makes the test execution simpler and standard, so changes in internal task don't need big changes in the CI config. It also documents and makes easy for developers to run the tests.

Fixes #2640

refactored test to not depend on STDOUT

create a .rspec file to pass some options used while the code is in the testing environment

add coveralls and simplecov as a development dependencies so we can run coverage tests

remove unnecessary .bundle directory from the artifact

In a release, the .bundle/config has a BUNDLE_PATH setting that can
interfere by misleading the plugin manager into not seeing the
vendor/bundle path.

Fixes #2647

Fixes #2475
2015-02-20 17:31:43 +00:00
Pere Urbon-Bayes
d479015abb fix a typo on the test all plugins rake task
Fixes #2441
2015-01-27 17:17:21 +00:00
Pere Urbon-Bayes
002104f3c8 Added review comments and improvements
fix the logstash_gem_path

added some love from @jsvd comments

added an LS_ENV variable that if set to test it install the plugin dependencies when running the plugin:install-all rake task2

made the vendor:deps task has a more concret behaviour and proper naming

cleanup the all plugins install to only do the installation

added some code to fetch dependencies throw some gemcode within the gemfile as @electrical proposed

Revert "added some code to fetch dependencies throw some gemcode within the gemfile as @electrical proposed"

This reverts commit 20c3243485129c388d1da4ac930a479f017e0d99.

fix a path on the run all plugins test rake task

Fixes #2371
2015-01-21 16:47:26 +00:00
Pere Urbon-Bayes
f3912f81db Applyed all necessary review improvements fix the gempath
update the test:plugins to test:all-plugins at it makes more sense

fix the path to run the bin logstash file

added order rand

Fixes #2373
2015-01-19 19:41:46 +00:00
Pere Urbon-Bayes
ec97d3d78c add a way to run all plugin test from a rake command
Fixes #2373
2015-01-19 19:41:46 +00:00
Pere Urbon-Bayes
b1a4c102da make sure the fail fast test also have the bootstrap for tests done
Fixes #2237
2014-12-17 17:31:32 +00:00
Pere Urbon-Bayes
d63e0ec9eb make sure the bundler task is just used once while bootstraping tests
Fixes #2237
2014-12-17 17:31:32 +00:00
Pere Urbon-Bayes
93aadb4d4c Make the test plugins install throw bundler, so we don't have to install them every time we run the test (it makes the thing slow). Refactor also the bundler install task into a single one, making calls to it when necessary.
Fixes #2237
2014-12-17 17:31:32 +00:00
Pere Urbon-Bayes
21cb877ec9 clear the gem paths before the test rake tasks so they find previously installed plugins
Fixes #2235
2014-12-17 12:52:29 +00:00
Richard Pijnenburg
f472d4af24 Cleanup plugins to move to plugin system
Plugins have moved to https://github.com/logstash-plugins/... into
individual repositories. One plugin per repo.

* Add plugin install task and add required plugins for testing
* Adding a plugin task to install plugins
* Added required plugins to a prep task for the test part.

Fixes #1984
2014-11-04 23:24:59 +00:00
Jordan Sissel
f35558e395 Make most things depend on bootstrap
Fixes #1940
2014-10-22 20:04:01 +00:00
Jordan Sissel
920a9e9362 Refactor build tooling to use Ruby
The new way to build is using 'rake' and not 'make'.

For some some attempt at compatibility, any 'make' invocation will
invoke rake. `make foo` will invoke `rake foo`.

This is for #1640

Major changes:
* rake is used instead of make. This allows us to more easily improve,
  debug, and test the build tooling.
* The build process can be bootstrapped from any Ruby that has
  Rake available. Bootstrapping rake is automatic will download JRuby and
  re-execute itself under JRuby.
* rpm and deb packages no longer list any java as dependencies. This
  is the result of user discomfort and discussions which concluded
  that there is no good way to "depend" on Java via rpm or deb.

Internal changes:
* Vendoring paths are project-based. Each vendor directory is now
  specific as vendor/<project>/... For example, JRuby is dumped into
  vendor/jruby/...

Other changes:

* We don't use jruby-complete anymore. We now use bin/jruby as the entry point.
  This improves the startup time of Logstash on my laptop by approximately
  1.5 seconds (~25% speedup of startup time). Testing `bin/logstash
  version` speed; 5 runs each:

  * Before patch: 7.27 7.55 7.26 7.42 7.47
  * After patch: 5.54 5.97 5.67 5.58 5.54

Smaller changes:
- removed 'env' subcommand
- Remove 'deps' subcommand. Replaced by 'rake vendor:all'
- Add gem_home method for this platform's GEM_HOME
- .travis.yml removed because we haven't used Travis in a long time.
- bin/logstash exits nonzero with a useful message if JRuby can't be found.

Notes:

Invoke bundler via system() call instead of Bundler::CLI.
After some discussion in #bundler on freenode (very helpful!) on how
to do what we need to do with bundler, I was encouraged to not use the
bundler api from ruby, so system() it is.

Patch slightly modified to fit the new model of packaging (ruby instead
of shell invocation), but original author is obazoud in #1008.

Richard Pijnenburg helped with some of the code and was lovely in
helping test it.

Fixes #1640
2014-10-22 17:04:55 +00:00