The module LogStash::PluginManager requires the file `lib/pluginmanager/plugin_aliases.yml` is created,
which happend during the Gradle's 'copyPluginAlias' executed as part of Rake's 'bootstrap'.
* simplify the plugins-metadata.json file
* sort and update the plugin list in the rakelib/plugins-metadata.json
* remove dependency on twitter input for testing
* sorted Gemfile.template (grouped by group)
* remove default plugins from Gemfile.template
Fixes#10509
* bump jruby to 9.2
* don't rely on logstash-base docker image
* work around webmock ruby 2.5 support
* ensure data folder exists in docker
* change fixnum and bignum to integer
* FileUtils.rmdir to rm_rf
this is because from 2.3 to 2.5 FileUtils.rmdir will throw an exception
if the directory isn't empty. On 2.3 the operation will just not delete
the directory silently.
* bump jruby to 9.2.5.0 and fix test
* make rake default task since prepare pack needs it
* Resolve compiler warnings (#10247)
There are 3 types of compiler warnings that are either resolved or suppressed:
1. Rawtypes: In JRuby 9.2, `RubyArray` is a generic, so references throughout
our codebase to the now "raw" type trigger warnings. In most cases we cannot
actually resolve the issue, since the JRuby-provided methods for creating
`RubyArray`s still return the raw type, so these have been suppressed.
2. Deprecations:
- `RubyString#intern19()` -> `RubyString#intern()`
- `RubyString#downcase19(ThreadContext)` -> `RubyString#downcase(ThreadContext)`
- `NativeException`: remove import & reference directly; suppress usage
warnings
- `RaiseException()`: migrate to equivalent non-deprecated methods wherever
possible; in some cases where we are using this in conjunction with the
also-deprecated `NativeException` to preserve java stacktraces, there
seems to be no non-deprecated path forward, so these cases have been
suppressed.
3. Redundant Casts
- Resolved
* JRuby 9.2 bundler shenanigans (#10266)
* Revert "Revert "remove forced dependency on old bundler (#9395)""
This reverts commit bef984143d.
* plugin management: update internal bundler to 1.17.x APIs
* deps: update dev dependency webmock to version compatible with JRuby 9.2
* spec: update Pack fixture to include manticore version that doesn't conflict
* build: update gradle to version that has Java 11 support
* java11: resolve or suppress deprecation warnings
* Remove superfluous flag opting into ParNew GC implementation
When opting into CMS garbage collector with `XX:+UseConcMarkSweepGC`, the
young generation collector ParNew has been the default since Java 8, making
the `XX:+UseParNew` flag redundant; the flag was removed in Java 9, and
should no longer be specified to work with modern Javas.
https://bugs.openjdk.java.net/browse/JDK-8006478https://openjdk.java.net/jeps/214
* spec: set thread name to example description for easier debugging
* spec: prevent errors in testing specs by checking against skip list before using
* no-op: remove use of `HashMap#computeIfAbsent` on single-threaded code
> This method will, on a best-effort basis, throw a `ConcurrentModificationException`
> if it is detected that the mapping function modifies this map during computation.
>
> -- https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/HashMap.html#computeIfAbsent(K,java.util.function.Function)
* qa: by default, run integration against Elastic Stack 6.5.x
To support development on Logstash on top of Java 11, default to testing
against an Elastic Stack that is capable of running on Java 11.
* qa: ignore deprecation warnings when comparing offline pack output
* qa: add Java 9+ support to ChildProcess dev dependency
this can safely be removed when the childprocess gem supports Java9+
https://github.com/enkessler/childprocess/pull/141
* qa: allow connections to localhost in webmock
* bump jrjackson version
* fix filebeat integration tests
* spec: ensure license compliance spec runs first
The license compliance spec that validates the licenses of bundled
plugins appears to not be compatible with the hooks that we inject
into bundler for plugin management, and will fail in obscure ways
when run after those hooks have been added. Since those hooks are
not necessary for validating licenses, the easiest solution was to
ensure that those specs run first, before the VM has been poluted.
Since the gradle/junit/rspec bridge that is currently in place
runs all specs in the same JVM, we also need to make sure that the
rspec "world" is reset before a run, to ensure that it doesn't
retain spec definitions from previous runs.
Also updates the rake invocation, although I'm not sure it is used
any more.
final (I hope) fixes for consolidated versioning.
After testing with rake artifact:zip, need to try both files.
Add desc to rake test:install-* tasks, tired of
not seeing them in rake -vT
changes requested via review
Fixes#8373
This commit includes:
* A base Dockerfile and script to push to a Docker repo
* A per-build Dockerfile (derived from the base)
* Updates to the test scripts to allow for more parallel builds
* Docker wrappers for the tests scripts
* Update for the integration test readme to manually run the tests
* Clean up the output of the Java tests
* Remove offline tag for tests (no longer needed that we don't use docker dependent services)
This commit does NOT include:
* Changes needed for the CI system to use Docker
Fixes#8223
Work done by @guyboertje and @ph
Since JRuby 1.7.25 is now EOL we are migrating Logstash to use JRuby 9k and JDK8 only,
Not much needed updating to make this work, its was mostly a drop in replacement from the previous version.
The major point was the change in the implementation of Time in JRuby, JRuby now use `java.time`
instead of joda time, this allow JRuby to have nanoseconds precision on time object.
With the addition of the qa acceptance test suite we have decide that we
don't need to maintain this suite also since the acceptance test are
running really similar test.
Fixes: #5744Fixes#5760
fixed timezone issue
extracted logstash-core and reorganized specs
extracted logstash-core-event
extract java Event into logstash-core-event-java in a proper gem
remove obsolete jruby_event bootstrapping
fix require path
add java code bootstrap
use logstash-core-event/logstash-core-event.rb
remove obsolete files
basic instructions
LogStash::Json need to be initialized from event
update jruby and gradle versions
update compile:logstash-core-event-java rake task
WIP tasks refactor
fix gem.files
skip test if class is not defined
fix gem related tasks for new structure
add gem spec dirs in core tests
bootstrap java implementation when requiring timestamp
new Cloner class and Event clone impl
fix array fields assignments, see #4140
don't rely on json implementation ordering
fix skipped last iterpolation char
remove implementation specific unnecessary check
also require ruby classes
define error class in ruby
raise exception on invalid format
remove implementation specific tests and extract and put logger related test in pending
missing bits for having all core timestamp specs pass
run all core specs
remove leftover
comment regex
missing encoding header
revert to logstash-core-event by default
finished proper gemification
useless require
dynamically pick specs depending on logstash-core-event-* implementation
logstash root package version
missing file for proper gemification
do not build java event by default
always check for root logstash lib dir
fix concurrent-ruby version confict
fix rebase conflict
re-enable specs
user vars instead of constants
move non core code in bootstrap
document version files
move version file
remove useless code
use version in logstash-core
fix gem files list
put back concurrent-ruby version constrain as in master
add dependency on logstash-core-event
remove dependency on logstash-core to avoid circular dependency
fix rebase conflict
remove circular dependency
fix specs
update README
- Release acceptance test, which tests ports and run Logstash.
- Default plugins, which runs the same plugin installation and
environment for windows
- Simple integration, which integrates Logstash with Elasticsearch and a
simple configuration
Fixes#3994
make the eager loading patterns consistent in the spec_helper.rb file
make the setup-simplecov task not a dependency, but an explicit task only executed when ENV['COVERAGE'] is defined
refactor eager loading code plus add some documentation to the setup-simplecov task
Added more comments to the test:setup task
Fixes#3465
remove unused load path
use either local core lib dir or logstash-core gem
include spec in logstash-core
do not include lib, spec and locales dirs in package root
environment bootstrapping in lib/bootstrap.rb
added comments
move pluginmanager out of logstash-core
kafla installation solved by pluginmanager refactor
refactor bootstrap code
refactor shell scripts to simplify and DRY, indirect rspec command to our ruby launcher
add bin/rspec
cut by half the bin/plugin and bin/rspec startup time
fix drip context
use printf instead of tr
updated Windows bin scripts
missing gemspec in gemspec
use gem instead of gemspec so our plugin manager can correctly install logstash-core
generate packages including locally built logstash-core gem
move jenkins developmnent dependencies into Gemfile, they do not belong in logstash-core
path leftover
clean help agent help usage message and remore rspec in available command
comments cosmetics
update Bundler dependency, all recent testing have been made with 1.9.4
updated .lock file with regard to updated Gemfile
cleanup gemspec, Gemfile and regen .lock file
added progress output
avoid dual require on version
closes#3096
as a request during review, I made the signature of bundler_setup use the same as invoke_bundler
remove empty line in bundler_setup
ammend last commit
Fixes#2726
squashed 113 commits by @ph, @colinsurprenant, @jordansissel, @purbon, @jsvd
cleaning the environment.rb
initial work
remove multiples polyglot
force to load rubygems
notice
point to one config
update lock
bootstrap will package the standard application until we have the plugin manager on top of the gemfile
add the gem path to find bundler
fixing dependencies
add gemfiles and bundle
comment platform requiremments in the gemfile
Set `Gem.paths = ...` to the GEM_HOME
Without this, `bin/logstash` will error:
LoadError: no such file to load -- bundler
require at org/jruby/RubyKernel.java:1071
require at C:/Users/jls/Documents/GitHub/logstash/vendor/jruby/lib/ruby/shared/rubygems/core_ext/kernel_require
.rb:55
set_gem_paths! at C:/Users/jls/Documents/GitHub/logstash/lib/logstash/environment.rb:64
(root) at C:\Users\jls\Documents\GitHub\logstash\lib\logstash\runner.rb:8
Patch JRuby's jar_installer.rb with a newer version
jar-dependencies is shipped by default with JRuby. However, the version
it ships with has a bug, and despite upgrading jar-dependencies to
0.1.7, which has the fix, JRuby continues to load the "shipped" version
which has a bug.
This patch will patch JRuby by copying in our own jar_installer.rb from
jar-dependencies master, This patching occurs while unpacking JRuby just
after `vendor:jruby`.
The jar_installer.rb is MIT-licensed and no changes were made.
Add `test:plugins` to test all installed plugins
Tell rspec to do the wildcards instead of trying to make the shell do it.
This fixes test runs on Windows (which doesn't do wildcards like this).
The solution was to tell RSpec to look for all files matching a certain path pattern
within the GEM_HOME.
do not run bundle install twice for test:plugins
Gemfile manipulation classes
Gemfile now in root
install POC with new Gemfile classes
specs for the Gemfile classes
header, some bug fixes
spec header
WIP pluginmanager refactor
generated format
updated .lock
unused file
removed --proxy option
fail elegantly
initial support for update
refactor invoke_bundler, debug output
removed old .lock file
no plugin Gemfile
use clamp error handling
reenable list, some cleanups
support for more finders
refactored using new finders
refactor using is_installed_plugin? and some cleanups
verify valid plugin and update all now only update plugins not all gems
support multiple plugins argument, --force option to bypass validation
list only explicitly installed plugins by default, --all option to include dependencies
support multiple plugins argument
renamed all_installed_plugins to all_installed_plugins_gem_specs and return specs, added comments
remove then
update_gem bug, comments
moved gems as development dependencies
updated .lock file
support for :without
--development option, better output feedback, cleanups
abort when no plugins
cosmetic
param fix, cleanups
removed deprecated .lock files
remove deprecated bundler_install_command and struggling with set_gem_paths
refactored all bundler stuff into LogStash::Bundler
rakelib refactor
refactored bundler code
Bundler.real_fuckin_reset
real_fuckin_reset is now reset_settings, removed unnecessary resetings, cleanups
duplicate plugins requirements merging
cleanups
added vendor:rake, cleanups
broke test tasks into 2 steps, first invoke install then test
rake task name fix
proper error exit code
Freezing the jar-dependencies, ruby-maven, tools
set gem paths on error
renamed TEST_PLUGINS to CORE_PLUGINS, added comments
replace test by core, cleanups
DRYied test tasks
missing gemspec
fixed output
correct image build for artifacts
removed deprecated Gemfiles
fix update with multiple plugins arguments
fix update with multiple plugins arguments
remove deprecated code
implicit begin, explicit block
spec for LogStash::Bundler
remove explit require of rubygems
renamed Environment.set_gem_paths! to bundler_setup!
rename Dsl to DSL
respect to_s contract and return empty string not nil
rename DSL.parse gemfile argument to gemfile_content
better error message
renamed --force to --[no-]verify
refactored exit error handling
use explicit array items in iterators
fix --development switch validation, cleaned bundler_options hash init
renamed is_xxx? methods to xxx?
use direct method call on collection
removed double negative, typo
move maven tools monkey patch into maven_tools_patch.rb
renamed build-image to prepare
fix forgotten argument name change
namespace cleanups
spec files glob pattern
cleanup
cleanup
cleanup
Fix the file-dependencies version to 0.1.5 to fix path issues on windows
monkeypatch Rubygems to silence ffi warning
list all plugin gems by default, new options
adding rake task to get faster feedback loop on windows
refactor bat scripts
Use an unreleased filedependencies gem and skip installing drupal
bump file-dependencies
use the released gem of file-dependencies
Add a set of scripts that simplify: * The necessary setup for the CI test to be executed. * The actuall CI test execution, having a different scripts for each different workflow. This makes the test execution simpler and standard, so changes in internal task don't need big changes in the CI config. It also documents and makes easy for developers to run the tests.
Fixes#2640
refactored test to not depend on STDOUT
create a .rspec file to pass some options used while the code is in the testing environment
add coveralls and simplecov as a development dependencies so we can run coverage tests
remove unnecessary .bundle directory from the artifact
In a release, the .bundle/config has a BUNDLE_PATH setting that can
interfere by misleading the plugin manager into not seeing the
vendor/bundle path.
Fixes#2647Fixes#2475
fix the logstash_gem_path
added some love from @jsvd comments
added an LS_ENV variable that if set to test it install the plugin dependencies when running the plugin:install-all rake task2
made the vendor:deps task has a more concret behaviour and proper naming
cleanup the all plugins install to only do the installation
added some code to fetch dependencies throw some gemcode within the gemfile as @electrical proposed
Revert "added some code to fetch dependencies throw some gemcode within the gemfile as @electrical proposed"
This reverts commit 20c3243485129c388d1da4ac930a479f017e0d99.
fix a path on the run all plugins test rake task
Fixes#2371
Plugins have moved to https://github.com/logstash-plugins/... into
individual repositories. One plugin per repo.
* Add plugin install task and add required plugins for testing
* Adding a plugin task to install plugins
* Added required plugins to a prep task for the test part.
Fixes#1984
