This does a few small things:
1. Prefer invoking rake tasks via ./gradlew vs rake. This prevents Java from segfaulting on docker for unknown reasons
2. Remove use of COPYING.csv which has already been removed
Fixes#9583
* `clean` actually cleans all dynamically created files, in particular it now properly cleans the generated Gemfile so
that changes to the Gemfile.template reflect in a rerun of `bundler`
* `rubyTests` and `test` are now one-off and will automatically bootstrap JRuby and Gems required by the tests if necessary
* Fixed Readme to document the now much simpler test targets
* All rake tasks remain unchanged and still work exactly as they did before
Fixes#8583
fix underfined env var check
check for defined JRUBYSRCDIR env var
check for errorlevel after launching rake tasks
cosmetics
call rake
keep original path
cosmetic
cosmetic
This commit includes:
* A base Dockerfile and script to push to a Docker repo
* A per-build Dockerfile (derived from the base)
* Updates to the test scripts to allow for more parallel builds
* Docker wrappers for the tests scripts
* Update for the integration test readme to manually run the tests
* Clean up the output of the Java tests
* Remove offline tag for tests (no longer needed that we don't use docker dependent services)
This commit does NOT include:
* Changes needed for the CI system to use Docker
Fixes#8223
this relies on jruby being installed locally with the same version as
the one contained in the .ruby-version file at the root of the project
The jruby must be installed in %JRUBYSRCDIR%\%JRUBYVERSION%
When running the `ci_acceptance` test we need to make sure we don't have
any Gemfile or *.lock laying around. Deleting a non existant file would
make the test fails.
Fixes#7515
The acceptance test in our CI infrastructure doesn't clear the workspace between run
this mean the lock of the Gemfile can be sticky from a previous run, before generating any package
we will clear them out to make sure we use the latest version of theses files from the template.
If we don't do this we will run into gem Conflict error.
Fixes#7435
This PR is based on the proposal from #6846, it does the following
- Change the format of the plugin help page
- Create an external file for each plugin type that will get included by the plugin
- Add a script to generate the PR on each plugin repo with the static doc
- Add a script `ci/ci_docs.sh` that will be used to generate the list of plugins that can be installed in logstash from all the available plugins from the community organization
This PR is the building block for change in https://github.com/elastic/docs-tools/issues/3Fixes#6941
The ci_integration.sh script uses double brackets ('[[') test
which is not portable. For instance, running it with dash you
get the error:
./ci/ci_integration.sh: 10: ./ci/ci_integration.sh: [[: not found
This change makes the script use bash, that supports this kind
of test.
Fixes#6939
Logstash's plugin manager will now follow proxy configuration from the environment.
If you configure `http_proxy` and `https_proxy`, the manager will now use theses information for all the ruby http
connection and will also pass that information down to maven.
Fixes: #6619, #6528Fixes#6825
This new command replace the old workflow of `pack`, `unpack` and the `install --local`, and wrap all the logic into one uniform way of installing plugins.
The work is based on the flow developed for installing an x-pack inside Logstash, when you call prepare-offline-pack, the specified plugins and their dependencies will be packaged in a zip.
And this zip can be installed with the same flow as the pack.
Definition:
Source Logstash: Where you run the prepare-offline-pack.
Target Logstash: Where you install the offline package.
PROS:
- If you install a .gem in the source logstash, the .gem and his dependencies will be bundled.
- The install flow doesn't need to have access to the internet.
- Nothing special need to be setup in the target logstash environment.
CONS:
- The is one minor drawback, the plugins need to have their JARS bundled with them for this flow to work, this is currently the case for all the official plugins.
- The source Logstash need to have access to the internet when you install plugins before packaging them.
Usage examples:
bin/logstash-plugin prepare-offline-pack logstash-input-beats
bin/logstash-plugin prepare-offline-pack logstash-filter-jdbc logstash-input-beats
bin/logstash-plugin prepare-offline-pack logstash-filter-*
bin/logstash-plugin prepare-offline-pack logstash-filter-* logstash-input-beats
How to install:
bin/logstash-plugin install file:///tmp/logstash-offline-plugins-XXXX.zip
Fixes#6404
**Motivation**: We have decided to rewrite the documentation generator to add more features
to our documentation and to fix some outstanding bugs.
This PR addresses the following problem:
- Hard to add new features to the code base.
- The code was tied with Logstash.
- No easy way to publish the doc for a single plugin
- The plugin author was not able to test their documentation changes
- The reported errors were hard to understand.
- No easy way to automate it.
- the minimal requirement on base classes.
Fixes#5720
* make sure downloaded logstashes use the new (arch independant files name)
* add specific privileged bootstrap script for debian 8 that is able to install java8 for this distro
* wrap the bootstrap scripts within the platform class, like this logic is in one specific place to select platform specific components
* add platform specific bootstrap file for sles-12 and also scripts to download latest logstash rpm version
* add custom bootstrap script for ubuntu-1404 to be sure it loads all ca certificates into the system
* make the necessary changes to compile with the new package system
* correct logstash path usage
* remove unnecessary dependancy to rake inside the acceptance test framework
* fix script execution under ubuntu environments, making sure it runs on bash
Fixes#5350
Also:
* Refactored the specs organization to make the resoning behind simplier,
introducing the idea of an artifact subject that it wraps the
interactions with the platform.
* Add methods to destroy, bootstrap and halt a list of machines either all of them or the ones listed under a given platform name.
* Introduced more clear namespacing in the rakefile.
* Updated the list of available platforms for acceptance testing, including latest ubuntu, oel, fedora, debian and suse versions
Fixes#5350
This change introduce a new command ci/ci_acceptance which is
responsable of building the packages, bootstraping the acceptance test
environment, launching the VMs and running the tests.
You can use the following command to target specific platform.
```
ci/ci_acceptance.sh all
ci/ci_acceptance.sh centos
ci/ci_acceptance.sh debian
```
This PR also add a new rake task to build all the artifacts in a single
run called rake artifact:all it make sure we only install the plugin
only once and make the build process faster.
Fixes#5350
This should fix scenarios where `rake bootstrap` fails due to bundler
failures and then some future command like `rake test:core` fails
because there's missing gems.
Fixes#5162
