Remove usage of plugin Gradle-License-Report to generate CSV report of used dependencies.
This commit replaces the removed plugin with the custom `ListProjectDependencies` task that inspects Gradle's project dependencies and produces a CSV file containing the list.
Starting with version 7.10.0 the name of LS packages changed, adding os and CPU architecture in the name. This change broke the downloading of those from the benchmarking tool. This commit fixes it, composing correctly the name, based on the version it has to download.
* Add logstash-integration-elastic_enterprise_search to plugins-metadata.json
* Remove old elastic_app_search plugin and set integration as default
* Add license information for workplace search gem
Since the introduction of this block:
```
"pipeline" : {
"workers" : 16,
"batch_size" : 125,
"batch_delay" : 50
},
```
to the node stats API, the benchmarking tool has been broken. This commit fixes the
tool, and updates the payload in the tests to reflect the current payload.
This commit updates the license information for the license dependency report.
Specifically, this adds a notice for `racc`, a different version of which is now
pulled in by nokogiri from the version included with jruby.
* Add optional sourceURL to license report CSV
This commit adds an extra optional column 'sourceURL' to the license report. This
column contains a pointer to the source code, which is optional for most dependencies,
but a requirement for some, such as the Red Hat Universal Base Image.
This commit also populates the 'copyright' field, which previously was an used
column in the CSV definition
Relates #12297
Conversion to seconds of values under 1_000_000_000 nanoseconds translates to value 0, and this led to NaN when used as denominator in a division.
A value of 996_920_400 nanoseconds once converted to seconds is not rounded to 1 second by to 0, this manifest on Windows OS
* Update gradle version to 6.3
Gradle versions prior to 6.3 cannot run under JDK14.
This commit upgrades the version of Gradle to 6.3, and removes all deprecation warnings that can currently be removed.
Changes include:
* Increase gradle memory to 2g
* Increase gradle memory in the license check job to 2g
* Replace use of `testCompile`
* Replace `runtime` with `runtimeOnly`
* Remove`compile` depedencies from gradle files
* Replace deprecated archive methods
* Fix dependencies report build
* Make jruby dependencies 'api', fix archiveVersion
* Set `duplicatesStrategy` for all tasks of type Copy
* Use `configureEach` for global 'withType' calls
** Use the recommended Tasks API calls
(https://blog.gradle.org/preview-avoiding-task-configuration-time)
* Run `./gradlew wrapper` earlier to improve caching
* Use copy with chown for resources that need to be run during `./gradlew wrapper`
YAML.parse returns Psych nodes that then need to be converted to plain ruby objects.
Calling YAML.safe_load outputs basic ruby objects already and also increases security as it greatly restricts the classes it deserializes.
Fixes#11208
By removing the default plugins from the Gemfile.template
the current task that modified the template was not working correctly.
This commit either replaces the dependency entry if it exists or
otherwise creates it.
* bump jruby to 9.2
* don't rely on logstash-base docker image
* work around webmock ruby 2.5 support
* ensure data folder exists in docker
* change fixnum and bignum to integer
* FileUtils.rmdir to rm_rf
this is because from 2.3 to 2.5 FileUtils.rmdir will throw an exception
if the directory isn't empty. On 2.3 the operation will just not delete
the directory silently.
* bump jruby to 9.2.5.0 and fix test
* make rake default task since prepare pack needs it
* Resolve compiler warnings (#10247)
There are 3 types of compiler warnings that are either resolved or suppressed:
1. Rawtypes: In JRuby 9.2, `RubyArray` is a generic, so references throughout
our codebase to the now "raw" type trigger warnings. In most cases we cannot
actually resolve the issue, since the JRuby-provided methods for creating
`RubyArray`s still return the raw type, so these have been suppressed.
2. Deprecations:
- `RubyString#intern19()` -> `RubyString#intern()`
- `RubyString#downcase19(ThreadContext)` -> `RubyString#downcase(ThreadContext)`
- `NativeException`: remove import & reference directly; suppress usage
warnings
- `RaiseException()`: migrate to equivalent non-deprecated methods wherever
possible; in some cases where we are using this in conjunction with the
also-deprecated `NativeException` to preserve java stacktraces, there
seems to be no non-deprecated path forward, so these cases have been
suppressed.
3. Redundant Casts
- Resolved
* JRuby 9.2 bundler shenanigans (#10266)
* Revert "Revert "remove forced dependency on old bundler (#9395)""
This reverts commit bef984143d.
* plugin management: update internal bundler to 1.17.x APIs
* deps: update dev dependency webmock to version compatible with JRuby 9.2
* spec: update Pack fixture to include manticore version that doesn't conflict
* build: update gradle to version that has Java 11 support
* java11: resolve or suppress deprecation warnings
* Remove superfluous flag opting into ParNew GC implementation
When opting into CMS garbage collector with `XX:+UseConcMarkSweepGC`, the
young generation collector ParNew has been the default since Java 8, making
the `XX:+UseParNew` flag redundant; the flag was removed in Java 9, and
should no longer be specified to work with modern Javas.
https://bugs.openjdk.java.net/browse/JDK-8006478https://openjdk.java.net/jeps/214
* spec: set thread name to example description for easier debugging
* spec: prevent errors in testing specs by checking against skip list before using
* no-op: remove use of `HashMap#computeIfAbsent` on single-threaded code
> This method will, on a best-effort basis, throw a `ConcurrentModificationException`
> if it is detected that the mapping function modifies this map during computation.
>
> -- https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/HashMap.html#computeIfAbsent(K,java.util.function.Function)
* qa: by default, run integration against Elastic Stack 6.5.x
To support development on Logstash on top of Java 11, default to testing
against an Elastic Stack that is capable of running on Java 11.
* qa: ignore deprecation warnings when comparing offline pack output
* qa: add Java 9+ support to ChildProcess dev dependency
this can safely be removed when the childprocess gem supports Java9+
https://github.com/enkessler/childprocess/pull/141
* qa: allow connections to localhost in webmock
* bump jrjackson version
* fix filebeat integration tests
* spec: ensure license compliance spec runs first
The license compliance spec that validates the licenses of bundled
plugins appears to not be compatible with the hooks that we inject
into bundler for plugin management, and will fail in obscure ways
when run after those hooks have been added. Since those hooks are
not necessary for validating licenses, the easiest solution was to
ensure that those specs run first, before the VM has been poluted.
Since the gradle/junit/rspec bridge that is currently in place
runs all specs in the same JVM, we also need to make sure that the
rspec "world" is reset before a run, to ensure that it doesn't
retain spec definitions from previous runs.
Also updates the rake invocation, although I'm not sure it is used
any more.
