With the move to JRuby9k which use the 2.3 syntax, this make the
following constant deprecated
```
config_19.setCompatVersion(CompatVersion.RUBY1_9);
```
Fixes#7446
Make sure when you set a logstash version that we are not wiping
the content of the versions file but instead merge it so we preserve
jruby information.
Fixes#7434
We have multiple places to declare the jruby version, this commit
centralize everything into the `version.yml` file. This is now used by
the ruby part and the gradle build file to correctly fetch the jruby
library.
Fixes#7434
This commit further improves hashing performance, using SourceWithMetadata
objects where present to determine Vertex IDs and the hash of the source
file. If those objects are not present LIR will revert to graph hashing,
which is slower, but always works. This is still useful for things like
testing, so it makes sense to leave that code in.
In the future it might be nice to extract the hashing code using the
strategy pattern to clean things up.
This commit also improves the Hashable interface, moving hashSource into
its own interface, letting classes that do their own hashing simply
implement uniqueHash instead of requiring them to also implement
hashSource.
A requirement of these changes was also that SourceWithMetadata no
longer accept null or empty arguments. This patch now enforces that
requirement. This also is nicer from an API standpoint, as you now know
that if a SourceWithMetadata object exists, it will actually have all
its fields in use.
Fixes#7408
Fixes#7379
Prior to this patch large configuration files would take a long time to
load. This patch improves the graph hashing algorithm in the following
ways:
1. Caches expensive to compute paths
2. Use parallel computation where possible to speed calculations
3. Removes unnecessary validation on Graph#chainVertices
4. Validates the entire graph in PipelineIR which allows the cache to be
used for both validation and hashing
5. Validation now only checks for duplicate IDs, not duplicate hashes.
If the user supplies their own IDs this check will be sped up.
6. Hashing only consists of calculating the unique hash for the leaf
nodes. Since hashing is sensitive to the full ancestry of a given node
this is sufficient vs. previously where each vertex's hash was
combined. We will still calculate the hash for each vertex if no ID is
specified however.
Fixes#7408
`RubyHash#visitAll` and `RubyArray#getList` are now deprecated.
- RubyArray now implements List, so we don't need to convert it.
- We need to be a bit more explicit with the visitAll and pass the ruby
context.
Fixes: #7427Fixes#7425
Work done by @guyboertje and @ph
Since JRuby 1.7.25 is now EOL we are migrating Logstash to use JRuby 9k and JDK8 only,
Not much needed updating to make this work, its was mostly a drop in replacement from the previous version.
The major point was the change in the implementation of Time in JRuby, JRuby now use `java.time`
instead of joda time, this allow JRuby to have nanoseconds precision on time object.
* Handle ingest on_failure processor
Ingest `on_failure` processor allows to define a new sub-piplene when a failure
happens in the current processor. Logstash does not have an on_failure but this can
be emulated using the tag and conditional.
Also added a set processor converter which is typically used in `on_failure`.
* generalize on_failure method
* Add comments
* Fix indentation issue
Add an initial set of metrics for the dead letter queue.
Metrics are supplied under the pipeline in the following format:
"dead_letter_queue": {
"queue_size_in_bytes": ...,
}
Metrics are populated via a PeriodicPoller
Also fixed up calculation of currentQueueSize to take account
of version headers, which was previously being skipped.
Additionally, whether the dlq is enabled, and if so, the path
of the dlq is supplied under the pipelines API endpoint
Resolves#7287Fixes#7338
The ability to change the log level as described at https://www.elastic.co/guide/en/logstash/current/logging.html#_logging_apis appears to no longer work. The root cause of this issue is that log4j2 context we initialize is different from the log4j2 context that we are setting via the API. The fix here is to mirror the functionality of org.apache.logging.log4j.core.config.Configurator.setLevel Java method, but instead use the logging_context initialized in our code via JRuby.
Fixes#7277Fixes#7321
This PR is based on the proposal from #6846, it does the following
- Change the format of the plugin help page
- Create an external file for each plugin type that will get included by the plugin
- Add a script to generate the PR on each plugin repo with the static doc
- Add a script `ci/ci_docs.sh` that will be used to generate the list of plugins that can be installed in logstash from all the available plugins from the community organization
This PR is the building block for change in https://github.com/elastic/docs-tools/issues/3Fixes#6941
The old version (1.0.0) had a broken version of Concurrent::Timer that
did not work on all machines for reasons that are still unclear, but may
be related to the Java version
Fixes#7368
* Add Append processor converto
* Fix bug in convert processor
* Updated comment
* add JSON processor convertor
* use the right name for the variable
* Fix the double space in convert
