The Logstash version of the Apache V2 license differs from the one
published at apache.org and in our other repos by a single newline.
This commit adds the newline.
Fixes#9707
This makes it consistent with the rest of the class.
I noticed we have some test failures due to @logger not existing. Probably an issue
from when the logger was javafied. This is causing test failures like: https://logstash-ci.elastic.co/job/elastic+logstash+6.x+multijob-unix-compatibility/os=oraclelinux/140/consoleFull
Will open a separate issue about the missing @logger var
```
An exception happened when converging configuration {:exception=>NoMethodError, :message=>"undefined method `error' for nil:NilClass", :backtrace=>["/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/logstash-core/lib/logstash/config/source_loader.rb:55:in `block in fetch'", "org/jruby/ext/thread/Mutex.java:148:in `synchronize'", "/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/logstash-core/lib/logstash/config/source_loader.rb:54:in `fetch'", "/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/logstash-core/lib/logstash/agent.rb:133:in `converge_state_and_update'", "/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/logstash-core/lib/logstash/agent.rb:84:in `execute'", "/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/logstash-core/spec/support/shared_contexts.rb:26:in `block in (root)'", "org/jruby/RubyBasicObject.java:1728:in `instance_exec'", "/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/vendor/bundle/jruby/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:350:in `run'", "/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/vendor/bundle/jruby/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:509:in `block in run_owned_hooks_for'", "org/jruby/RubyArray.java:1734:in `each'", "/var/lib/jenkins/workspace/elastic+logstash+6.x+multijob-unix-compatibility/os/oraclelinux/vendor/bundle/jruby/2.3.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:508:in `run_owned_hooks_for'", "/var/
```
Fixes#9691
This correctly CDs back and forth between dirs and tracks things
with variables to make it easier on the brain than relative paths like '..'
Fixes#9657
It just blocks and doesn't handle the concurrency situation. One can think of the network of connected pipelines as a DAG (We explicitly ask users not to create cycles in our docs). In fact there are two different correct answers to the pipeline shutdown and reload problem.
When reloading pipelines we should assume the user understands whatever changes they're making to the topology. If a downstream pipeline is to be removed, we can assume that's intentional. We don't lose any data from upstream pipelines since those block until that downstream pipeline is restored. To accomplish this none of the `PipelineBus` methods block by default.
When shutting down Logstash we must: 1.) not lose in-flight data, and 2.) not deadlock. The only way to do both is to shutdown the pipelines in order. All state changes happen simultaneously on all piping via multithreading. As a result we don't need to implement a Topological sort or other algorithm to order dependencies, we simply issue a shutdown to all pipelines, and have ones that are dependent block waiting for upstream pipelines.
This patch also correctly handles the case where multiple signals cause pipeline actions to be created simultaneously. If we see two concurrent creates or stops, one of those actions becomes a noop.
Currently the Logstash plugin API has lifecycle methods for starting and stopping, but not reloading. We need to call a different method when a `pipeline` input is stopped due to a pipeline reload vs an agent shutdown. Ideally, we would enrich our plugin API. In the interest of expedience here, however, I added a flag to the `PipelineBus` that changes the shutdown mode of `pipeline` inputs, to be either blocking or non-blocking. It is switched to blocking if a shutdown is triggered.
This also reverts b78d32dede in favor of a better more concurrent approach without mutexes
This is a forward port of https://github.com/elastic/logstash/pull/9650 to master / 6.x
Fixes#9656
Currently, any invocation of clone, including pipeline->pipeline comms and the clone filter
won't actually clone the string data, just the structure. We may be missing other value types too.
This is a start however.
Fixes#9652Fixes#9653
