add test for subclasses of codec, filter, input and output
Register codec clone, use String clone not dup so frozen strings stay so
Fixes#4444Fixes#4473
We were getting some bizzare
errors (https://github.com/elastic/logstash/pull/4391#issuecomment-169477728)
on the file output with the new OutputDelegator changes. By switching
away from method(:mymethod) to blocks with define_singleton_method they
went away.
This bug is only present with Jruby 1.7.x (tested with 1.7.22 and
1.7.23). Jruby 9.x is fine.
Fixes#4391
When you run multiples pipeline and the config code get evaluated, on
every evaluation the class cached is clear, everytime you were calling a
`func` method you had the latest evaluated code. The `filter_func` and
the `output_func` need to be unique for every instance of the pipeline,
this PR replace the `def` with a `define_single_method` call ensuring
the uniqueness of the code on every instance.
This PR is based on #4254Fixes#4298
slightly adjust the metaphor for the class that is responsible for
monitoring a pipeline's shutdown process, report on its status and
intervene if it's stalled
This is predicated on the fact that with the ng_pipeline it is expected that workers
will spend a significant amount of time in iowait due to outputs like the Elasticsearch
output. In benchmarks based on real-world Apache log files the best performance came out
of scenarios where pipeline_workers > num_cpu_cores. Setting this to default to the # of
cores is a defensive decision that should handle cases where users have particularly IO
heavy inputs.
For most users we should recommend setting the number of workers to be as high as possible
until performance decreases.
Previous benchmark information:
https://github.com/elastic/logstash/pull/4340Fixes#4414
Expose the Java's `#offer` method of the `SynchronousQueue` class,
This method allow the input developper to correctly apply back pressure
to the network clients.
If you use `#push` it will block forever until the pipeline free some
space, this behavior is sufficient if the pipeline outputs are healthy.
But if the output stale the backpressure will be applied up to the input
(producers) when reading a file this scenario is fine since we will just
stop reading the file until the thread unblock again.
In the context of network clients, the story is a bit different the
clients will timeout and try to reconnect to restransmit their payload
creating multiple new connection thread block on the queue. In some case
this will lead into a OOM issues.
This PR is the first step to communicate that the queue is under
pressure.
Fixes#4408
This PR allow the user to set the name of the logstash instance with the
`-n` or `--name` option, if no name is provided the agent will fallback
to the current hostname of the machine.
This PR also add the `node_uuid` to the agent which is unique between
each new instance of the agent, it will lazy generate a UUID v4 on
invocation.
Fixes: #4353Fixes#4356
* move cli argument handling from agent to runner
* add a short-help message
* add interactive shell option
* log fatal messages to terminal when logging to file
* change docs:generate task to use bundle exec
Fixes#3872
* start logstash with --allow-unsafe-shutdown to force_exit on stalled shutdown
* by default --allow-unsafe-shutdown is disabled
* stall detection kicks in when SIGTERM/SIGINT is received
* check if inflight event count isn't going down and if there are blocked/blocking plugin threads
updates requested by code review
changes requested by colin: make workers override from -w arg
do not set workers unless user actually specified it via cmdline
fix defaults printing
add describe block to improve test output readability
Closes#4130
fixed timezone issue
extracted logstash-core and reorganized specs
extracted logstash-core-event
extract java Event into logstash-core-event-java in a proper gem
remove obsolete jruby_event bootstrapping
fix require path
add java code bootstrap
use logstash-core-event/logstash-core-event.rb
remove obsolete files
basic instructions
LogStash::Json need to be initialized from event
update jruby and gradle versions
update compile:logstash-core-event-java rake task
WIP tasks refactor
fix gem.files
skip test if class is not defined
fix gem related tasks for new structure
add gem spec dirs in core tests
bootstrap java implementation when requiring timestamp
new Cloner class and Event clone impl
fix array fields assignments, see #4140
don't rely on json implementation ordering
fix skipped last iterpolation char
remove implementation specific unnecessary check
also require ruby classes
define error class in ruby
raise exception on invalid format
remove implementation specific tests and extract and put logger related test in pending
missing bits for having all core timestamp specs pass
run all core specs
remove leftover
comment regex
missing encoding header
revert to logstash-core-event by default
finished proper gemification
useless require
dynamically pick specs depending on logstash-core-event-* implementation
logstash root package version
missing file for proper gemification
do not build java event by default
always check for root logstash lib dir
fix concurrent-ruby version confict
fix rebase conflict
re-enable specs
user vars instead of constants
move non core code in bootstrap
document version files
move version file
remove useless code
use version in logstash-core
fix gem files list
put back concurrent-ruby version constrain as in master
add dependency on logstash-core-event
remove dependency on logstash-core to avoid circular dependency
fix rebase conflict
remove circular dependency
fix specs
update README
