github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-24 06:37:19 -04:00
Code Issues Projects Releases Packages Wiki Activity

Up default # of pipeline workers to 100% of cores

Browse source 
This is predicated on the fact that with the ng_pipeline it is expected that workers
will spend a significant amount of time in iowait due to outputs like the Elasticsearch
output. In benchmarks based on real-world Apache log files the best performance came out
of scenarios where pipeline_workers > num_cpu_cores. Setting this to default to the # of
cores is a defensive decision that should handle cases where users have particularly IO
heavy inputs.

For most users we should recommend setting the number of workers to be as high as possible
until performance decreases.

Previous benchmark information:
https://github.com/elastic/logstash/pull/4340

Fixes #4414
This commit is contained in:
Andrew Cholakian 2016-01-04 13:59:05 -06:00
parent bed04ccfbf
commit bb24631e4f
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
logstash-core/lib/logstash/pipeline.rb
View file

@ -20,7 +20,7 @@ module LogStash; class Pipeline
attr_reader :inputs, :filters, :outputs, :worker_threads, :events_consumed, :events_filtered, :reporter, :pipeline_id, :logger
DEFAULT_SETTINGS = {
:default_pipeline_workers => LogStash::Config::CpuCoreStrategy.fifty_percent,
:default_pipeline_workers => LogStash::Config::CpuCoreStrategy.maximum,
:pipeline_batch_size => 125,
:pipeline_batch_delay => 5, # in milliseconds
:flush_interval => 5, # in seconds