github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-25 07:07:54 -04:00
Code Issues Projects Releases Packages Wiki Activity

Performance: Use RubyArray.hash for metric's fast lookup key

Browse source 
The existing implementation uses the RubyArray as the key for the fast lookup Map. Under the covers the Map implementation is comparing equal operators (many times per event), and JRuby Array equals operator is VERY expensive since it literally walks each value of array for each equality. Based on profiling via YourKit, the JRuby Array equals operator is a very hot method consuming upto 60% of sampled CPU cycles while under high load (and no other CPU dominators).

This change is to use the .hash value of the JRuby Array as the key of the fast lookup Map. This implementation still calls .hash for each and every call, which is also expensive, since it also walks the arrays to compute the hash. However, the equality check of the hash value is very fast, and net gain is significant. Upto a 15% increase of throughput.

Fixes #7772

Fixes #7798
This commit is contained in:
Jake Landis 2017-07-24 21:17:40 -05:00
parent e7651789cc
commit 9b4d7c51ae
1 changed files with 9 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

13
logstash-core/lib/logstash/instrument/metric_store.rb
View file

@ -51,10 +51,12 @@ module LogStash module Instrument
# BUT. If the value is not present in the `@fast_lookup` the value will be inserted and we assume that we don't # BUT. If the value is not present in the `@fast_lookup` the value will be inserted and we assume that we don't
# have it in the `@metric_store` for structured search so we add it there too. # have it in the `@metric_store` for structured search so we add it there too.
value = @fast_lookup.get(namespaces.dup << key) # array.hash as the key since it is faster then using the array itself, see #7772
fast_lookup_key = (namespaces.dup << key).hash
value = @fast_lookup.get(fast_lookup_key)
if value.nil? if value.nil?
value = block_given? ? yield(key) : default_value value = block_given? ? yield(key) : default_value
@fast_lookup.put(namespaces.dup << key, value) @fast_lookup.put(fast_lookup_key, value)
@structured_lookup_mutex.synchronize do @structured_lookup_mutex.synchronize do
# If we cannot find the value this mean we need to save it in the store. # If we cannot find the value this mean we need to save it in the store.
fetch_or_store_namespaces(namespaces).fetch_or_store(key, value) fetch_or_store_namespaces(namespaces).fetch_or_store(key, value)
@ -163,7 +165,7 @@ module LogStash module Instrument
end end
def has_metric?(*path) def has_metric?(*path)
@fast_lookup[path] @fast_lookup[path.hash]
end end
# Return all the individuals Metric, # Return all the individuals Metric,
@ -185,8 +187,9 @@ module LogStash module Instrument
def prune(path) def prune(path)
key_paths = key_paths(path).map(&:to_sym) key_paths = key_paths(path).map(&:to_sym)
@structured_lookup_mutex.synchronize do @structured_lookup_mutex.synchronize do
keys_to_delete = @fast_lookup.keys.select {|namespace| (key_paths - namespace[0..-2]).empty? } fetch_or_store_namespaces(key_paths).each do |key, v|
keys_to_delete.each {|k| @fast_lookup.delete(k) } @fast_lookup.delete((key_paths.dup << key).hash)
end
delete_from_map(@store, key_paths) delete_from_map(@store, key_paths)
end end
end end