github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-23 22:27:21 -04:00
Code Issues Projects Releases Packages Wiki Activity

add changes made to monitoring APIs for alpha5

Browse source 
Fixes #5701
This commit is contained in:
DeDe Morton 2016-07-27 12:27:43 -07:00
parent 3aa592f46d
commit 50578b49b4
1 changed files with 134 additions and 139 deletions
Download patch file Download diff file Expand all files Collapse all files

273
docs/static/monitoring-apis.asciidoc vendored
View file

@ -14,7 +14,7 @@ about Logstash:
You can use the root resource to retrieve general information about the Logstash instance, including
the hostname and version.
the host and version.
[source,js]
--------------------------------------------------
@ -26,7 +26,7 @@ Example response:
["source","js",subs="attributes"]
--------------------------------------------------
{
"hostname": "skywalker",
"host": "skywalker",
"version": "{logstash_version}",
"http_address": "127.0.0.1:9600"
}
@ -82,9 +82,7 @@ GET /_node/<types>
Where `<types>` is optional and specifies the types of node info you want to return.
You can limit the info that's returned by specifying one of the following types:
//TODO: For 5.0.0 Alpha4, this endpoint does not support specifying a comma-separated list like you can for the Elasticsearch cluster APIs. When this issue is fixed, we need to change the wording above to say: "You can limit the info that's returned by combining any of the following types"
You can limit the info that's returned by combining any of the following types: added[5.0.0-alpha5,Support for specifying a comma-separated list of types]
[horizontal]
`pipeline`::
@ -174,11 +172,11 @@ Example response:
--------------------------------------------------
[[plugins-api]]
=== Plugins API
=== Plugins Info API
experimental[]
The plugins API gets information about all Logstash plugins that are currently installed.
The plugins info API gets information about all Logstash plugins that are currently installed.
This API basically returns the output of running the `bin/logstash-plugin list --verbose` command.
[source,js]
@ -193,18 +191,19 @@ Example response:
["source","js",subs="attributes"]
--------------------------------------------------
{
"total": 91,
"plugins": [
{
"name": "logstash-codec-collectd",
"version": "3.0.1"
"version": "3.0.2"
},
{
"name": "logstash-codec-dots",
"version": "3.0.1"
"version": "3.0.2"
},
{
"name": "logstash-codec-edn",
"version": "3.0.1"
"version": "3.0.2"
},
.
.
@ -228,60 +227,67 @@ GET /_node/stats/<types>
Where `<types>` is optional and specifies the types of stats you want to return.
By default, all stats are returned. You can limit this by specifying one of the following types:
//TODO: Update the above description when support for specifying a comma-separated list of types is added back in. See above comment.
By default, all stats are returned. You can limit the info that's returned by combining any of the following types: added[5.0.0-alpha5,Support for specifying a comma-separated list of types]
[horizontal]
`events`::
Gets event information since startup.
`jvm`::
Gets JVM stats, including stats about threads. added[5.0.0-alpha3,Adds thread count]
`process`::
Gets process stats, including stats about file descriptors, memory consumption, and CPU usage. added[5.0.0-alpha3]
`mem`::
Gets memory usage stats. added[5.0.0-alpha4]
==== Event Stats
The following request returns a JSON document that shows the number of events
that were input, filtered, and output by Logstash since startup:
[source,js]
--------------------------------------------------
GET /_node/stats/events
--------------------------------------------------
Example response:
[source,js]
--------------------------------------------------
{
"events" : {
"in" : 59685,
"filtered" : 59685,
"out" : 59685
}
--------------------------------------------------
`pipeline`::
Gets pipeline stats. See <<pipeline-stats-api>> for examples.
==== JVM Stats
The following request returns a JSON document containing JVM stats:
The following request returns a JSON document containing JVM stats:
[source,js]
--------------------------------------------------
GET /_node/stats/jvm
--------------------------------------------------
Example response:
Example response: added[5.0.0-alpha5,JVM stats now include memory stats that were previously returned by `/_node/stats/mem`]
[source,js]
--------------------------------------------------
{
"jvm" : {
"threads" : {
"count" : 32,
"peak_count" : 34
"jvm": {
"threads": {
"count": 32,
"peak_count": 33
},
"mem": {
"heap_used_in_bytes": 290715552,
"heap_used_percent": 14,
"heap_committed_in_bytes": 519045120,
"heap_max_in_bytes": 2075918336,
"non_heap_used_in_bytes": 181911616,
"non_heap_committed_in_bytes": 193249280,
"pools": {
"survivor": {
"peak_used_in_bytes": 8912896,
"used_in_bytes": 9358024,
"peak_max_in_bytes": 35782656,
"max_in_bytes": 71565312,
"committed_in_bytes": 17825792
},
"old": {
"peak_used_in_bytes": 106400040,
"used_in_bytes": 164247880,
"peak_max_in_bytes": 715849728,
"max_in_bytes": 1431699456,
"committed_in_bytes": 357957632
},
"young": {
"peak_used_in_bytes": 71630848,
"used_in_bytes": 117109648,
"peak_max_in_bytes": 286326784,
"max_in_bytes": 572653568,
"committed_in_bytes": 143261696
}
}
}
}
--------------------------------------------------
@ -300,63 +306,16 @@ Example response:
[source,js]
--------------------------------------------------
{
"process" : {
"peak_open_file_descriptors" : 64,
"max_file_descriptors" : 10240,
"open_file_descriptors" : 64,
"mem" : {
"total_virtual_in_bytes" : 5278068736
"process": {
"open_file_descriptors": 60,
"peak_open_file_descriptors": 65,
"max_file_descriptors": 10240,
"mem": {
"total_virtual_in_bytes": 5364461568
},
"cpu" : {
"total_in_millis" : 103290097000,
"percent" : 0
}
}
--------------------------------------------------
==== Mem Stats
The following request returns a JSON document containing memory stats:
[source,js]
--------------------------------------------------
GET /_node/stats/mem
--------------------------------------------------
Example response:
[source,js]
--------------------------------------------------
{
"mem": {
"heap_used_in_bytes": 262641008,
"heap_used_percent": 12,
"heap_committed_in_bytes": 519045120,
"heap_max_in_bytes": 2075918336,
"non_heap_used_in_bytes": 184060512,
"non_heap_committed_in_bytes": 195870720,
"pools": {
"survivor": {
"peak_used_in_bytes": 8912896,
"used_in_bytes": 9280256,
"peak_max_in_bytes": 35782656,
"max_in_bytes": 71565312,
"committed_in_bytes": 17825792
},
"old": {
"peak_used_in_bytes": 112649208,
"used_in_bytes": 170055984,
"peak_max_in_bytes": 715849728,
"max_in_bytes": 1431699456,
"committed_in_bytes": 357957632
},
"young": {
"peak_used_in_bytes": 71630848,
"used_in_bytes": 83304768,
"peak_max_in_bytes": 286326784,
"max_in_bytes": 572653568,
"committed_in_bytes": 143261696
}
"cpu": {
"total_in_millis": 101294404000,
"percent": 0
}
}
--------------------------------------------------
@ -393,22 +352,13 @@ Example response:
"filtered": 100,
"out": 100
},
"pipeline": {
"plugins": {
"inputs": [],
"filters": [
{
"id": "geoip_6aea6f9a-c100-4c39-ab00-e34d54d6e573",
"id": "grok_c4900bd0-29ef-44a5-b44c-f6ffef3ddf8c",
"events": {
"duration_in_millis": 145,
"in": 100,
"out": 100
},
"name": "geoip"
},
{
"id": "grok_4000f2bf-4baf-4258-b72f-56e0b6f15fc1",
"events": {
"duration_in_millis": 48,
"duration_in_millis": 43,
"in": 100,
"out": 100
},
@ -417,13 +367,22 @@ Example response:
"message": 1
},
"name": "grok"
},
{
"id": "geoip_130740d3-cad0-4ae5-96dd-7ef8f0eb1adb",
"events": {
"duration_in_millis": 116,
"in": 100,
"out": 100
},
"name": "geoip"
}
],
"outputs": [
{
"id": "elasticsearch_db05752c-2e30-4f77-a3d7-167e915d470f",
"id": "elasticsearch_2f22c8b5-3d63-426e-a4cf-08e48af29538",
"events": {
"duration_in_millis": 232,
"duration_in_millis": 533,
"in": 100,
"out": 100
},
@ -460,16 +419,13 @@ Example response:
[source,js]
--------------------------------------------------
{
"host": "skywalker",
"version": "5.0.0-alpha4",
"http_address": "127.0.0.1:9600",
"hot_threads": {
"time": "2016-06-24T18:19:48-07:00",
"time": "2016-07-26T18:39:08-07:00",
"busiest_threads": 3,
"threads": [
{
"name": "LogStash::Runner",
"percent_of_cpu_time": 0.09,
"percent_of_cpu_time": 0.17,
"state": "timed_waiting",
"traces": [
"java.lang.Object.wait(Native Method)",
@ -486,9 +442,9 @@ Example response:
},
{
"name": "Ruby-0-Thread-17",
"percent_of_cpu_time": 0.05,
"percent_of_cpu_time": 0.11,
"state": "timed_waiting",
"path": "/Users/dedemorton/BuildTesting/5.0.0-alpha4/logstash-5.0.0-alpha4/logstash-core/lib/logstash/pipeline.rb:467",
"path": "/Users/username/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:471",
"traces": [
"java.lang.Object.wait(Native Method)",
"org.jruby.RubyThread.sleep(RubyThread.java:1002)",
@ -497,10 +453,27 @@ Example response:
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)",
"org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)",
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)",
"rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/dedemorton/BuildTesting/5.0.0-alpha4/logstash-5.0.0-alpha4/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)",
"rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/dedemorton/BuildTesting/5.0.0-alpha4/logstash-5.0.0-alpha4/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)",
"rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)",
"rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)",
"org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)"
]
},
{
"name": "[main]-pipeline-manager",
"percent_of_cpu_time": 0.04,
"state": "timed_waiting",
"traces": [
"java.lang.Object.wait(Native Method)",
"java.lang.Thread.join(Thread.java:1253)",
"org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
"org.jruby.RubyThread.join(RubyThread.java:697)",
"org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
"org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)",
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)",
"org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)"
]
}
]
}
@ -525,24 +498,46 @@ Example of a human-readable response:
[source,js]
--------------------------------------------------
::: {Ringo Kid}{Gv3UrzR3SqmPQIgfG4qJMA}{127.0.0.1}{127.0.0.1:9300}
Hot threads at 2016-01-13T16:55:49.988Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
::: {}
Hot threads at 2016-07-26T18:46:18-07:00, busiestThreads=3:
================================================================================
0.15 % of of cpu usage by timed_waiting thread named 'LogStash::Runner'
java.lang.Object.wait(Native Method)
java.lang.Thread.join(Thread.java:1253)
org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
org.jruby.RubyThread.join(RubyThread.java:697)
org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
--------------------------------------------------------------------------------
0.11 % of of cpu usage by timed_waiting thread named 'Ruby-0-Thread-17'
/Users/username/BuildTesting/logstash-5.0.0logstash-core/lib/logstash/pipeline.rb:471
java.lang.Object.wait(Native Method)
org.jruby.RubyThread.sleep(RubyThread.java:1002)
org.jruby.RubyKernel.sleep(RubyKernel.java:803)
org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)
rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)
org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)
--------------------------------------------------------------------------------
0.04 % of of cpu usage by timed_waiting thread named '[main]-pipeline-manager'
java.lang.Object.wait(Native Method)
java.lang.Thread.join(Thread.java:1253)
org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
org.jruby.RubyThread.join(RubyThread.java:697)
org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)
0.0% (216micros out of 500ms) cpu usage by thread 'elasticsearch[Ringo Kid][transport_client_timer][T#1]{Hashed wheel timer #1}'
10/10 snapshots sharing following 5 elements
java.lang.Thread.sleep(Native Method)
org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:445)
org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:364)
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
java.lang.Thread.run(Thread.java:745)
0.0% (216micros out of 500ms) cpu usage by thread 'elasticsearch[Ringo Kid][transport_client_timer][T#1]{Hashed wheel timer #1}'
10/10 snapshots sharing following 5 elements
java.lang.Thread.sleep(Native Method)
org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:445)
org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:364)
org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
java.lang.Thread.run(Thread.java:745)
--------------------------------------------------
See <<monitoring-common-options, Common Options>> for a list of options that can be applied to all