mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 14:47:19 -04:00
562 lines
18 KiB
Text
562 lines
18 KiB
Text
[[monitoring]]
|
|
== Monitoring APIs
|
|
|
|
experimental[]
|
|
|
|
Logstash provides the following monitoring APIs to retrieve runtime metrics
|
|
about Logstash:
|
|
|
|
* <<node-info-api>>
|
|
* <<plugins-api>>
|
|
* <<node-stats-api>>
|
|
* <<hot-threads-api>>
|
|
|
|
|
|
You can use the root resource to retrieve general information about the Logstash instance, including
|
|
the host and version.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
["source","js",subs="attributes"]
|
|
--------------------------------------------------
|
|
{
|
|
"host": "skywalker",
|
|
"version": "{logstash_version}",
|
|
"http_address": "127.0.0.1:9600"
|
|
}
|
|
--------------------------------------------------
|
|
|
|
NOTE: By default, the monitoring API attempts to bind to `tcp:9600`. If this port is already in use by another Logstash
|
|
instance, you need to launch Logstash with the `--http.port` flag specified to bind to a different port. See
|
|
<<command-line-flags>> for more information.
|
|
|
|
[float]
|
|
[[monitoring-common-options]]
|
|
=== Common Options
|
|
|
|
The following options can be applied to all of the Logstash monitoring APIs.
|
|
|
|
[float]
|
|
==== Pretty Results
|
|
|
|
When appending `?pretty=true` to any request made, the JSON returned
|
|
will be pretty formatted (use it for debugging only!).
|
|
|
|
[float]
|
|
==== Human-Readable Output
|
|
|
|
NOTE: For Logstash {logstash_version}, the `human` option is supported for the <<hot-threads-api>>
|
|
only. When you specify `human=true`, the results are returned in plain text instead of
|
|
JSON format. The default is false.
|
|
|
|
Statistics are returned in a format suitable for humans
|
|
(eg `"exists_time": "1h"` or `"size": "1kb"`) and for computers
|
|
(eg `"exists_time_in_millis": 3600000` or `"size_in_bytes": 1024`).
|
|
The human-readable values can be turned off by adding `?human=false`
|
|
to the query string. This makes sense when the stats results are
|
|
being consumed by a monitoring tool, rather than intended for human
|
|
consumption. The default for the `human` flag is
|
|
`false`.
|
|
|
|
[[node-info-api]]
|
|
=== Node Info API
|
|
|
|
experimental[]
|
|
|
|
The node info API retrieves information about the node.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/<types>
|
|
--------------------------------------------------
|
|
|
|
Where `<types>` is optional and specifies the types of node info you want to return.
|
|
|
|
You can limit the info that's returned by combining any of the following types in a comma-separated list:
|
|
|
|
[horizontal]
|
|
`pipeline`::
|
|
Gets pipeline-specific information and settings.
|
|
`os`::
|
|
Gets node-level info about the OS.
|
|
`jvm`::
|
|
Gets node-level JVM info, including info about threads.
|
|
|
|
==== Pipeline Info
|
|
|
|
The following request returns a JSON document that shows pipeline info, such as the number of workers,
|
|
batch size, and batch delay:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/pipeline
|
|
--------------------------------------------------
|
|
|
|
If you want to view additional information about the pipeline, such as stats for each configured input, filter,
|
|
or output stage, see the <<pipeline-stats>> section under the <<node-stats-api>>.
|
|
|
|
Example response:
|
|
|
|
["source","js",subs="attributes"]
|
|
--------------------------------------------------
|
|
{
|
|
"pipeline": {
|
|
"workers": 8,
|
|
"batch_size": 125,
|
|
"batch_delay": 5,
|
|
"config_reload_automatic": true,
|
|
"config_reload_interval": 3
|
|
|
|
}
|
|
--------------------------------------------------
|
|
|
|
==== OS Info
|
|
|
|
The following request returns a JSON document that shows the OS name, architecture, version, and
|
|
available processors:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/os
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"os": {
|
|
"name": "Mac OS X",
|
|
"arch": "x86_64",
|
|
"version": "10.11.2",
|
|
"available_processors": 8
|
|
}
|
|
--------------------------------------------------
|
|
|
|
==== JVM Info
|
|
|
|
The following request returns a JSON document that shows node-level JVM stats, such as the JVM process id, version,
|
|
VM info, memory usage, and info about garbage collectors:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/jvm
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"jvm": {
|
|
"pid": 8187,
|
|
"version": "1.8.0_65",
|
|
"vm_name": "Java HotSpot(TM) 64-Bit Server VM",
|
|
"vm_version": "1.8.0_65",
|
|
"vm_vendor": "Oracle Corporation",
|
|
"start_time_in_millis": 1474305161631,
|
|
"mem": {
|
|
"heap_init_in_bytes": 268435456,
|
|
"heap_max_in_bytes": 1037959168,
|
|
"non_heap_init_in_bytes": 2555904,
|
|
"non_heap_max_in_bytes": 0
|
|
},
|
|
"gc_collectors": [
|
|
"ParNew",
|
|
"ConcurrentMarkSweep"
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
|
|
[[plugins-api]]
|
|
=== Plugins Info API
|
|
|
|
experimental[]
|
|
|
|
The plugins info API gets information about all Logstash plugins that are currently installed.
|
|
This API basically returns the output of running the `bin/logstash-plugin list --verbose` command.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/plugins
|
|
--------------------------------------------------
|
|
|
|
The output is a JSON document.
|
|
|
|
Example response:
|
|
|
|
["source","js",subs="attributes"]
|
|
--------------------------------------------------
|
|
{
|
|
"total": 91,
|
|
"plugins": [
|
|
{
|
|
"name": "logstash-codec-collectd",
|
|
"version": "3.0.2"
|
|
},
|
|
{
|
|
"name": "logstash-codec-dots",
|
|
"version": "3.0.2"
|
|
},
|
|
{
|
|
"name": "logstash-codec-edn",
|
|
"version": "3.0.2"
|
|
},
|
|
.
|
|
.
|
|
.
|
|
]
|
|
--------------------------------------------------
|
|
|
|
[[node-stats-api]]
|
|
=== Node Stats API
|
|
|
|
experimental[]
|
|
|
|
The node stats API retrieves runtime stats about Logstash.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/stats/<types>
|
|
--------------------------------------------------
|
|
|
|
Where `<types>` is optional and specifies the types of stats you want to return.
|
|
|
|
By default, all stats are returned. You can limit the info that's returned by combining any of the following types in a comma-separated list:
|
|
|
|
[horizontal]
|
|
`jvm`::
|
|
Gets JVM stats, including stats about threads, memory usage, and garbage collectors.
|
|
`process`::
|
|
Gets process stats, including stats about file descriptors, memory consumption, and CPU usage.
|
|
`mem`::
|
|
Gets memory usage stats.
|
|
`pipeline`::
|
|
Gets runtime stats about the Logstash pipeline.
|
|
|
|
==== JVM Stats
|
|
|
|
The following request returns a JSON document containing JVM stats:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/stats/jvm
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"jvm": {
|
|
"threads": {
|
|
"count": 33,
|
|
"peak_count": 34
|
|
},
|
|
"mem": {
|
|
"heap_used_in_bytes": 276974824,
|
|
"heap_used_percent": 13,
|
|
"heap_committed_in_bytes": 519045120,
|
|
"heap_max_in_bytes": 2075918336,
|
|
"non_heap_used_in_bytes": 182122272,
|
|
"non_heap_committed_in_bytes": 193372160,
|
|
"pools": {
|
|
"survivor": {
|
|
"peak_used_in_bytes": 8912896,
|
|
"used_in_bytes": 11182152,
|
|
"peak_max_in_bytes": 35782656,
|
|
"max_in_bytes": 71565312,
|
|
"committed_in_bytes": 17825792
|
|
},
|
|
"old": {
|
|
"peak_used_in_bytes": 111736080,
|
|
"used_in_bytes": 171282544,
|
|
"peak_max_in_bytes": 715849728,
|
|
"max_in_bytes": 1431699456,
|
|
"committed_in_bytes": 357957632
|
|
},
|
|
"young": {
|
|
"peak_used_in_bytes": 71630848,
|
|
"used_in_bytes": 94510128,
|
|
"peak_max_in_bytes": 286326784,
|
|
"max_in_bytes": 572653568,
|
|
"committed_in_bytes": 143261696
|
|
}
|
|
}
|
|
},
|
|
"gc": {
|
|
"collectors": {
|
|
"old": {
|
|
"collection_time_in_millis": 48,
|
|
"collection_count": 2
|
|
},
|
|
"young": {
|
|
"collection_time_in_millis": 316,
|
|
"collection_count": 23
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
==== Process Stats
|
|
|
|
The following request returns a JSON document containing process stats:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/stats/process
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"process": {
|
|
"open_file_descriptors": 60,
|
|
"peak_open_file_descriptors": 65,
|
|
"max_file_descriptors": 10240,
|
|
"mem": {
|
|
"total_virtual_in_bytes": 5364461568
|
|
},
|
|
"cpu": {
|
|
"total_in_millis": 101294404000,
|
|
"percent": 0
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
[[pipeline-stats]]
|
|
==== Pipeline Stats
|
|
|
|
The following request returns a JSON document containing pipeline stats,
|
|
including:
|
|
|
|
* the number of events that were input, filtered, or output by the pipeline
|
|
* stats for each configured filter or output stage
|
|
* info about config reload successes and failures
|
|
(when <<reloading-config,config reload>> is enabled)
|
|
|
|
NOTE: Detailed pipeline stats for input plugins are not currently available, but
|
|
will be available in a future release. For now, the node stats API returns an
|
|
empty set array for inputs (`"inputs": []`).
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/stats/pipeline
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"pipeline": {
|
|
"events": {
|
|
"duration_in_millis": 7863504,
|
|
"in": 100,
|
|
"filtered": 100,
|
|
"out": 100
|
|
},
|
|
"plugins": {
|
|
"inputs": [],
|
|
"filters": [
|
|
{
|
|
"id": "grok_20e5cb7f7c9e712ef9750edf94aefb465e3e361b-2",
|
|
"events": {
|
|
"duration_in_millis": 48,
|
|
"in": 100,
|
|
"out": 100
|
|
},
|
|
"matches": 100,
|
|
"patterns_per_field": {
|
|
"message": 1
|
|
},
|
|
"name": "grok"
|
|
},
|
|
{
|
|
"id": "geoip_20e5cb7f7c9e712ef9750edf94aefb465e3e361b-3",
|
|
"events": {
|
|
"duration_in_millis": 141,
|
|
"in": 100,
|
|
"out": 100
|
|
},
|
|
"name": "geoip"
|
|
}
|
|
],
|
|
"outputs": [
|
|
{
|
|
"id": "20e5cb7f7c9e712ef9750edf94aefb465e3e361b-4",
|
|
"events": {
|
|
"in": 100,
|
|
"out": 100
|
|
},
|
|
"name": "elasticsearch"
|
|
}
|
|
]
|
|
},
|
|
"reloads": {
|
|
"last_error": null,
|
|
"successes": 0,
|
|
"last_success_timestamp": null,
|
|
"last_failure_timestamp": null,
|
|
"failures": 0
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
See <<monitoring-common-options, Common Options>> for a list of options that can be applied to all
|
|
Logstash monitoring APIs.
|
|
|
|
|
|
[[hot-threads-api]]
|
|
=== Hot Threads API
|
|
|
|
experimental[]
|
|
|
|
The hot threads API gets the current hot threads for Logstash. A hot thread is a
|
|
Java thread that has high CPU usage and executes for a longer than normal period
|
|
of time.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/hot_threads
|
|
--------------------------------------------------
|
|
|
|
The output is a JSON document that contains a breakdown of the top hot threads for
|
|
Logstash.
|
|
|
|
Example response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"hot_threads": {
|
|
"time": "2016-09-19T10:44:13-07:00",
|
|
"busiest_threads": 3,
|
|
"threads": [
|
|
{
|
|
"name": "LogStash::Runner",
|
|
"percent_of_cpu_time": 0.17,
|
|
"state": "timed_waiting",
|
|
"traces": [
|
|
"java.lang.Object.wait(Native Method)",
|
|
"java.lang.Thread.join(Thread.java:1253)",
|
|
"org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
|
|
"org.jruby.RubyThread.join(RubyThread.java:697)",
|
|
"org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
|
|
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
|
|
"org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
|
|
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)",
|
|
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)",
|
|
"org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)"
|
|
]
|
|
},
|
|
{
|
|
"name": "Ruby-0-Thread-17",
|
|
"percent_of_cpu_time": 0.11,
|
|
"state": "timed_waiting",
|
|
"path": "/Users/username/logstash-5.0.0/logstash-core/lib/logstash/pipeline.rb:471",
|
|
"traces": [
|
|
"java.lang.Object.wait(Native Method)",
|
|
"org.jruby.RubyThread.sleep(RubyThread.java:1002)",
|
|
"org.jruby.RubyKernel.sleep(RubyKernel.java:803)",
|
|
"org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)",
|
|
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)",
|
|
"org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)",
|
|
"org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)",
|
|
"rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)",
|
|
"rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)",
|
|
"org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)"
|
|
]
|
|
},
|
|
{
|
|
"name": "[main]-pipeline-manager",
|
|
"percent_of_cpu_time": 0.04,
|
|
"state": "timed_waiting",
|
|
"traces": [
|
|
"java.lang.Object.wait(Native Method)",
|
|
"java.lang.Thread.join(Thread.java:1253)",
|
|
"org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
|
|
"org.jruby.RubyThread.join(RubyThread.java:697)",
|
|
"org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
|
|
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
|
|
"org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
|
|
"org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)",
|
|
"org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)",
|
|
"org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)"
|
|
]
|
|
}
|
|
]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
|
|
The parameters allowed are:
|
|
|
|
[horizontal]
|
|
`threads`:: The number of hot threads to return. The default is 3.
|
|
`human`:: If true, returns plain text instead of JSON format. The default is false.
|
|
`ignore_idle_threads`:: If true, does not return idle threads. The default is true.
|
|
|
|
You can use the `?human` parameter to return the document in a human-readable format.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/hot_threads?human=true
|
|
--------------------------------------------------
|
|
|
|
Example of a human-readable response:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
::: {}
|
|
Hot threads at 2016-07-26T18:46:18-07:00, busiestThreads=3:
|
|
================================================================================
|
|
0.15 % of cpu usage by timed_waiting thread named 'LogStash::Runner'
|
|
java.lang.Object.wait(Native Method)
|
|
java.lang.Thread.join(Thread.java:1253)
|
|
org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
|
|
org.jruby.RubyThread.join(RubyThread.java:697)
|
|
org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
|
|
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
|
|
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
|
|
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
|
|
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
|
|
org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
|
|
--------------------------------------------------------------------------------
|
|
0.11 % of cpu usage by timed_waiting thread named 'Ruby-0-Thread-17'
|
|
/Users/username/BuildTesting/logstash-5.0.0logstash-core/lib/logstash/pipeline.rb:471
|
|
java.lang.Object.wait(Native Method)
|
|
org.jruby.RubyThread.sleep(RubyThread.java:1002)
|
|
org.jruby.RubyKernel.sleep(RubyKernel.java:803)
|
|
org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)
|
|
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)
|
|
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
|
|
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)
|
|
rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)
|
|
rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/username/BuildTesting/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)
|
|
org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)
|
|
--------------------------------------------------------------------------------
|
|
0.04 % of cpu usage by timed_waiting thread named '[main]-pipeline-manager'
|
|
java.lang.Object.wait(Native Method)
|
|
java.lang.Thread.join(Thread.java:1253)
|
|
org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)
|
|
org.jruby.RubyThread.join(RubyThread.java:697)
|
|
org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)
|
|
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)
|
|
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)
|
|
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:683)
|
|
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:286)
|
|
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:81)
|
|
|
|
--------------------------------------------------
|
|
|
|
See <<monitoring-common-options, Common Options>> for a list of options that can be applied to all
|
|
Logstash monitoring APIs.
|