[[monitoring]]
== Monitoring APIs

experimental[]

Logstash provides the following monitoring APIs to retrieve runtime metrics
about Logstash:

* <<node-info-api>>
* <<plugins-api>>
* <<node-stats-api>>
* <<pipeline-stats-api>>
* <<hot-threads-api>>


You can use the root resource to retrieve general information about the Logstash instance, including
the hostname and version.

[source,js]
--------------------------------------------------
GET /
--------------------------------------------------

Example response:

["source","js",subs="attributes"]
--------------------------------------------------
{
   "hostname": "skywalker",
   "version": "{logstash_version}",
   "http_address": "127.0.0.1:9600"
}
--------------------------------------------------

NOTE: By default, the monitoring API attempts to bind to `tcp:9600`. If this port is already in use by another Logstash
instance, you need to launch Logstash with the `--http.port` flag specified to bind to a different port. See 
<<command-line-flags>> for more information.

[float]
[[monitoring-common-options]]
=== Common Options

The following options can be applied to all of the Logstash monitoring APIs.

[float]
==== Pretty Results

When appending `?pretty=true` to any request made, the JSON returned
will be pretty formatted (use it for debugging only!). Another option is
to set `?format=yaml` which will cause the result to be returned in the
(sometimes) more readable yaml format.

[float]
==== Human-Readable Output

NOTE: For Logstash {logstash_version}, the `human` option is supported for the <<hot-threads-api>>
only. When you specify `human=true`, the results are returned in plain text instead of
JSON format. The default is false.

Statistics are returned in a format suitable for humans
(eg `"exists_time": "1h"` or `"size": "1kb"`) and for computers
(eg `"exists_time_in_millis": 3600000` or `"size_in_bytes": 1024`).
The human-readable values can be turned off by adding `?human=false`
to the query string. This makes sense when the stats results are
being consumed by a monitoring tool, rather than intended for human
consumption.  The default for the `human` flag is
`false`.

[[node-info-api]]
=== Node Info API

added[5.0.0-alpha4]

experimental[]

The node info API retrieves information about the node.

[source,js]
--------------------------------------------------
GET /_node/<types>
--------------------------------------------------

Where `<types>` is optional and specifies the types of node info you want to return.

You can limit the info that's returned by specifying one of the following types: 

//TODO: For 5.0.0 Alpha4, this endpoint does not support specifying a comma-separated list like you can for the Elasticsearch cluster APIs. When this issue is fixed, we need to change the wording above to say: "You can limit the info that's returned by combining any of the following types"

[horizontal]
`pipeline`::
Gets pipeline-specific information and settings.
`os`::
Gets node-level info about the OS.
`jvm`::
Gets node-level JVM info, including info about threads.

==== Pipeline Info

The following request returns a JSON document that shows pipeline info, such as the number of workers,
batch size, and batch delay:

[source,js]
--------------------------------------------------
GET /_node/pipeline
--------------------------------------------------

See <<pipeline-stats-api>> if you want to view additional information, such as stats for each configured input, filter,
or output stage.

Example response:

["source","js",subs="attributes"]
--------------------------------------------------
{
  "pipeline": {
    "workers": 8,
    "batch_size": 125,
    "batch_delay": 5
  }
--------------------------------------------------

==== OS Info

The following request returns a JSON document that shows the OS name, architecture, version, and
available processors:

[source,js]
--------------------------------------------------
GET /_node/os
--------------------------------------------------

Example response:

[source,js]
--------------------------------------------------
{
  "os": {
    "name": "Mac OS X",
    "arch": "x86_64",
    "version": "10.11.2",
    "available_processors": 8
  }
--------------------------------------------------

==== JVM Info

The following request returns a JSON document that shows node-level JVM stats, such as the JVM process id, version,
VM info, and memory usage:

[source,js]
--------------------------------------------------
GET /_node/jvm
--------------------------------------------------

Example response:

[source,js]
--------------------------------------------------
{
  "jvm": {
    "pid": 31580,
    "version": "1.8.0_65",
    "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
    "vm_version": "1.8.0_65",
    "vm_vendor": "Oracle Corporation",
    "start_time_in_millis": 1466799661753,
    "mem": {
      "heap_init_in_bytes": 268435456,
      "heap_max_in_bytes": 1037959168,
      "non_heap_init_in_bytes": 2555904,
      "non_heap_max_in_bytes": 0
    }
  }
--------------------------------------------------

[[plugins-api]]
=== Plugins API

experimental[]

The plugins API gets information about all Logstash plugins that are currently installed.
This API basically returns the output of running the `bin/logstash-plugin list --verbose` command.

[source,js]
--------------------------------------------------
GET /_node/plugins
--------------------------------------------------

The output is a JSON document.

Example response:

["source","js",subs="attributes"]
--------------------------------------------------
{
  "plugins": [
    {
      "name": "logstash-codec-collectd",
      "version": "3.0.1"
    },
    {
      "name": "logstash-codec-dots",
      "version": "3.0.1"
    },
    {
      "name": "logstash-codec-edn",
      "version": "3.0.1"
    },
    .
    .
    .
  ]
--------------------------------------------------

[[node-stats-api]]
=== Node Stats API

added[5.0.0-beta3,Replaces the Stats Info API]

experimental[]

The node stats API retrieves runtime stats about Logstash. 

[source,js]
--------------------------------------------------
GET /_node/stats/<types>
--------------------------------------------------

Where `<types>` is optional and specifies the types of stats you want to return.

By default, all stats are returned. You can limit this by specifying one of the following types: 

//TODO: Update the above description when support for specifying a comma-separated list of types is added back in. See above comment.

[horizontal]
`events`::
Gets event information since startup.
`jvm`::
Gets JVM stats, including stats about threads. added[5.0.0-alpha3,Adds thread count]
`process`::
Gets process stats, including stats about file descriptors, memory consumption, and CPU usage. added[5.0.0-alpha3] 
`mem`::
Gets memory usage stats. added[5.0.0-alpha4] 

==== Event Stats

The following request returns a JSON document that shows the number of events
that were input, filtered, and output by Logstash since startup:

[source,js]
--------------------------------------------------
GET /_node/stats/events
--------------------------------------------------

Example response:

[source,js]
--------------------------------------------------
{
  "events" : {
    "in" : 59685,
    "filtered" : 59685,
    "out" : 59685
  }
--------------------------------------------------

==== JVM Stats

The following request returns a JSON document containing JVM stats:

[source,js]
--------------------------------------------------
GET /_node/stats/jvm
--------------------------------------------------

Example response:

[source,js]
--------------------------------------------------
{
  "jvm" : {
    "threads" : {
      "count" : 32,
      "peak_count" : 34
    }
  }
--------------------------------------------------

==== Process Stats

The following request returns a JSON document containing process stats: 

[source,js]
--------------------------------------------------
GET /_node/stats/process
--------------------------------------------------

Example response:

[source,js]
--------------------------------------------------
{
  "process" : {
    "peak_open_file_descriptors" : 64,
    "max_file_descriptors" : 10240,
    "open_file_descriptors" : 64,
    "mem" : {
      "total_virtual_in_bytes" : 5278068736
    },
    "cpu" : {
      "total_in_millis" : 103290097000,
      "percent" : 0
    }
  }
--------------------------------------------------

==== Mem Stats

The following request returns a JSON document containing memory stats: 

[source,js]
--------------------------------------------------
GET /_node/stats/mem
--------------------------------------------------

Example response:

[source,js]
--------------------------------------------------
{
  "mem": {
    "heap_used_in_bytes": 262641008,
    "heap_used_percent": 12,
    "heap_committed_in_bytes": 519045120,
    "heap_max_in_bytes": 2075918336,
    "non_heap_used_in_bytes": 184060512,
    "non_heap_committed_in_bytes": 195870720,
    "pools": {
      "survivor": {
        "peak_used_in_bytes": 8912896,
        "used_in_bytes": 9280256,
        "peak_max_in_bytes": 35782656,
        "max_in_bytes": 71565312,
        "committed_in_bytes": 17825792
      },
      "old": {
        "peak_used_in_bytes": 112649208,
        "used_in_bytes": 170055984,
        "peak_max_in_bytes": 715849728,
        "max_in_bytes": 1431699456,
        "committed_in_bytes": 357957632
      },
      "young": {
        "peak_used_in_bytes": 71630848,
        "used_in_bytes": 83304768,
        "peak_max_in_bytes": 286326784,
        "max_in_bytes": 572653568,
        "committed_in_bytes": 143261696
      }
    }
  }
--------------------------------------------------

See <<monitoring-common-options, Common Options>> for a list of options that can be applied to all
Logstash monitoring APIs.

[[pipeline-stats-api]]
=== Pipeline Stats API

added[5.0.0-alpha4,Stats for input stages are not yet available]

experimental[]

The pipeline stats API retrieves runtime stats about the Logstash pipeline.

The following request returns a JSON document containing pipeline stats, including the number of events that were
input, filtered, or output by the pipeline. The request also returns stats for each configured input, filter, or
output stage. 

[source,js]
--------------------------------------------------
GET /_node/stats/pipeline
--------------------------------------------------

Example response:

[source,js]
--------------------------------------------------
{
  "pipeline": {
    "events": {
      "in": 100,
      "filtered": 100,
      "out": 100
    },
    "pipeline": {
      "inputs": [],
      "filters": [
        {
          "id": "geoip_6aea6f9a-c100-4c39-ab00-e34d54d6e573",
          "events": {
            "duration_in_millis": 145,
            "in": 100,
            "out": 100
          },
          "name": "geoip"
        },
        {
          "id": "grok_4000f2bf-4baf-4258-b72f-56e0b6f15fc1",
          "events": {
            "duration_in_millis": 48,
            "in": 100,
            "out": 100
          },
          "matches": 100,
          "patterns_per_field": {
            "message": 1
          },
          "name": "grok"
        }
      ],
      "outputs": [
        {
          "id": "elasticsearch_db05752c-2e30-4f77-a3d7-167e915d470f",
          "events": {
            "duration_in_millis": 232,
            "in": 100,
            "out": 100
          },
          "name": "elasticsearch"
        }
      ]
    }
  }
--------------------------------------------------

See <<monitoring-common-options, Common Options>> for a list of options that can be applied to all
Logstash monitoring APIs.


[[hot-threads-api]]
=== Hot Threads API

experimental[]

The hot threads API gets the current hot threads for Logstash. A hot thread is a
Java thread that has high CPU usage and executes for a longer than normal period
of time.

[source,js]
--------------------------------------------------
GET /_node/hot_threads
--------------------------------------------------

The output is a JSON document that contains a breakdown of the top hot threads for
Logstash. 

Example response:

[source,js]
--------------------------------------------------
{
  "host": "skywalker",
  "version": "5.0.0-alpha4",
  "http_address": "127.0.0.1:9600",
  "hot_threads": {
    "time": "2016-06-24T18:19:48-07:00",
    "busiest_threads": 3,
    "threads": [
      {
        "name": "LogStash::Runner",
        "percent_of_cpu_time": 0.09,
        "state": "timed_waiting",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "java.lang.Thread.join(Thread.java:1253)",
          "org.jruby.internal.runtime.NativeThread.join(NativeThread.java:75)",
          "org.jruby.RubyThread.join(RubyThread.java:697)",
          "org.jruby.RubyThread$INVOKER$i$0$1$join.call(RubyThread$INVOKER$i$0$1$join.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:663)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:198)",
          "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)",
          "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)",
          "org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)"
        ]
      },
      {
        "name": "Ruby-0-Thread-17",
        "percent_of_cpu_time": 0.05,
        "state": "timed_waiting",
        "path": "/Users/dedemorton/BuildTesting/5.0.0-alpha4/logstash-5.0.0-alpha4/logstash-core/lib/logstash/pipeline.rb:467",
        "traces": [
          "java.lang.Object.wait(Native Method)",
          "org.jruby.RubyThread.sleep(RubyThread.java:1002)",
          "org.jruby.RubyKernel.sleep(RubyKernel.java:803)",
          "org.jruby.RubyKernel$INVOKER$s$0$1$sleep.call(RubyKernel$INVOKER$s$0$1$sleep.gen)",
          "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:667)",
          "org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)",
          "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168)",
          "rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/dedemorton/BuildTesting/5.0.0-alpha4/logstash-5.0.0-alpha4/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb:84)",
          "rubyjit.Module$$stoppable_sleep_c19c1639527ca7d373b5093f339d26538f1c21ef1028566121.__file__(/Users/dedemorton/BuildTesting/5.0.0-alpha4/logstash-5.0.0-alpha4/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/interval.rb)",
          "org.jruby.ast.executable.AbstractScript.__file__(AbstractScript.java:46)"
        ]
      }
    ]
  }
}
--------------------------------------------------

The parameters allowed are:

[horizontal]
`threads`:: 	        The number of hot threads to return. The default is 3. 
`human`:: 	            If true, returns plain text instead of JSON format. The default is false. 
`ignore_idle_threads`:: If true, does not return idle threads. The default is true.

You can use the `?human` parameter to return the document in a human-readable format.

[source,js]
--------------------------------------------------
GET /_node/hot_threads?human=true
--------------------------------------------------

Example of a human-readable response: 

[source,js]
--------------------------------------------------
::: {Ringo Kid}{Gv3UrzR3SqmPQIgfG4qJMA}{127.0.0.1}{127.0.0.1:9300}
   Hot threads at 2016-01-13T16:55:49.988Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:

    0.0% (216micros out of 500ms) cpu usage by thread 'elasticsearch[Ringo Kid][transport_client_timer][T#1]{Hashed wheel timer #1}'
     10/10 snapshots sharing following 5 elements
       java.lang.Thread.sleep(Native Method)
       org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:445)
       org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:364)
       org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
       java.lang.Thread.run(Thread.java:745)

    0.0% (216micros out of 500ms) cpu usage by thread 'elasticsearch[Ringo Kid][transport_client_timer][T#1]{Hashed wheel timer #1}'
     10/10 snapshots sharing following 5 elements
       java.lang.Thread.sleep(Native Method)
       org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:445)
       org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:364)
       org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
       java.lang.Thread.run(Thread.java:745)
--------------------------------------------------

See <<monitoring-common-options, Common Options>> for a list of options that can be applied to all
Logstash monitoring APIs.
