mirror of
https://github.com/elastic/logstash.git
synced 2025-06-28 09:46:03 -04:00
4b163a9e91
Restructures troubleshooting docs in preparation for expanding content. Adds info for plugin tracing to help users track down plugins that might be causing problems. Co-authored-by: João Duarte <jsvd@users.noreply.github.com> Fixes: #12228
96 lines
2.5 KiB
Text
96 lines
2.5 KiB
Text
[discrete]
|
||
[[ts-plugin-tracing]]
|
||
==== Plugin tracing
|
||
|
||
When you troubleshoot {ls} deployments, you might check the node stats API to
|
||
look for plugins that are taking too long to process data, dropping data, or
|
||
never received the data in the first place. After you identify a plugin whose
|
||
metrics indicate a potential issue, it's time to identify which plugin this
|
||
is, and where it is declared in the configuration files.
|
||
|
||
While you can define an "id" for each plugin to facilitate this discovery,
|
||
naming each plugin is not practical, especially for very large pipelines
|
||
containing hundreds of plugins.
|
||
|
||
You can use the information from the Logstash APIs to fetch link an auto
|
||
generated ID of a plugin to its declaration (starting with 7.6.0).
|
||
|
||
Here's how:
|
||
|
||
. <<browse-stats>>
|
||
. <<find-declaration>>
|
||
. <<lookup-def>>
|
||
|
||
[discrete]
|
||
[[browse-stats]]
|
||
===== Browse stats API and find a plugin you want to investigate
|
||
|
||
Example: "Give me any filter whose in/out events counters don't match"
|
||
|
||
[source,shell]
|
||
-----
|
||
❯ curl -s localhost:9600/_node/stats | jq '.pipelines.main.plugins.filters[] | select(.events.in!=.events.out)'
|
||
-----
|
||
|
||
[source,json]
|
||
-----
|
||
{
|
||
"id": "75afda0f03a5af46279c4cba9408ca87664b9c988bf477e2a2cca535e59e856f",
|
||
"events": {
|
||
"in": 1,
|
||
"out": 0,
|
||
"duration_in_millis": 5
|
||
},
|
||
"name": "drop"
|
||
}
|
||
-----
|
||
|
||
[discrete]
|
||
[[find-declaration]]
|
||
===== Find the plugin declaration in the pipeline graph
|
||
|
||
Take the id from step 1 and use it to find the plugin in the pipeline graph:
|
||
|
||
[source,shell]
|
||
-----
|
||
❯ curl -s localhost:9600/_node/pipelines?graph=true | jq '.pipelines.main.graph.graph.vertices[] | select(.id=="75afda0f03a5af46279c4cba9408ca87664b9c988bf477e2a2cca535e59e856f")'
|
||
-----
|
||
|
||
[source,json]
|
||
-----
|
||
{
|
||
"config_name": "drop",
|
||
"plugin_type": "filter",
|
||
"meta": {
|
||
"source": {
|
||
"protocol": "file",
|
||
"id": "/private/tmp/logstash-7.9.1/cfg",
|
||
"line": 10,
|
||
"column": 5
|
||
}
|
||
},
|
||
"id": "75afda0f03a5af46279c4cba9408ca87664b9c988bf477e2a2cca535e59e856f",
|
||
"explicit_id": false,
|
||
"type": "plugin"
|
||
}
|
||
-----
|
||
|
||
[discrete]
|
||
[[lookup-def]]
|
||
===== Lookup the plugin's definition in the source files
|
||
|
||
Here's a simple script to do the lookup.
|
||
|
||
[source,shell]
|
||
-----
|
||
❯ cat /private/tmp/logstash-7.9.1/cfg | ruby -e 'line = 10; $stdin.read.split("\n").each_with_index {|l, i| puts "#{i+1}: #{l}" if (i+1).between?(line-1, line + 5) }'
|
||
9: } else if [message] == "d" {
|
||
10: drop {}
|
||
11: } else if [message] == "e" {
|
||
12: drop {}
|
||
13: } else if [message] == "f" {
|
||
14: drop {}
|
||
15: } else if [message] == "g" {
|
||
-----
|
||
|
||
Or, you can open the file and go to the line.
|