mirror of
https://github.com/elastic/logstash.git
synced 2025-04-23 06:08:19 -04:00
126 lines
5.5 KiB
Text
126 lines
5.5 KiB
Text
[[logging]]
|
|
=== Logging
|
|
|
|
Logstash emits internal logs during its operation, which are placed in `LS_HOME/logs` (or `/var/log/logstash` for
|
|
DEB/RPM). The default logging level is `INFO`. Logstash's logging framework is based on
|
|
http://logging.apache.org/log4j/2.x/[Log4j 2 framework], and much of its functionality is exposed directly to users.
|
|
|
|
When debugging problems, particularly problems with plugins, it can be helpful to increase the logging level to `DEBUG`
|
|
to emit more verbose messages. Previously, you could only set a log level that applied to the entire Logstash product.
|
|
Starting with 5.0, you can configure logging for a particular subsystem in Logstash. For example, if you are
|
|
debugging issues with Elasticsearch Output, you can increase log levels just for that component. This way
|
|
you can reduce noise due to excessive logging and focus on the problem area effectively.
|
|
|
|
==== Log file location
|
|
|
|
You can specify the log file location using `--path.logs` setting.
|
|
|
|
==== Log4j 2 Configuration
|
|
|
|
Logstash ships with a `log4j2.properties` file with out-of-the-box settings. You can modify this file directly to change the
|
|
rotation policy, type, and other https://logging.apache.org/log4j/2.x/manual/configuration.html#Loggers[log4j2 configuration].
|
|
You must restart Lostash to apply any changes that you make to this file.
|
|
|
|
==== Slowlog
|
|
|
|
Slow-log for Logstash adds the ability to log when a specific event takes an abnormal amount of time to make its way
|
|
through the pipeline. Just like the normal application log, you can find slow-logs in your `--path.logs` directory.
|
|
Slowlog is configured in the `logstash.yml` settings file with the following options:
|
|
|
|
[source,yaml]
|
|
------------------------------
|
|
slowlog.threshold.warn (default: -1)
|
|
slowlog.threshold.info (default: -1)
|
|
slowlog.threshold.debug (default: -1)
|
|
slowlog.threshold.trace (default: -1)
|
|
------------------------------
|
|
|
|
By default, these values are set to `-1nanos` to represent an infinite threshold where no slowlog will be invoked. These `slowlog.threshold`
|
|
fields are configured using a time-value format which enables a wide range of trigger intervals. The positive numeric ranges
|
|
can be specified using the following time units: `nanos` (nanoseconds), `micros` (microseconds), `ms` (milliseconds), `s` (second), `m` (minute),
|
|
`h` (hour), `d` (day).
|
|
|
|
Here is an example:
|
|
|
|
[source,yaml]
|
|
------------------------------
|
|
slowlog.threshold.warn: 2s
|
|
slowlog.threshold.info: 1s
|
|
slowlog.threshold.debug: 500ms
|
|
slowlog.threshold.trace: 100ms
|
|
------------------------------
|
|
|
|
In the above configuration, events that take longer than two seconds to be processed within a filter will be logged.
|
|
The logs will include the full event and filter configuration that are responsible for the slowness.
|
|
|
|
==== Logging APIs
|
|
|
|
You could modify the `log4j2.properties` file and restart your Logstash, but that is both tedious and leads to unnecessary
|
|
downtime. Instead, you can dynamically update logging levels through the logging API. These settings are effective
|
|
immediately and do not need a restart.
|
|
|
|
NOTE: By default, the logging API attempts to bind to `tcp:9600`. If this port is already in use by another Logstash
|
|
instance, you need to launch Logstash with the `--http.port` flag specified to bind to a different port. See
|
|
<<command-line-flags>> for more information.
|
|
|
|
To update logging levels, take the subsystem/module you are interested in and prepend
|
|
`logger.` to it. For example:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
curl -XPUT 'localhost:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d'
|
|
{
|
|
"logger.logstash.outputs.elasticsearch" : "DEBUG"
|
|
}
|
|
'
|
|
--------------------------------------------------
|
|
|
|
While this setting is in effect, Logstash will begin to emit DEBUG-level logs for __all__ the Elasticsearch outputs
|
|
specified in your configuration. Please note this new setting is transient and will not survive a restart.
|
|
|
|
Persistent changes should be added to `log4j2.properties`. For example:
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
logger.elasticsearchoutput.name = logstash.outputs.elasticsearch
|
|
logger.elasticsearchoutput.level = debug
|
|
--------------------------------------------------
|
|
|
|
To retrieve a list of logging subsystems available at runtime, you can do a `GET` request to `_node/logging`
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
curl -XGET 'localhost:9600/_node/logging?pretty'
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
["source","js"]
|
|
--------------------------------------------------
|
|
{
|
|
...
|
|
"loggers" : {
|
|
"logstash.agent" : "INFO",
|
|
"logstash.api.service" : "INFO",
|
|
"logstash.basepipeline" : "INFO",
|
|
"logstash.codecs.plain" : "INFO",
|
|
"logstash.codecs.rubydebug" : "INFO",
|
|
"logstash.filters.grok" : "INFO",
|
|
"logstash.inputs.beats" : "INFO",
|
|
"logstash.instrument.periodicpoller.jvm" : "INFO",
|
|
"logstash.instrument.periodicpoller.os" : "INFO",
|
|
"logstash.instrument.periodicpoller.persistentqueue" : "INFO",
|
|
"logstash.outputs.stdout" : "INFO",
|
|
"logstash.pipeline" : "INFO",
|
|
"logstash.plugins.registry" : "INFO",
|
|
"logstash.runner" : "INFO",
|
|
"logstash.shutdownwatcher" : "INFO",
|
|
"org.logstash.Event" : "INFO",
|
|
"slowlog.logstash.codecs.plain" : "TRACE",
|
|
"slowlog.logstash.codecs.rubydebug" : "TRACE",
|
|
"slowlog.logstash.filters.grok" : "TRACE",
|
|
"slowlog.logstash.inputs.beats" : "TRACE",
|
|
"slowlog.logstash.outputs.stdout" : "TRACE"
|
|
}
|
|
}
|
|
--------------------------------------------------
|