mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 22:57:16 -04:00
101 lines
4.3 KiB
Text
101 lines
4.3 KiB
Text
[[logging]]
|
|
=== Logging
|
|
|
|
Logstash emits internal logs during its operation, which are placed in `LS_HOME/logs` (or `/var/log/logstash` for
|
|
DEB/RPM). The default logging level is `INFO`. Logstash's logging framework is based on
|
|
http://logging.apache.org/log4j/2.x/[Log4j 2 framework], and much of its functionality is exposed directly to users.
|
|
|
|
When debugging problems, particularly problems with plugins, it can be helpful to increase the logging level to `DEBUG`
|
|
to emit more verbose messages. Previously, you could only set a log level that applied to the entire Logstash product.
|
|
Starting with 5.0, you can configure logging for a particular subsystem in Logstash. For example, if you are
|
|
debugging issues with Elasticsearch Output, you can increase log levels just for that component. This way
|
|
you can reduce noise due to excessive logging and focus on the problem area effectively.
|
|
|
|
==== Log file location
|
|
|
|
You can specify the log file location using `--path.logs` setting.
|
|
|
|
==== Log4j 2 Configuration
|
|
|
|
Logstash ships with a `log4j2.properties` file with out-of-the-box settings. You can modify this file directly to change the
|
|
rotation policy, type, and other https://logging.apache.org/log4j/2.x/manual/configuration.html#Loggers[log4j2 configuration].
|
|
You must restart Lostash to apply any changes that you make to this file.
|
|
|
|
==== Slowlog
|
|
|
|
Slow-log for Logstash adds the ability to log when a specific event takes an abnormal amount of time to make its way
|
|
through the pipeline. Just like the normal application log, you can find slow-logs in your `--path.logs` directory.
|
|
Slowlog is configured in the `logstash.yml` settings file with the following options:
|
|
|
|
[source,yaml]
|
|
------------------------------
|
|
slowlog.threshold.warn (default: -1)
|
|
slowlog.threshold.info (default: -1)
|
|
slowlog.threshold.debug (default: -1)
|
|
slowlog.threshold.trace (default: -1)
|
|
------------------------------
|
|
|
|
By default, these values are set to `-1nanos` to represent an infinite threshold where no slowlog will be invoked. These `slowlog.threshold`
|
|
fields are configured using a time-value format which enables a wide range of trigger intervals. The positive numeric ranges
|
|
can be specified using the following time units: `nanos` (nanoseconds), `micros` (microseconds), `ms` (milliseconds), `s` (second), `m` (minute),
|
|
`h` (hour), `d` (day).
|
|
|
|
Here is an example:
|
|
|
|
[source,yaml]
|
|
------------------------------
|
|
slowlog.threshold.warn: 2s
|
|
slowlog.threshold.info: 1s
|
|
slowlog.threshold.debug: 500ms
|
|
slowlog.threshold.trace: 100ms
|
|
------------------------------
|
|
|
|
In the above configuration, events that take longer than two seconds to be processed within a filter will be logged.
|
|
The logs will include the full event and filter configuration that are responsible for the slowness.
|
|
|
|
==== Logging APIs
|
|
|
|
You could modify the `log4j2.properties` file and restart your Logstash, but that is both tedious and leads to unnecessary
|
|
downtime. Instead, you can dynamically update logging levels through the logging API. These settings are effective
|
|
immediately and do not need a restart. To update logging levels, take the subsystem/module you are interested in and prepend
|
|
`logger.` to it. For example:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
PUT /_node/logging
|
|
{
|
|
"logger.logstash.outputs.elasticsearch" : "DEBUG"
|
|
}
|
|
--------------------------------------------------
|
|
|
|
While this setting is in effect, Logstash will begin to emit DEBUG-level logs for __all__ the Elasticsearch outputs
|
|
specified in your configuration. Please note this new setting is transient and will not survive a restart.
|
|
|
|
To retrieve a list of logging subsystems available at runtime, you can do a `GET` request to `_node/logging`
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
GET /_node/logging?pretty
|
|
--------------------------------------------------
|
|
|
|
Example response:
|
|
|
|
["source","js"]
|
|
--------------------------------------------------
|
|
{
|
|
...
|
|
"loggers" : {
|
|
"logstash.registry" : "WARN",
|
|
"logstash.instrument.periodicpoller.os" : "WARN",
|
|
"logstash.instrument.collector" : "WARN",
|
|
"logstash.runner" : "WARN",
|
|
"logstash.inputs.stdin" : "WARN",
|
|
"logstash.outputs.stdout" : "WARN",
|
|
"logstash.agent" : "WARN",
|
|
"logstash.api.service" : "WARN",
|
|
"logstash.instrument.periodicpoller.jvm" : "WARN",
|
|
"logstash.pipeline" : "WARN",
|
|
"logstash.codecs.line" : "WARN"
|
|
}
|
|
}
|
|
--------------------------------------------------
|