logstash/docs/static/settings-file.asciidoc

[[logstash-settings-file]]
=== Settings File

You can set options in the Logstash settings file, `logstash.yml`, to control Logstash execution. For example,
you can specify pipeline settings, the location of configuration files, logging options, and other settings.
Most of the settings in the `logstash.yml` file are also available as <<command-line-flags,command-line flags>>
when you run Logstash. Any flags that you set at the command line override the corresponding settings in the
`logstash.yml` file.

The `logstash.yml` file is written in http://yaml.org/[YAML]. Its location varies by platform (see
<<dir-layout>>). You can specify settings in hierarchical form or use flat keys. For example, to use
hierarchical form to set the pipeline batch size and batch delay, you specify:

[source,yaml]
-------------------------------------------------------------------------------------
pipeline:
  batch:
    size: 125
    delay: 5
-------------------------------------------------------------------------------------

To express the same values as flat keys, you specify:

[source,yaml]
-------------------------------------------------------------------------------------
pipeline.batch.size: 125
pipeline.batch.delay: 5
-------------------------------------------------------------------------------------

The `logstash.yml` file also supports bash-style interpolation of environment variables in
setting values.

[source,yaml]
-------------------------------------------------------------------------------------
pipeline:
  batch:
    size: ${BATCH_SIZE}
    delay: ${BATCH_DELAY:5}
node:
  name: "node_${LS_NODE_NAME}"
path:
   queue: "/tmp/${QUEUE_DIR:queue}"
-------------------------------------------------------------------------------------

Note that the `${VAR_NAME:default_value}` notation is supported, setting a default batch delay
of `5` and a default `path.queue` of `/tmp/queue` in the above example.

The `logstash.yml` file includes the following settings:

[options="header"]
|=======================================================================
| Setting | Description | Default value

| `node.name`
| A descriptive name for the node.
| Machine's hostname

| `path.data`
| The directory that Logstash and its plugins use for any persistent needs.
|`LOGSTASH_HOME/data`

| `pipeline.workers`
| The number of workers that will, in parallel, execute the filter and output stages of the pipeline.
  If you find that events are backing up, or that the
  CPU is not saturated, consider increasing this number to better utilize machine processing power.
| Number of the host's CPU cores

| `pipeline.output.workers`
| The number of workers to use per output plugin instance.
| `1`

| `pipeline.batch.size`
| The maximum number of events an individual worker thread will collect from inputs
  before attempting to execute its filters and outputs.
  Larger batch sizes are generally more efficient, but come at the cost of increased memory
  overhead. You may have to increase the JVM heap size by setting the `LS_HEAP_SIZE`
  variable to effectively use the option.
| `125`

| `pipeline.batch.delay`
| When creating pipeline event batches, how long in milliseconds to wait before dispatching an undersized
  batch to filters and workers.
| `5`

| `pipeline.unsafe_shutdown`
| When set to `true`, forces Logstash to exit during shutdown even if there are still inflight events
  in memory. By default, Logstash will refuse to quit until all received events
  have been pushed to the outputs. Enabling this option can lead to data loss during shutdown.
| `false`

| `path.config`
| The path to the Logstash config for the main pipeline. If you specify a directory or wildcard,
  config files are read from the directory in alphabetical order.
| Platform-specific. See <<dir-layout>>.

| `config.string`
| A string that contains the pipeline configuration to use for the main pipeline. Use the same syntax as
  the config file.
| None

| `config.test_and_exit`
| When set to `true`, checks that the configuration is valid and then exits. Note that grok patterns are not checked for
  correctness with this setting. Logstash can read multiple config files from a directory. If you combine this
  setting with `log.level: debug`, Logstash will log the combined config file, annotating
  each config block with the source file it came from.
| `false`

| `config.reload.automatic`
| When set to `true`, periodically checks if the configuration has changed and reloads the configuration whenever it is changed.
  This can also be triggered manually through the SIGHUP signal.
| `false`

| `config.reload.interval`
| How often in seconds Logstash checks the config files for changes.
| `3`

| `config.debug`
| When set to `true`, shows the fully compiled configuration as a debug log message. You must also set `log.level: debug`.
  WARNING: The log message will include any 'password' options passed to plugin configs as plaintext, and may result
  in plaintext passwords appearing in your logs!
| `false`

| `queue.type`
| The internal queuing model to use for event buffering. Specify `memory` for legacy in-memory based queuing, or `persisted` for disk-based ACKed queueing (<<persistent-queues,persistent queues>>).
| `memory`

| `path.queue`
| The directory path where the data files will be stored when persistent queues are enabled (`queue.type: persisted`).
| `path.data/queue`

| `queue.page_capacity`
| The size of the page data files used when persistent queues are enabled (`queue.type: persisted`). The queue data consists of append-only data files separated into pages.
| 250mb

| `queue.max_events`
| The maximum number of unread events in the queue when persistent queues are enabled (`queue.type: persisted`).
| 0 (unlimited)

| `queue.max_bytes`
| The total capacity of the queue in number of bytes. Make sure the capacity of your disk drive is greater than the value you specify here. If both `queue.max_events` and `queue.max_bytes` are specified, Logstash uses whichever criteria is reached first. 
| 1024mb (1g)

| `queue.checkpoint.acks`
| The maximum number of ACKed events before forcing a checkpoint when persistent queues are enabled (`queue.type: persisted`). Specify `queue.checkpoint.acks: 0` to set this value to unlimited.
|1024

| `queue.checkpoint.writes`
| The maximum number of written events before forcing a checkpoint when persistent queues are enabled (`queue.type: persisted`). Specify `queue.checkpoint.writes: 0` to set this value to unlimited.
| 1024

| `queue.checkpoint.interval`
| The interval in milliseconds when a checkpoint is forced on the head page when persistent queues are enabled (`queue.type: persisted`). Specify `queue.checkpoint.interval: 0` for no periodic checkpoint.
| 1000

| `dead_letter_queue.enable`
| Flag to instruct Logstash to enable the DLQ feature supported by plugins.
| `false`

| `path.dead_letter_queue`
| The directory path where the data files will be stored for the dead-letter queue.
| `path.data/dead_letter_queue`

| `http.host`
| The bind address for the metrics REST endpoint.
| `"127.0.0.1"`

| `http.port`
| The bind port for the metrics REST endpoint.
| `9600`

| `log.level`
a|
The log level. Valid options are:

* `fatal` 
* `error`
* `warn`
* `info`
* `debug`
* `trace`

| `info`

| `log.format`
| The log format. Set to `json` to log in JSON format, or `plain` to use `Object#.inspect`.
| `plain`

| `path.logs`
| The directory where Logstash will write its log to.
| `LOGSTASH_HOME/logs

| `path.plugins`
| Where to find custom plugins. You can specify this setting multiple times to include
  multiple paths. Plugins are expected to be in a specific directory hierarchy:
  `PATH/logstash/TYPE/NAME.rb` where `TYPE` is `inputs`, `filters`, `outputs`, or `codecs`,
  and `NAME` is the name of the plugin.
| Platform-specific. See <<dir-layout>>.

|=======================================================================