github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-24 22:57:16 -04:00
Code Issues Projects Releases Packages Wiki Activity

improve doc on settings file by describing options

Browse source 
Fixes #5713
This commit is contained in:
DeDe Morton 2016-07-29 19:26:20 -07:00
parent 5c7196a5e8
commit c19314f256
2 changed files with 116 additions and 30 deletions
Download patch file Download diff file Expand all files Collapse all files

57
docs/static/command-line-flags.asciidoc vendored
View file

@ -3,61 +3,68 @@
Logstash has the following flags. You can use the `--help` flag to display this information. Logstash has the following flags. You can use the `--help` flag to display this information.
You can also control Logstash execution by specifying options in the Logstash settings file. For more info, see <<logstash-settings-file>>. Instead of specifying options at the command line, we recommend that you control Logstash execution
by specifying options in the Logstash <<logstash-settings-file,settings file>>. Using a settings file
makes it easier for you to specify mutliple options, and it provides you with a single, versionable
file that you can use to start up Logstash consistently for each run.
Any flags that you set at the command line override the corresponding settings in the Logstash
<<logstash-settings-file,settings file>>.
added[5.0.0-alpha3, Command-line flags have dots instead of dashes in their names] added[5.0.0-alpha3, Command-line flags have dots instead of dashes in their names]
*`-f, --path.config CONFIGFILE`*:: *`-f, --path.config CONFIGFILE`*::
Load the Logstash config from a specific file or directory, or a wildcard. If Load the Logstash config from a specific file or directory. If
given a directory or wildcard, config files will be read from the directory in you specify a directory or wildcard, config files are read from the directory in
alphabetical order. alphabetical order.
*`-e, --config.string CONFIGSTRING`*:: *`-e, --config.string CONFIGSTRING`*::
Use the given string as the configuration data. Same syntax as the config file. Use the given string as the configuration data. Use the same syntax as the config file.
If no input is specified, `stdin { type => stdin }` is default. If no output If no input is specified, `stdin { type => stdin }` is used by default. If no output
is specified, `stdout { codec => rubydebug }}` is default. is specified, `stdout { codec => rubydebug }}` is used by default.
*`-w, --pipeline.workers COUNT`*:: *`-w, --pipeline.workers COUNT`*::
Sets the number of pipeline workers (threads) to run for filter processing (default: number of cores). Set the number of workers that will, in parallel, execute the filter and output stages of the pipeline.
If you find that events are backing up, or that the CPU is not saturated, consider increasing This defaults to the number of the host's CPU cores. If you find that events are backing up, or that
this number to better utilize machine processing power. the CPU is not saturated, consider increasing this number to better utilize machine processing power.
*`-b, --pipeline.batch.size SIZE`*:: *`-b, --pipeline.batch.size SIZE`*::
This parameter defines the maximum number of events an individual worker thread will collect Define the maximum number of events an individual worker thread will collect from inputs
before attempting to execute its filters and outputs. Default is 125 events. before attempting to execute its filters and outputs. The default is 125 events.
Larger batch sizes are generally more efficient, but come at the cost of increased memory Larger batch sizes are generally more efficient, but come at the cost of increased memory
overhead. You may have to increase the JVM heap size by setting the `LS_HEAP_SIZE` overhead. You may have to increase the JVM heap size by setting the `LS_HEAP_SIZE`
variable to effectively use the option. variable to effectively use the option.
*`-u, --pipeline.batch.delay DELAY_IN_MS`*:: *`-u, --pipeline.batch.delay DELAY_IN_MS`*::
When creating pipeline event batches, how long to wait while polling for the next event. When creating pipeline event batches, how long in milliseconds to wait before dispatching an undersized
Default is 5ms. batch to filters and workers.
The default is 5ms.
*`-l, --path.log FILE`*:: *`-l, --path.log FILE`*::
Log to a given path. Default is to log to stdout Log to a given file. The default is to log to stdout
*`--log.level`*:: *`--log.level`*::
Set the log level to "quiet", "verbose", "warn" (default), or "debug". Set the log level to "warn" (default), "quiet", "verbose", or "debug".
*`--log.format FORMAT`*:: *`--log.format FORMAT`*::
Set to "json" to log in JSON format, or "plain" (default) to use `Object#.inspect`. Set to "json" to log in JSON format, or "plain" (default) to use `Object#.inspect`.
*`--path.settings SETTINGS_DIR`*:: *`--path.settings SETTINGS_DIR`*::
Directory containing the `logstash.yml` <<logstash-settings-file,settings file>>. Set the directory containing the `logstash.yml` <<logstash-settings-file,settings file>>.
*`--node.name NAME`*:: *`--node.name NAME`*::
Set a descriptive name for the node. If no value is specified, defaults to the current hostname. Set a descriptive name for the node. If no value is specified, defaults to the machine's hostname.
*`--config.debug`*:: *`--config.debug`*::
Print the compiled config ruby code out as a debug log (you must also have `--log.level=debug` enabled). Show the fully compiled configuration as a debug log message (you must also have `--log.level=debug` enabled).
WARNING: This will include any 'password' options passed to plugin configs as plaintext, and may result WARNING: The log message will include any 'password' options passed to plugin configs as plaintext, and may result
in plaintext passwords appearing in your logs! in plaintext passwords appearing in your logs!
*`-V, --version`*:: *`-V, --version`*::
Display the version of Logstash. Display the version of Logstash.
*`-p, --path.plugins`*:: *`-p, --path.plugins`*::
A path of where to find plugins. This flag can be given multiple times to include Where to find custom plugins. This flag can be given multiple times to include
multiple paths. Plugins are expected to be in a specific directory hierarchy: multiple paths. Plugins are expected to be in a specific directory hierarchy:
`PATH/logstash/TYPE/NAME.rb` where `TYPE` is `inputs`, `filters`, `outputs`, or `codecs`, `PATH/logstash/TYPE/NAME.rb` where `TYPE` is `inputs`, `filters`, `outputs`, or `codecs`,
and `NAME` is the name of the plugin. and `NAME` is the name of the plugin.
@ -70,21 +77,21 @@ added[5.0.0-alpha3, Command-line flags have dots instead of dashes in their name
each config block with the source file it came from. each config block with the source file it came from.
*`-r, --config.reload.automatic`*:: *`-r, --config.reload.automatic`*::
Monitor configuration changes and reload the configuration whenever it is changed. Periodically check if the configuration has changed and reload the configuration whenever it is changed.
*`--config.reload.interval RELOAD_INTERVAL`*:: *`--config.reload.interval RELOAD_INTERVAL`*::
Specifies how often Logstash checks the config files for changes. The default is every 3 seconds. Set how often in seconds Logstash checks the config files for changes. The default is every 3 seconds.
*`--http.host HTTP_HOST`*:: *`--http.host HTTP_HOST`*::
Web API binding host (default: "127.0.0.1") The bind address for the metrics REST endpoint. The default is "127.0.0.1".
*`--http.port HTTP_PORT`*:: *`--http.port HTTP_PORT`*::
Web API http port (default: 9600) The bind port for the metrics REST endpoint. The default is 9600.
*`--pipeline.unsafe_shutdown`*:: *`--pipeline.unsafe_shutdown`*::
Force Logstash to exit during shutdown even if there are still inflight events Force Logstash to exit during shutdown even if there are still inflight events
in memory. By default, Logstash will refuse to quit until all received events in memory. By default, Logstash will refuse to quit until all received events
have been pushed to the outputs. have been pushed to the outputs. Enabling this option can lead to data loss during shutdown.
*`-h, --help`*:: *`-h, --help`*::
Print help Print help

89
docs/static/settings-file.asciidoc vendored
View file

@ -3,10 +3,11 @@
added[5.0.0-alpha3] added[5.0.0-alpha3]
You can set options in the Logstash settings file, `logstash.yml`, to control Logstash execution. Each setting in the You can set options in the Logstash settings file, `logstash.yml`, to control Logstash execution. For example,
`logstash.yml` file corresponds to a <<command-line-flags,command-line flag>>. you can specify pipeline settings, the location of configuration files, logging options, and other settings.
Most of the settings in the `logstash.yml` file are also available as <<command-line-flags,command-line flags>>
Any flags that you set at the command line override the corresponding settings in the `logstash.yml` file. when you run Logstash. Any flags that you set at the command line override the corresponding settings in the
`logstash.yml` file.
The `logstash.yml` file, which is written in http://http://yaml.org/[YAML], is located in `LOGSTASH_HOME/config`. You can The `logstash.yml` file, which is written in http://http://yaml.org/[YAML], is located in `LOGSTASH_HOME/config`. You can
specify settings in hierarchical form or use flat keys. For example, to use hierarchical form to set the pipeline batch specify settings in hierarchical form or use flat keys. For example, to use hierarchical form to set the pipeline batch
@ -28,4 +29,82 @@ pipeline.batch.size: 125
pipeline.batch.delay: 5 pipeline.batch.delay: 5
------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------
See <<command-line-flags>> for a description of the available options. The `logstash.yml` file includes the following settings:
*`node.name`*::
A descriptive name for the node. If no value is specified, the default is the machine's hostname.
*`path.data`*::
The directory that Logstash and its plugins use for any persistent needs. The default is `LOGSTASH_HOME/data`.
*`pipeline.workers`*::
The number of workers that will, in parallel, execute the filter and output stages of the pipeline.
This defaults to the number of the host's CPU cores. If you find that events are backing up, or that the
CPU is not saturated, consider increasing this number to better utilize machine processing power.
*`pipeline.output.workers`*::
The number of workers to use per output plugin instance.
*`pipeline.batch.size`*::
The maximum number of events an individual worker thread will collect from inputs
before attempting to execute its filters and outputs. The default is 125 events.
Larger batch sizes are generally more efficient, but come at the cost of increased memory
overhead. You may have to increase the JVM heap size by setting the `LS_HEAP_SIZE`
variable to effectively use the option.
*`pipeline.batch.delay`*::
When creating pipeline event batches, how long in milliseconds to wait before dispatching an undersized
batch to filters and workers. The default is 5ms.
*`pipeline.unsafe_shutdown`*::
When set to true, forces Logstash to exit during shutdown even if there are still inflight events
in memory. By default, Logstash will refuse to quit until all received events
have been pushed to the outputs. Enabling this option can lead to data loss during shutdown.
*`path.config`*::
The path to the Logstash config for the main pipeline. If you specify a directory or wildcard,
config files are read from the directory in alphabetical order.
*`config.string`*::
A string that contains the pipeline configuration to use for the main pipeline. Use the same syntax as
the config file.
*`config.test_and_exit`*::
When set to true, checks that the configuration is valid and then exits. Note that grok patterns are not checked for
correctness with this setting. Logstash can read multiple config files from a directory. If you combine this
setting with `log.level: debug`, Logstash will log the combined config file, annotating
each config block with the source file it came from.
*`config.reload.automatic`*::
When set to true, periodically checks if the configuration has changed and reloads the configuration whenever it is changed.
This can also be triggered manually through the SIGHUP signal.
*`config.reload.interval`*::
How often in seconds Logstash checks the config files for changes. The default is every 3 seconds.
*`config.debug`*::
When set to true, shows the fully compiled configuration as a debug log message. You must also set `log.level: debug`.
WARNING: The log message will include any 'password' options passed to plugin configs as plaintext, and may result
in plaintext passwords appearing in your logs!
*`http.host`*::
The bind address for the metrics REST endpoint. The default is "127.0.0.1".
*`http.port`*::
The bind port for the metrics REST endpoint. The default is 9600.
*`log.level`*::
The log level. Valid options are `warn` (default), `quiet`, `verbose`, or `debug`.
*`log.format`*::
The log format. Set to `json` to log in JSON format, or `plain` (default) to use `Object#.inspect`.
*`path.log`*::
The file to log to. The default is to log to stdout.
*`path.plugins`*::
Where to find custom plugins. You can specify this setting multiple times to include
multiple paths. Plugins are expected to be in a specific directory hierarchy:
`PATH/logstash/TYPE/NAME.rb` where `TYPE` is `inputs`, `filters`, `outputs`, or `codecs`,
and `NAME` is the name of the plugin.