mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 14:47:19 -04:00
53a346c57f
add ECK reference to doc Fixed: #15471 Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
196 lines
6.4 KiB
Text
196 lines
6.4 KiB
Text
[[setup-logstash]]
|
|
== Setting Up and Running Logstash
|
|
|
|
Before reading this section, see <<installing-logstash>> for basic installation instructions to get you started.
|
|
|
|
This section includes additional information on how to set up and run Logstash, including:
|
|
|
|
* <<dir-layout>>
|
|
* <<config-setting-files>>
|
|
* <<logstash-settings-file>>
|
|
* <<keystore>>
|
|
* <<running-logstash-command-line>>
|
|
* <<running-logstash>>
|
|
* <<docker>>
|
|
* <<docker-config>>
|
|
* <<running-logstash-kubernetes>>
|
|
* <<running-logstash-windows>>
|
|
* <<logging>>
|
|
* <<shutdown>>
|
|
|
|
|
|
[[dir-layout]]
|
|
=== Logstash Directory Layout
|
|
|
|
This section describes the default directory structure that is created when you unpack the Logstash installation packages.
|
|
|
|
[[zip-targz-layout]]
|
|
==== Directory Layout of `.zip` and `.tar.gz` Archives
|
|
|
|
The `.zip` and `.tar.gz` packages are entirely self-contained. All files and
|
|
directories are, by default, contained within the home directory -- the directory
|
|
created when unpacking the archive.
|
|
|
|
This is very convenient because you don't have to create any directories to start using Logstash, and uninstalling
|
|
Logstash is as easy as removing the home directory. However, it is advisable to change the default locations of the
|
|
config and the logs directories so that you do not delete important data later on.
|
|
|
|
[cols="<h,<,<m,<m",options="header",]
|
|
|=======================================================================
|
|
| Type | Description | Default Location | Setting
|
|
| home
|
|
| Home directory of the Logstash installation.
|
|
| `{extract.path}`- Directory created by unpacking the archive
|
|
d|
|
|
|
|
| bin
|
|
| Binary scripts, including `logstash` to start Logstash
|
|
and `logstash-plugin` to install plugins
|
|
| `{extract.path}/bin`
|
|
d|
|
|
|
|
| settings
|
|
| Configuration files, including `logstash.yml` and `jvm.options`
|
|
| `{extract.path}/config`
|
|
| `path.settings`
|
|
|
|
| logs
|
|
| Log files
|
|
| `{extract.path}/logs`
|
|
| `path.logs`
|
|
|
|
| plugins
|
|
| Local, non Ruby-Gem plugin files. Each plugin is contained in a subdirectory. Recommended for development only.
|
|
| `{extract.path}/plugins`
|
|
| `path.plugins`
|
|
|
|
| data
|
|
| Data files used by logstash and its plugins for any persistence needs.
|
|
| `{extract.path}/data`
|
|
| `path.data`
|
|
|
|
|=======================================================================
|
|
|
|
[[deb-layout]]
|
|
==== Directory Layout of Debian and RPM Packages
|
|
|
|
The Debian package and the RPM package each place config files, logs, and the settings files in the appropriate
|
|
locations for the system:
|
|
|
|
[cols="<h,<,<m,<m",options="header",]
|
|
|=======================================================================
|
|
| Type | Description | Default Location | Setting
|
|
| home
|
|
| Home directory of the Logstash installation.
|
|
| `/usr/share/logstash`
|
|
d|
|
|
|
|
| bin
|
|
| Binary scripts including `logstash` to start Logstash
|
|
and `logstash-plugin` to install plugins
|
|
| `/usr/share/logstash/bin`
|
|
d|
|
|
|
|
| settings
|
|
| Configuration files, including `logstash.yml` and `jvm.options`
|
|
| `/etc/logstash`
|
|
| `path.settings`
|
|
|
|
| conf
|
|
| Logstash pipeline configuration files
|
|
| `/etc/logstash/conf.d/*.conf`
|
|
| See `/etc/logstash/pipelines.yml`
|
|
|
|
| logs
|
|
| Log files
|
|
| `/var/log/logstash`
|
|
| `path.logs`
|
|
|
|
| plugins
|
|
| Local, non Ruby-Gem plugin files. Each plugin is contained in a subdirectory. Recommended for development only.
|
|
| `/usr/share/logstash/plugins`
|
|
| `path.plugins`
|
|
|
|
| data
|
|
| Data files used by logstash and its plugins for any persistence needs.
|
|
| `/var/lib/logstash`
|
|
| `path.data`
|
|
|
|
|=======================================================================
|
|
|
|
[[docker-layout]]
|
|
==== Directory Layout of Docker Images
|
|
|
|
The Docker images are created from the `.tar.gz` packages, and follow a
|
|
similar directory layout.
|
|
|
|
[cols="<h,<,<m,<m",options="header",]
|
|
|=======================================================================
|
|
| Type | Description | Default Location | Setting
|
|
| home
|
|
| Home directory of the Logstash installation.
|
|
| `/usr/share/logstash`
|
|
d|
|
|
|
|
| bin
|
|
| Binary scripts, including `logstash` to start Logstash
|
|
and `logstash-plugin` to install plugins
|
|
| `/usr/share/logstash/bin`
|
|
d|
|
|
|
|
| settings
|
|
| Configuration files, including `logstash.yml` and `jvm.options`
|
|
| `/usr/share/logstash/config`
|
|
| `path.settings`
|
|
|
|
| conf
|
|
| Logstash pipeline configuration files
|
|
| `/usr/share/logstash/pipeline`
|
|
| `path.config`
|
|
|
|
| plugins
|
|
| Local, non Ruby-Gem plugin files. Each plugin is contained in a subdirectory. Recommended for development only.
|
|
| `/usr/share/logstash/plugins`
|
|
| `path.plugins`
|
|
|
|
| data
|
|
| Data files used by logstash and its plugins for any persistence needs.
|
|
| `/usr/share/logstash/data`
|
|
| `path.data`
|
|
|
|
|=======================================================================
|
|
|
|
NOTE: Logstash Docker containers do not create log files by default. They log
|
|
to standard output.
|
|
|
|
[[config-setting-files]]
|
|
=== Logstash Configuration Files
|
|
|
|
Logstash has two types of configuration files: _pipeline configuration files_, which define the Logstash processing
|
|
pipeline, and _settings files_, which specify options that control Logstash startup and execution.
|
|
|
|
[[pipeline-config-files]]
|
|
==== Pipeline Configuration Files
|
|
|
|
You create pipeline configuration files when you define the stages of your Logstash processing pipeline. On deb and
|
|
rpm, you place the pipeline configuration files in the `/etc/logstash/conf.d` directory. Logstash tries to load only
|
|
files with `.conf` extension in the `/etc/logstash/conf.d directory` and ignores all other files.
|
|
|
|
See <<configuration>> for more info.
|
|
|
|
[[settings-files]]
|
|
==== Settings Files
|
|
|
|
The settings files are already defined in the Logstash installation. Logstash includes the following settings files:
|
|
|
|
*`logstash.yml`*::
|
|
Contains Logstash configuration flags. You can set flags in this file instead of passing the flags at the command
|
|
line. Any flags that you set at the command line override the corresponding settings in the `logstash.yml` file. See <<logstash-settings-file>> for more info.
|
|
*`pipelines.yml`*::
|
|
Contains the framework and instructions for running multiple pipelines in a single Logstash instance. See <<multiple-pipelines>> for more info.
|
|
*`jvm.options`*::
|
|
Contains JVM configuration flags. Use this file to set initial and maximum values for
|
|
total heap space. You can also use this file to set the locale for Logstash.
|
|
Specify each flag on a separate line. All other settings in this file are
|
|
considered expert settings.
|
|
*`log4j2.properties`*:: Contains default settings for `log4j 2` library. See <<log4j2>> for more info.
|