[[setup-logstash]]
== Setting Up and Running Logstash

Before reading this section, see <<installing-logstash>> for basic installation instructions to get you started.

This section includes additional information on how to set up and run Logstash, including:

* <<dir-layout>>
* <<config-setting-files>>
* <<logstash-settings-file>>
* <<keystore>>
* <<running-logstash-command-line>>
* <<running-logstash>>
* <<docker>>
* <<docker-config>>
* <<running-logstash-kubernetes>>
* <<running-logstash-windows>>
* <<logging>>
* <<shutdown>>


[[dir-layout]]
=== Logstash Directory Layout

This section describes the default directory structure that is created when you unpack the Logstash installation packages.

[[zip-targz-layout]]
==== Directory Layout of `.zip` and `.tar.gz` Archives

The `.zip` and `.tar.gz` packages are entirely self-contained. All files and
directories are, by default, contained within the home directory -- the directory
created when unpacking the archive.

This is very convenient because you don't have to create any directories to start using Logstash, and uninstalling
Logstash is as easy as removing the home directory.  However, it is advisable to change the default locations of the
config and the logs directories so that you do not delete important data later on.

[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home
  | Home directory of the Logstash installation.
  | `{extract.path}`- Directory created by unpacking the archive
 d|

| bin
  | Binary scripts, including `logstash` to start Logstash
    and `logstash-plugin` to install plugins
  | `{extract.path}/bin`
 d|

| settings
  | Configuration files, including `logstash.yml` and `jvm.options`
  | `{extract.path}/config`
  | `path.settings`

| logs
  | Log files
  | `{extract.path}/logs`
  | `path.logs`

| plugins
  | Local, non Ruby-Gem plugin files. Each plugin is contained in a subdirectory. Recommended for development only.
  | `{extract.path}/plugins`
  | `path.plugins`

| data
  | Data files used by logstash and its plugins for any persistence needs.
  | `{extract.path}/data`
  | `path.data`

|=======================================================================

[[deb-layout]]
==== Directory Layout of Debian and RPM Packages

The Debian package and the RPM package each place config files, logs, and the settings files in the appropriate
locations for the system:

[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home
  | Home directory of the Logstash installation.
  | `/usr/share/logstash`
 d|

| bin
  | Binary scripts including `logstash` to start Logstash
    and `logstash-plugin` to install plugins
  | `/usr/share/logstash/bin`
 d|

| settings
  | Configuration files, including `logstash.yml` and `jvm.options`
  | `/etc/logstash`
  | `path.settings`

| conf
  | Logstash pipeline configuration files
  | `/etc/logstash/conf.d/*.conf`
  | See `/etc/logstash/pipelines.yml`

| logs
  | Log files
  | `/var/log/logstash`
  | `path.logs`

| plugins
  | Local, non Ruby-Gem plugin files. Each plugin is contained in a subdirectory. Recommended for development only.
  | `/usr/share/logstash/plugins`
  | `path.plugins`

| data
  | Data files used by logstash and its plugins for any persistence needs.
  | `/var/lib/logstash`
  | `path.data`

|=======================================================================

[[docker-layout]]
==== Directory Layout of Docker Images

The Docker images are created from the `.tar.gz` packages, and follow a
similar directory layout.

[cols="<h,<,<m,<m",options="header",]
|=======================================================================
| Type | Description | Default Location | Setting
| home
  | Home directory of the Logstash installation.
  | `/usr/share/logstash`
 d|

| bin
  | Binary scripts, including `logstash` to start Logstash
    and `logstash-plugin` to install plugins
  | `/usr/share/logstash/bin`
 d|

| settings
  | Configuration files, including `logstash.yml` and `jvm.options`
  | `/usr/share/logstash/config`
  | `path.settings`

| conf
  | Logstash pipeline configuration files
  | `/usr/share/logstash/pipeline`
  | `path.config`

| plugins
  | Local, non Ruby-Gem plugin files. Each plugin is contained in a subdirectory. Recommended for development only.
  | `/usr/share/logstash/plugins`
  | `path.plugins`

| data
  | Data files used by logstash and its plugins for any persistence needs.
  | `/usr/share/logstash/data`
  | `path.data`

|=======================================================================

NOTE: Logstash Docker containers do not create log files by default. They log
to standard output.

[[config-setting-files]]
=== Logstash Configuration Files

Logstash has two types of configuration files: _pipeline configuration files_, which define the Logstash processing
pipeline, and _settings files_, which specify options that control Logstash startup and execution.

[[pipeline-config-files]]
==== Pipeline Configuration Files

You create pipeline configuration files when you define the stages of your Logstash processing pipeline. On deb and
rpm, you place the pipeline configuration files in the `/etc/logstash/conf.d` directory. Logstash tries to load only
files with `.conf` extension in the `/etc/logstash/conf.d directory` and ignores all other files.

See <<configuration>> for more info.

[[settings-files]]
==== Settings Files

The settings files are already defined in the Logstash installation. Logstash includes the following settings files:

*`logstash.yml`*::
  Contains Logstash configuration flags. You can set flags in this file instead of passing the flags at the command
  line. Any flags that you set at the command line override the corresponding settings in the `logstash.yml` file. See <<logstash-settings-file>> for more info.
*`pipelines.yml`*::
  Contains the framework and instructions for running multiple pipelines in a single Logstash instance. See <<multiple-pipelines>> for more info.
*`jvm.options`*::
  Contains JVM configuration flags. Use this file to set initial and maximum values for
  total heap space. You can also use this file to set the locale for Logstash.
  Specify each flag on a separate line. All other settings in this file are
  considered expert settings.
*`log4j2.properties`*:: Contains default settings for `log4j 2` library. See <<log4j2>> for more info.