logstash/docs/static/getting-started-with-logstash.asciidoc

[[getting-started-with-logstash]]
== Getting Started with Logstash

This section guides you through the process of installing Logstash and verifying that everything is running properly.
After learning how to stash your first event, you go on to create a more advanced pipeline that takes Apache web logs as
input, parses the logs, and writes the parsed data to an Elasticsearch cluster. Then you learn how to stitch together multiple input and output plugins to unify data from a variety of disparate sources.

This section includes the following topics:

* <<ls-jvm>>
* <<installing-logstash>>
* <<first-event>>
* {logstash-ref}/advanced-pipeline.html[Parsing Logs with Logstash]
* {logstash-ref}/multiple-input-output-plugins.html[Stitching Together Multiple Input and Output Plugins]

include::jvm.asciidoc[]

[[installing-logstash]]
=== Installing Logstash

[float]
[[installing-binary]]
=== Installing from a Downloaded Binary

The {ls} binaries are available from 
https://www.elastic.co/downloads/logstash[https://www.elastic.co/downloads].
Download the Logstash installation file for your host environment--TARG.GZ, DEB,
ZIP, or RPM. 

Unpack the file. Do not install Logstash into a directory path that
contains colon (:) characters.

[NOTE]
--
These packages are free to use under the Elastic license. They contain open 
source and free commercial features and access to paid commercial features.  
{kibana-ref}/managing-licenses.html[Start a 30-day trial] to try out all of the 
paid commercial features. See the 
https://www.elastic.co/subscriptions[Subscriptions] page for information about 
Elastic license levels. 

Alternatively, you can download an `oss` package, which contains only features 
that are available under the Apache 2.0 license. 
--

On supported Linux operating systems, you can use a package manager to install Logstash.

[float]
[[package-repositories]]
=== Installing from Package Repositories

We also have repositories available for APT and YUM based distributions. Note
that we only provide binary packages, but no source packages, as the packages
are created as part of the Logstash build.

We have split the Logstash package repositories by version into separate urls
to avoid accidental upgrades across major versions. For all {major-version}.y
releases use {major-version} as version number.

We use the PGP key
https://pgp.mit.edu/pks/lookup?op=vindex&search=0xD27D666CD88E42B4[D88E42B4],
Elastic's Signing Key, with fingerprint

    4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4

to sign all our packages. It is available from https://pgp.mit.edu.

[float]
==== APT

ifeval::["{release-state}"=="unreleased"]

Version {logstash_version} of Logstash has not yet been released.

endif::[]

ifeval::["{release-state}"!="unreleased"]

Download and install the Public Signing Key:

[source,sh]
--------------------------------------------------
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
--------------------------------------------------

You may need to install the `apt-transport-https` package on Debian before proceeding:

[source,sh]
--------------------------------------------------
sudo apt-get install apt-transport-https
--------------------------------------------------

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released

ifeval::["{release-state}"=="released"]

Save the repository definition to  +/etc/apt/sources.list.d/elastic-{major-version}.list+:

["source","sh",subs="attributes"]
--------------------------------------------------
echo "deb https://artifacts.elastic.co/packages/{major-version}/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}.list
--------------------------------------------------

endif::[]

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase

ifeval::["{release-state}"=="prerelease"]

Save the repository definition to  +/etc/apt/sources.list.d/elastic-{major-version}-prerelease.list+:

["source","sh",subs="attributes"]
--------------------------------------------------
echo "deb https://artifacts.elastic.co/packages/{major-version}-prerelease/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}-prerelease.list
--------------------------------------------------

endif::[]

[WARNING]
==================================================
Use the `echo` method described above to add the Logstash repository.  Do not
use `add-apt-repository` as it will add a `deb-src` entry as well, but we do not
provide a source package. If you have added the `deb-src` entry, you will see an
error like the following:

    Unable to find expected entry 'main/source/Sources' in Release file (Wrong sources.list entry or malformed file)

Just delete the `deb-src` entry from the `/etc/apt/sources.list` file and the
installation should work as expected.
==================================================

Run `sudo apt-get update` and the repository is ready for use. You can install
it with:

[source,sh]
--------------------------------------------------
sudo apt-get update && sudo apt-get install logstash
--------------------------------------------------

See {logstash-ref}/running-logstash.html[Running Logstash] for details about managing Logstash as a system service.

endif::[]

[float]
==== YUM

ifeval::["{release-state}"=="unreleased"]

Version {logstash_version} of Logstash has not yet been released.

endif::[]

ifeval::["{release-state}"!="unreleased"]

Download and install the public signing key:

[source,sh]
--------------------------------------------------
sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
--------------------------------------------------

Add the following in your `/etc/yum.repos.d/` directory
in a file with a `.repo` suffix, for example `logstash.repo`

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase

ifeval::["{release-state}"=="prerelease"]

["source","sh",subs="attributes"]
--------------------------------------------------
[logstash-{major-version}]
name=Elastic repository for {major-version} packages
baseurl=https://artifacts.elastic.co/packages/{major-version}-prerelease/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
--------------------------------------------------

endif::[]

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released

ifeval::["{release-state}"=="released"]

["source","sh",subs="attributes"]
--------------------------------------------------
[logstash-{major-version}]
name=Elastic repository for {major-version} packages
baseurl=https://artifacts.elastic.co/packages/{major-version}/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
--------------------------------------------------

endif::[]

And your repository is ready for use. You can install it with:

[source,sh]
--------------------------------------------------
sudo yum install logstash
--------------------------------------------------

WARNING: The repositories do not work with older rpm based distributions
         that still use RPM v3, like CentOS5.

See the {logstash-ref}/running-logstash.html[Running Logstash] document for managing Logstash as a system service.

endif::[]

[float]
[[brew]]
=== Installing {ls} on macOS with Homebrew

Elastic publishes Homebrew formulae so you can install {ls} with the
https://brew.sh/[Homebrew] package manager.

To install with Homebrew, you first need to tap the Elastic Homebrew repository:

[source,sh]
-------------------------
brew tap elastic/tap
-------------------------

After you've tapped the Elastic Homebrew repo, you can use `brew install` to
install the default distribution of {ls}:

[source,sh]
-------------------------
brew install elastic/tap/logstash-full
-------------------------

This installs the most recently released default distribution of {ls}.
To install the OSS distribution, specify `elastic/tap/logstash-oss`.


[float]
[[brew-start]]
==== Starting {ls} with Homebrew

To have launchd start elastic/tap/logstash-full now and restart at login, run:

[source,sh]
-----
brew services start elastic/tap/logstash-full
-----

To run {ls}, in the foreground, run:

[source,sh]
-----
logstash
-----


==== Docker

Images are available for running Logstash as a Docker container. They are
available from the Elastic Docker registry.

See <<docker,Running Logstash on Docker>> for
details on how to configure and run Logstash Docker containers.

[[first-event]]
=== Stashing Your First Event

First, let's test your Logstash installation by running the most basic _Logstash pipeline_.

A Logstash pipeline has two required elements, `input` and `output`, and one optional element, `filter`. The input
plugins consume data from a source, the filter plugins modify the data as you specify, and the output plugins write
the data to a destination.

//TODO: REPLACE WITH NEW IMAGE

image::static/images/basic_logstash_pipeline.png[]

To test your Logstash installation, run the most basic Logstash pipeline. For
example:

["source","sh",subs="attributes"]
--------------------------------------------------
cd logstash-{logstash_version}
bin/logstash -e 'input { stdin { } } output { stdout {} }'
--------------------------------------------------

NOTE: The location of the `bin` directory varies by platform. See {logstash-ref}/dir-layout.html[Directory layout]
to find the location of `bin\logstash` on your system.

[IMPORTANT]
.macOS Gatekeeper warnings
====
Apple's rollout of stricter notarization requirements affected the notarization
of the {version} {ls} artifacts. If macOS Catalina displays a dialog when you
first run {ls} that interrupts it, you will need to take an action to allow it
to run.
To prevent Gatekeeper checks on the {ls} files, run the following command on the
downloaded `.tar.gz` archive or the directory to which was extracted:
[source,sh]
----
xattr -d -r com.apple.quarantine <archive-or-directory>
----
For example, if the `.tar.gz` file was extracted to the default
logstash-{version} directory, the command is:
[source,sh,subs="attributes"]
----
xattr -d -r com.apple.quarantine logstash-{version}
----
Alternatively, you can add a security override if a Gatekeeper popup appears by
following the instructions in the _How to open an app that hasn’t been notarized
or is from an unidentified developer_ section of
https://support.apple.com/en-us/HT202491[Safely open apps on your Mac].
====



The `-e` flag enables you to specify a configuration directly from the command line. Specifying configurations at the
command line lets you quickly test configurations without having to edit a file between iterations.
The pipeline in the example takes input from the standard input, `stdin`, and moves that input to the standard output,
`stdout`, in a structured format.

After starting Logstash, wait until you see "Pipeline main started" and then enter `hello world` at the command prompt:

[source,shell]
hello world
2013-11-21T01:22:14.405+0000 0.0.0.0 hello world

Logstash adds timestamp and IP address information to the message. Exit Logstash by issuing a *CTRL-D* command in the
shell where Logstash is running.

Congratulations! You've created and run a basic Logstash pipeline. Next, you learn how to create a more realistic pipeline.