logstash/docs/static/getting-started-with-logstash.asciidoc

[[getting-started-with-logstash]]
== Getting Started with Logstash

This section guides you through the process of installing Logstash and verifying that everything is running properly.
After learning how to stash your first event, you go on to create a more advanced pipeline that takes Apache web logs as
input, parses the logs, and writes the parsed data to an Elasticsearch cluster. Then you learn how to stitch together multiple input and output plugins to unify data from a variety of disparate sources.

This section includes the following topics:

* <<installing-logstash>>
* <<first-event>>
* {logstash-ref}/advanced-pipeline.html[Parsing Logs with Logstash]
* {logstash-ref}/multiple-input-output-plugins.html[Stitching Together Multiple Input and Output Plugins]

[[installing-logstash]]
=== Installing Logstash

NOTE: Logstash requires Java 8 or Java 11. Use the
http://www.oracle.com/technetwork/java/javase/downloads/index.html[official Oracle distribution] or an open-source
distribution such as http://openjdk.java.net/[OpenJDK].

To check your Java version, run the following command:

[source,shell]
java -version

On systems with Java installed, this command produces output similar to the following:

[source,shell]
-----
java version "11.0.1" 2018-10-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode)
-----

On some Linux systems, you may also need to have the `JAVA_HOME` environment
exported before attempting the install, particularly if you installed Java from
a tarball.  This is because Logstash uses Java during installation to
automatically detect your environment and install the correct startup method
(SysV init scripts, Upstart, or systemd).  If Logstash is unable to find the
JAVA_HOME environment variable during package installation time, you may get an
error message, and Logstash will be unable to start properly.

IMPORTANT: Users have reported issues with Debian or RPM install packages and Java 11. 
We are investigating and tracking in https://github.com/elastic/logstash/issues/10593[Issue #10593].

[float]
[[installing-binary]]
=== Installing from a Downloaded Binary

Download the https://www.elastic.co/downloads/logstash[Logstash installation file] that matches your host environment.
Unpack the file. Do not install Logstash into a directory path that contains colon (:) characters.

[NOTE]
--
These packages are free to use under the Elastic license. They contain open 
source and free commercial features and access to paid commercial features.  
{xpack-ref}/license-management.html[Start a 30-day trial] to try out all of the 
paid commercial features. See the 
https://www.elastic.co/subscriptions[Subscriptions] page for information about 
Elastic license levels. 

Alternatively, you can download an `oss` package, which contains only features 
that are available under the Apache 2.0 license. 
--

On supported Linux operating systems, you can use a package manager to install Logstash.

[float]
[[package-repositories]]
=== Installing from Package Repositories

We also have repositories available for APT and YUM based distributions. Note
that we only provide binary packages, but no source packages, as the packages
are created as part of the Logstash build.

We have split the Logstash package repositories by version into separate urls
to avoid accidental upgrades across major versions. For all {major-version}.y
releases use {major-version} as version number.

We use the PGP key
https://pgp.mit.edu/pks/lookup?op=vindex&search=0xD27D666CD88E42B4[D88E42B4],
Elastic's Signing Key, with fingerprint

    4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4

to sign all our packages. It is available from https://pgp.mit.edu.

[float]
==== APT

ifeval::["{release-state}"=="unreleased"]

Version {logstash_version} of Logstash has not yet been released.

endif::[]

ifeval::["{release-state}"!="unreleased"]

Download and install the Public Signing Key:

[source,sh]
--------------------------------------------------
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
--------------------------------------------------

You may need to install the `apt-transport-https` package on Debian before proceeding:

[source,sh]
--------------------------------------------------
sudo apt-get install apt-transport-https
--------------------------------------------------

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released

ifeval::["{release-state}"=="released"]

Save the repository definition to  +/etc/apt/sources.list.d/elastic-{major-version}.list+:

["source","sh",subs="attributes"]
--------------------------------------------------
echo "deb https://artifacts.elastic.co/packages/{major-version}/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}.list
--------------------------------------------------

endif::[]

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase

ifeval::["{release-state}"=="prerelease"]

Save the repository definition to  +/etc/apt/sources.list.d/elastic-{major-version}-prerelease.list+:

["source","sh",subs="attributes"]
--------------------------------------------------
echo "deb https://artifacts.elastic.co/packages/{major-version}-prerelease/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}-prerelease.list
--------------------------------------------------

endif::[]

[WARNING]
==================================================
Use the `echo` method described above to add the Logstash repository.  Do not
use `add-apt-repository` as it will add a `deb-src` entry as well, but we do not
provide a source package. If you have added the `deb-src` entry, you will see an
error like the following:

    Unable to find expected entry 'main/source/Sources' in Release file (Wrong sources.list entry or malformed file)

Just delete the `deb-src` entry from the `/etc/apt/sources.list` file and the
installation should work as expected.
==================================================

Run `sudo apt-get update` and the repository is ready for use. You can install
it with:

[source,sh]
--------------------------------------------------
sudo apt-get update && sudo apt-get install logstash
--------------------------------------------------

See {logstash-ref}/running-logstash.html[Running Logstash] for details about managing Logstash as a system service.

endif::[]

[float]
==== YUM

ifeval::["{release-state}"=="unreleased"]

Version {logstash_version} of Logstash has not yet been released.

endif::[]

ifeval::["{release-state}"!="unreleased"]

Download and install the public signing key:

[source,sh]
--------------------------------------------------
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
--------------------------------------------------

Add the following in your `/etc/yum.repos.d/` directory
in a file with a `.repo` suffix, for example `logstash.repo`

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase

ifeval::["{release-state}"=="prerelease"]

["source","sh",subs="attributes"]
--------------------------------------------------
[logstash-{major-version}]
name=Elastic repository for {major-version} packages
baseurl=https://artifacts.elastic.co/packages/{major-version}-prerelease/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
--------------------------------------------------

endif::[]

// THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released

ifeval::["{release-state}"=="released"]

["source","sh",subs="attributes"]
--------------------------------------------------
[logstash-{major-version}]
name=Elastic repository for {major-version} packages
baseurl=https://artifacts.elastic.co/packages/{major-version}/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
--------------------------------------------------

endif::[]

And your repository is ready for use. You can install it with:

[source,sh]
--------------------------------------------------
sudo yum install logstash
--------------------------------------------------

WARNING: The repositories do not work with older rpm based distributions
         that still use RPM v3, like CentOS5.

See the {logstash-ref}/running-logstash.html[Running Logstash] document for managing Logstash as a system service.

endif::[]

==== Docker

Images are available for running Logstash as a Docker container. They are
available from the Elastic Docker registry.

See <<docker,Running Logstash on Docker>> for
details on how to configure and run Logstash Docker containers.

[[first-event]]
=== Stashing Your First Event

First, let's test your Logstash installation by running the most basic _Logstash pipeline_.

A Logstash pipeline has two required elements, `input` and `output`, and one optional element, `filter`. The input
plugins consume data from a source, the filter plugins modify the data as you specify, and the output plugins write
the data to a destination.

//TODO: REPLACE WITH NEW IMAGE

image::static/images/basic_logstash_pipeline.png[]

To test your Logstash installation, run the most basic Logstash pipeline. For
example:

["source","sh",subs="attributes"]
--------------------------------------------------
cd logstash-{logstash_version}
bin/logstash -e 'input { stdin { } } output { stdout {} }'
--------------------------------------------------

NOTE: The location of the `bin` directory varies by platform. See {logstash-ref}/dir-layout.html[Directory layout]
to find the location of `bin\logstash` on your system.

The `-e` flag enables you to specify a configuration directly from the command line. Specifying configurations at the
command line lets you quickly test configurations without having to edit a file between iterations.
The pipeline in the example takes input from the standard input, `stdin`, and moves that input to the standard output,
`stdout`, in a structured format.

After starting Logstash, wait until you see "Pipeline main started" and then enter `hello world` at the command prompt:

[source,shell]
hello world
2013-11-21T01:22:14.405+0000 0.0.0.0 hello world

Logstash adds timestamp and IP address information to the message. Exit Logstash by issuing a *CTRL-D* command in the
shell where Logstash is running.

Congratulations! You've created and run a basic Logstash pipeline. Next, you learn how to create a more realistic pipeline.