[[getting-started-with-logstash]] == Getting Started with Logstash This section guides you through the process of installing Logstash and verifying that everything is running properly. After learning how to stash your first event, you go on to create a more advanced pipeline that takes Apache web logs as input, parses the logs, and writes the parsed data to an Elasticsearch cluster. Then you learn how to stitch together multiple input and output plugins to unify data from a variety of disparate sources. This section includes the following topics: * <> * <> * {logstash-ref}/advanced-pipeline.html[Parsing Logs with Logstash] * {logstash-ref}/multiple-input-output-plugins.html[Stitching Together Multiple Input and Output Plugins] [[installing-logstash]] === Installing Logstash NOTE: Logstash requires Java 8 or Java 11. Use the http://www.oracle.com/technetwork/java/javase/downloads/index.html[official Oracle distribution] or an open-source distribution such as http://openjdk.java.net/[OpenJDK]. To check your Java version, run the following command: [source,shell] java -version On systems with Java installed, this command produces output similar to the following: [source,shell] ----- java version "11.0.1" 2018-10-16 LTS Java(TM) SE Runtime Environment 18.9 (build 11.0.1+13-LTS) Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.1+13-LTS, mixed mode) ----- On some Linux systems, you may also need to have the `JAVA_HOME` environment exported before attempting the install, particularly if you installed Java from a tarball. This is because Logstash uses Java during installation to automatically detect your environment and install the correct startup method (SysV init scripts, Upstart, or systemd). If Logstash is unable to find the JAVA_HOME environment variable during package installation time, you may get an error message, and Logstash will be unable to start properly. IMPORTANT: Users have reported issues with Debian or RPM install packages and Java 11. We are investigating and tracking in https://github.com/elastic/logstash/issues/10593[Issue #10593]. [float] [[installing-binary]] === Installing from a Downloaded Binary Download the https://www.elastic.co/downloads/logstash[Logstash installation file] that matches your host environment. Unpack the file. Do not install Logstash into a directory path that contains colon (:) characters. [NOTE] -- These packages are free to use under the Elastic license. They contain open source and free commercial features and access to paid commercial features. {xpack-ref}/license-management.html[Start a 30-day trial] to try out all of the paid commercial features. See the https://www.elastic.co/subscriptions[Subscriptions] page for information about Elastic license levels. Alternatively, you can download an `oss` package, which contains only features that are available under the Apache 2.0 license. -- On supported Linux operating systems, you can use a package manager to install Logstash. [float] [[package-repositories]] === Installing from Package Repositories We also have repositories available for APT and YUM based distributions. Note that we only provide binary packages, but no source packages, as the packages are created as part of the Logstash build. We have split the Logstash package repositories by version into separate urls to avoid accidental upgrades across major versions. For all {major-version}.y releases use {major-version} as version number. We use the PGP key https://pgp.mit.edu/pks/lookup?op=vindex&search=0xD27D666CD88E42B4[D88E42B4], Elastic's Signing Key, with fingerprint 4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4 to sign all our packages. It is available from https://pgp.mit.edu. [float] ==== APT ifeval::["{release-state}"=="unreleased"] Version {logstash_version} of Logstash has not yet been released. endif::[] ifeval::["{release-state}"!="unreleased"] Download and install the Public Signing Key: [source,sh] -------------------------------------------------- wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - -------------------------------------------------- You may need to install the `apt-transport-https` package on Debian before proceeding: [source,sh] -------------------------------------------------- sudo apt-get install apt-transport-https -------------------------------------------------- // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released ifeval::["{release-state}"=="released"] Save the repository definition to +/etc/apt/sources.list.d/elastic-{major-version}.list+: ["source","sh",subs="attributes"] -------------------------------------------------- echo "deb https://artifacts.elastic.co/packages/{major-version}/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}.list -------------------------------------------------- endif::[] // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase ifeval::["{release-state}"=="prerelease"] Save the repository definition to +/etc/apt/sources.list.d/elastic-{major-version}-prerelease.list+: ["source","sh",subs="attributes"] -------------------------------------------------- echo "deb https://artifacts.elastic.co/packages/{major-version}-prerelease/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}-prerelease.list -------------------------------------------------- endif::[] [WARNING] ================================================== Use the `echo` method described above to add the Logstash repository. Do not use `add-apt-repository` as it will add a `deb-src` entry as well, but we do not provide a source package. If you have added the `deb-src` entry, you will see an error like the following: Unable to find expected entry 'main/source/Sources' in Release file (Wrong sources.list entry or malformed file) Just delete the `deb-src` entry from the `/etc/apt/sources.list` file and the installation should work as expected. ================================================== Run `sudo apt-get update` and the repository is ready for use. You can install it with: [source,sh] -------------------------------------------------- sudo apt-get update && sudo apt-get install logstash -------------------------------------------------- See {logstash-ref}/running-logstash.html[Running Logstash] for details about managing Logstash as a system service. endif::[] [float] ==== YUM ifeval::["{release-state}"=="unreleased"] Version {logstash_version} of Logstash has not yet been released. endif::[] ifeval::["{release-state}"!="unreleased"] Download and install the public signing key: [source,sh] -------------------------------------------------- rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch -------------------------------------------------- Add the following in your `/etc/yum.repos.d/` directory in a file with a `.repo` suffix, for example `logstash.repo` // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase ifeval::["{release-state}"=="prerelease"] ["source","sh",subs="attributes"] -------------------------------------------------- [logstash-{major-version}] name=Elastic repository for {major-version} packages baseurl=https://artifacts.elastic.co/packages/{major-version}-prerelease/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md -------------------------------------------------- endif::[] // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released ifeval::["{release-state}"=="released"] ["source","sh",subs="attributes"] -------------------------------------------------- [logstash-{major-version}] name=Elastic repository for {major-version} packages baseurl=https://artifacts.elastic.co/packages/{major-version}/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md -------------------------------------------------- endif::[] And your repository is ready for use. You can install it with: [source,sh] -------------------------------------------------- sudo yum install logstash -------------------------------------------------- WARNING: The repositories do not work with older rpm based distributions that still use RPM v3, like CentOS5. See the {logstash-ref}/running-logstash.html[Running Logstash] document for managing Logstash as a system service. endif::[] ==== Docker Images are available for running Logstash as a Docker container. They are available from the Elastic Docker registry. See <> for details on how to configure and run Logstash Docker containers. [[first-event]] === Stashing Your First Event First, let's test your Logstash installation by running the most basic _Logstash pipeline_. A Logstash pipeline has two required elements, `input` and `output`, and one optional element, `filter`. The input plugins consume data from a source, the filter plugins modify the data as you specify, and the output plugins write the data to a destination. //TODO: REPLACE WITH NEW IMAGE image::static/images/basic_logstash_pipeline.png[] To test your Logstash installation, run the most basic Logstash pipeline. For example: ["source","sh",subs="attributes"] -------------------------------------------------- cd logstash-{logstash_version} bin/logstash -e 'input { stdin { } } output { stdout {} }' -------------------------------------------------- NOTE: The location of the `bin` directory varies by platform. See {logstash-ref}/dir-layout.html[Directory layout] to find the location of `bin\logstash` on your system. The `-e` flag enables you to specify a configuration directly from the command line. Specifying configurations at the command line lets you quickly test configurations without having to edit a file between iterations. The pipeline in the example takes input from the standard input, `stdin`, and moves that input to the standard output, `stdout`, in a structured format. After starting Logstash, wait until you see "Pipeline main started" and then enter `hello world` at the command prompt: [source,shell] hello world 2013-11-21T01:22:14.405+0000 0.0.0.0 hello world Logstash adds timestamp and IP address information to the message. Exit Logstash by issuing a *CTRL-D* command in the shell where Logstash is running. Congratulations! You've created and run a basic Logstash pipeline. Next, you learn how to create a more realistic pipeline.