[[getting-started-with-logstash]] == Getting Started with Logstash This section guides you through the process of installing Logstash and verifying that everything is running properly. After learning how to stash your first event, you go on to create a more advanced pipeline that takes Apache web logs as input, parses the logs, and writes the parsed data to an Elasticsearch cluster. Then you learn how to stitch together multiple input and output plugins to unify data from a variety of disparate sources. This section includes the following topics: * <> * <> * <> * {logstash-ref}/advanced-pipeline.html[Parsing Logs with Logstash] * {logstash-ref}/multiple-input-output-plugins.html[Stitching Together Multiple Input and Output Plugins] include::jvm.asciidoc[] [[installing-logstash]] === Installing Logstash [float] [[installing-binary]] === Installing from a Downloaded Binary The {ls} binaries are available from https://www.elastic.co/downloads/logstash[https://www.elastic.co/downloads]. Download the Logstash installation file for your host environment--TARG.GZ, DEB, ZIP, or RPM. Unpack the file. Do not install Logstash into a directory path that contains colon (:) characters. [NOTE] -- These packages are free to use under the Elastic license. They contain open source and free commercial features and access to paid commercial features. {kibana-ref}/managing-licenses.html[Start a 30-day trial] to try out all of the paid commercial features. See the https://www.elastic.co/subscriptions[Subscriptions] page for information about Elastic license levels. Alternatively, you can download an `oss` package, which contains only features that are available under the Apache 2.0 license. -- On supported Linux operating systems, you can use a package manager to install Logstash. [float] [[package-repositories]] === Installing from Package Repositories We also have repositories available for APT and YUM based distributions. Note that we only provide binary packages, but no source packages, as the packages are created as part of the Logstash build. We have split the Logstash package repositories by version into separate urls to avoid accidental upgrades across major versions. For all {major-version}.y releases use {major-version} as version number. We use the PGP key https://pgp.mit.edu/pks/lookup?op=vindex&search=0xD27D666CD88E42B4[D88E42B4], Elastic's Signing Key, with fingerprint 4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4 to sign all our packages. It is available from https://pgp.mit.edu. [float] ==== APT ifeval::["{release-state}"=="unreleased"] Version {logstash_version} of Logstash has not yet been released. endif::[] ifeval::["{release-state}"!="unreleased"] Download and install the Public Signing Key: [source,sh] -------------------------------------------------- wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - -------------------------------------------------- You may need to install the `apt-transport-https` package on Debian before proceeding: [source,sh] -------------------------------------------------- sudo apt-get install apt-transport-https -------------------------------------------------- // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released ifeval::["{release-state}"=="released"] Save the repository definition to +/etc/apt/sources.list.d/elastic-{major-version}.list+: ["source","sh",subs="attributes"] -------------------------------------------------- echo "deb https://artifacts.elastic.co/packages/{major-version}/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}.list -------------------------------------------------- endif::[] // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase ifeval::["{release-state}"=="prerelease"] Save the repository definition to +/etc/apt/sources.list.d/elastic-{major-version}-prerelease.list+: ["source","sh",subs="attributes"] -------------------------------------------------- echo "deb https://artifacts.elastic.co/packages/{major-version}-prerelease/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-{major-version}-prerelease.list -------------------------------------------------- endif::[] [WARNING] ================================================== Use the `echo` method described above to add the Logstash repository. Do not use `add-apt-repository` as it will add a `deb-src` entry as well, but we do not provide a source package. If you have added the `deb-src` entry, you will see an error like the following: Unable to find expected entry 'main/source/Sources' in Release file (Wrong sources.list entry or malformed file) Just delete the `deb-src` entry from the `/etc/apt/sources.list` file and the installation should work as expected. ================================================== Run `sudo apt-get update` and the repository is ready for use. You can install it with: [source,sh] -------------------------------------------------- sudo apt-get update && sudo apt-get install logstash -------------------------------------------------- See {logstash-ref}/running-logstash.html[Running Logstash] for details about managing Logstash as a system service. endif::[] [float] ==== YUM ifeval::["{release-state}"=="unreleased"] Version {logstash_version} of Logstash has not yet been released. endif::[] ifeval::["{release-state}"!="unreleased"] Download and install the public signing key: [source,sh] -------------------------------------------------- sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch -------------------------------------------------- Add the following in your `/etc/yum.repos.d/` directory in a file with a `.repo` suffix, for example `logstash.repo` // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == prerelase ifeval::["{release-state}"=="prerelease"] ["source","sh",subs="attributes"] -------------------------------------------------- [logstash-{major-version}] name=Elastic repository for {major-version} packages baseurl=https://artifacts.elastic.co/packages/{major-version}-prerelease/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md -------------------------------------------------- endif::[] // THIS IS A NESTED STATEMENT - This block executes if release-state != unreleased and release-state == released ifeval::["{release-state}"=="released"] ["source","sh",subs="attributes"] -------------------------------------------------- [logstash-{major-version}] name=Elastic repository for {major-version} packages baseurl=https://artifacts.elastic.co/packages/{major-version}/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md -------------------------------------------------- endif::[] And your repository is ready for use. You can install it with: [source,sh] -------------------------------------------------- sudo yum install logstash -------------------------------------------------- WARNING: The repositories do not work with older rpm based distributions that still use RPM v3, like CentOS5. See the {logstash-ref}/running-logstash.html[Running Logstash] document for managing Logstash as a system service. endif::[] [float] [[brew]] === Installing {ls} on macOS with Homebrew Elastic publishes Homebrew formulae so you can install {ls} with the https://brew.sh/[Homebrew] package manager. To install with Homebrew, you first need to tap the Elastic Homebrew repository: [source,sh] ------------------------- brew tap elastic/tap ------------------------- After you've tapped the Elastic Homebrew repo, you can use `brew install` to install the default distribution of {ls}: [source,sh] ------------------------- brew install elastic/tap/logstash-full ------------------------- This installs the most recently released default distribution of {ls}. To install the OSS distribution, specify `elastic/tap/logstash-oss`. [float] [[brew-start]] ==== Starting {ls} with Homebrew To have launchd start elastic/tap/logstash-full now and restart at login, run: [source,sh] ----- brew services start elastic/tap/logstash-full ----- To run {ls}, in the foreground, run: [source,sh] ----- logstash ----- ==== Docker Images are available for running Logstash as a Docker container. They are available from the Elastic Docker registry. See <> for details on how to configure and run Logstash Docker containers. [[first-event]] === Stashing Your First Event First, let's test your Logstash installation by running the most basic _Logstash pipeline_. A Logstash pipeline has two required elements, `input` and `output`, and one optional element, `filter`. The input plugins consume data from a source, the filter plugins modify the data as you specify, and the output plugins write the data to a destination. //TODO: REPLACE WITH NEW IMAGE image::static/images/basic_logstash_pipeline.png[] To test your Logstash installation, run the most basic Logstash pipeline. For example: ["source","sh",subs="attributes"] -------------------------------------------------- cd logstash-{logstash_version} bin/logstash -e 'input { stdin { } } output { stdout {} }' -------------------------------------------------- NOTE: The location of the `bin` directory varies by platform. See {logstash-ref}/dir-layout.html[Directory layout] to find the location of `bin\logstash` on your system. [IMPORTANT] .macOS Gatekeeper warnings ==== Apple's rollout of stricter notarization requirements affected the notarization of the {version} {ls} artifacts. If macOS Catalina displays a dialog when you first run {ls} that interrupts it, you will need to take an action to allow it to run. To prevent Gatekeeper checks on the {ls} files, run the following command on the downloaded `.tar.gz` archive or the directory to which was extracted: [source,sh] ---- xattr -d -r com.apple.quarantine ---- For example, if the `.tar.gz` file was extracted to the default logstash-{version} directory, the command is: [source,sh,subs="attributes"] ---- xattr -d -r com.apple.quarantine logstash-{version} ---- Alternatively, you can add a security override if a Gatekeeper popup appears by following the instructions in the _How to open an app that hasn’t been notarized or is from an unidentified developer_ section of https://support.apple.com/en-us/HT202491[Safely open apps on your Mac]. ==== The `-e` flag enables you to specify a configuration directly from the command line. Specifying configurations at the command line lets you quickly test configurations without having to edit a file between iterations. The pipeline in the example takes input from the standard input, `stdin`, and moves that input to the standard output, `stdout`, in a structured format. After starting Logstash, wait until you see "Pipeline main started" and then enter `hello world` at the command prompt: [source,shell] hello world 2013-11-21T01:22:14.405+0000 0.0.0.0 hello world Logstash adds timestamp and IP address information to the message. Exit Logstash by issuing a *CTRL-D* command in the shell where Logstash is running. Congratulations! You've created and run a basic Logstash pipeline. Next, you learn how to create a more realistic pipeline.