[[logstash-reference]] = Logstash Reference include::{docs-root}/shared/versions/stack/{source_branch}.asciidoc[] include::{docs-root}/shared/attributes.asciidoc[] include::./include/attributes-ls.asciidoc[] include::./include/attributes-lsplugins.asciidoc[] :include-xpack: true :lang: en :xls-repo-dir: {docdir}/../x-pack/docs/{lang} :log-repo-dir: {docdir} :plugins-repo-dir: {docdir}/../../logstash-docs/docs :docker-repo: docker.elastic.co/logstash/logstash :docker-image: {docker-repo}:{logstash_version} :versioned_docs: false :jdk: 1.8.0 :lsissue: https://github.com/elastic/logstash/issues :lsplugindocs: https://www.elastic.co/guide/en/logstash-versioned-plugins/current :tab-widget-dir: {docdir}/static/tab-widgets [[introduction]] == Logstash Introduction Logstash is an open source data collection engine with real-time pipelining capabilities. Logstash can dynamically unify data from disparate sources and normalize the data into destinations of your choice. Cleanse and democratize all your data for diverse advanced downstream analytics and visualization use cases. While Logstash originally drove innovation in log collection, its capabilities extend well beyond that use case. Any type of event can be enriched and transformed with a broad array of input, filter, and output plugins, with many native codecs further simplifying the ingestion process. Logstash accelerates your insights by harnessing a greater volume and variety of data. [serverless] .Logstash to {serverless-full} **** You'll use the {ls} <> to send data to {serverless-full}. Note these differences between {es-serverless} and both {ess} and self-managed {es}: * Use *API keys* to access {serverless-full} from {ls}. Any user-based security settings in your in your <> configuration are ignored and may cause errors. * {serverless-full} uses *data streams* and {ref}/data-stream-lifecycle.html[{dlm} ({dlm-init})] instead of {ilm} ({ilm-init}). Any {ilm-init} settings in your <> configuration are ignored and may cause errors. * *{ls} monitoring* is available through the https://github.com/elastic/integrations/blob/main/packages/logstash/_dev/build/docs/README.md[{ls} Integration] in {serverless-docs}/observability/what-is-observability-serverless[Elastic Observability] on {serverless-full}. .Known issue for {ls} to {es-serverless}. The logstash-output-elasticsearch `hosts` setting defaults to port :9200. Set the value to port :443 instead. **** // The pass blocks here point to the correct repository for the edit links in the guide. // Introduction // Getting Started with Logstash include::static/getting-started-with-logstash.asciidoc[] // Advanced LS Pipelines include::static/advanced-pipeline.asciidoc[] // Processing Pipeline include::static/life-of-an-event.asciidoc[] // Elastic Common Schema (ECS) include::static/ecs-compatibility.asciidoc[] // Processing details include::static/processing-info.asciidoc[] // Logstash setup include::static/setting-up-logstash.asciidoc[] include::static/settings-file.asciidoc[] include::static/keystore.asciidoc[] include::static/running-logstash-command-line.asciidoc[] include::static/running-logstash.asciidoc[] include::static/docker.asciidoc[] include::static/running-logstash-kubernetes.asciidoc[] include::static/running-logstash-windows.asciidoc[] include::static/logging.asciidoc[] include::static/shutdown.asciidoc[] // Upgrading Logstash include::static/upgrading.asciidoc[] // Configuring pipelines include::static/pipeline-configuration.asciidoc[] // Security include::static/security/logstash.asciidoc[] // Advanced Logstash Configuration include::static/configuration-advanced.asciidoc[] include::static/multiple-pipelines.asciidoc[] include::static/pipeline-pipeline-config.asciidoc[] include::static/reloading-config.asciidoc[] include::static/managing-multiline-events.asciidoc[] include::static/glob-support.asciidoc[] include::static/ingest-convert.asciidoc[] include::static/field-reference.asciidoc[] //The `field-reference.asciidoc` file (included above) contains a //`role="exclude"` attribute to pull in the topic and make it linkable in the LS //Ref, but not appear in the main TOC. The `exclude`attribute was carrying //forward for all subsequent topics under the `configuration.asciidoc` heading. //This include should remain after includes for all other topics under the //`Advanced Logstash Configuration` heading. // Logstash-to-Logstash include::static/ls-ls-config.asciidoc[] // Centralized configuration managements include::static/config-management.asciidoc[] include::static/management/configuring-centralized-pipelines.asciidoc[] // EA Integrations to Logstash // (Planting near module content for now. Will likely move it up in info architecture.) include::static/ea-integrations.asciidoc[] // Working with Logstash Modules include::static/modules.asciidoc[] include::static/arcsight-module.asciidoc[] include::static/netflow-module.asciidoc[] include::static/azure-module.asciidoc[] // Working with Filebeat Modules include::static/filebeat-modules.asciidoc[] // Working with Winlogbeat Modules include::static/winlogbeat-modules.asciidoc[] // Data resiliency include::static/resiliency.asciidoc[] include::static/mem-queue.asciidoc[] include::static/persistent-queues.asciidoc[] include::static/dead-letter-queues.asciidoc[] // Transforming Data include::static/transforming-data.asciidoc[] // Deploying & Scaling include::static/deploying.asciidoc[] // GeoIP Database Management include::static/geoip-database-management.asciidoc[] // Troubleshooting performance include::static/performance-checklist.asciidoc[] // Monitoring include::static/monitoring/monitoring-ea-intro.asciidoc[] include::static/monitoring/monitoring-overview.asciidoc[] include::static/monitoring/monitoring.asciidoc[] // Working with Plugins include::static/plugin-manager.asciidoc[] // These files do their own pass blocks include::{plugins-repo-dir}/plugins/integrations.asciidoc[] include::{plugins-repo-dir}/plugins/inputs.asciidoc[] include::{plugins-repo-dir}/plugins/outputs.asciidoc[] include::{plugins-repo-dir}/plugins/filters.asciidoc[] include::{plugins-repo-dir}/plugins/codecs.asciidoc[] // FAQ and Troubleshooting :edit_url!: include::static/best-practice.asciidoc[] include::static/config-details.asciidoc[] include::static/troubleshoot/troubleshooting.asciidoc[] // Contributing to Logstash :edit_url: include::static/contributing-to-logstash.asciidoc[] include::static/input.asciidoc[] include::static/codec.asciidoc[] include::static/filter.asciidoc[] include::static/output.asciidoc[] // Logstash Community Maintainer Guide include::static/maintainer-guide.asciidoc[] // Plugin doc guidelines include::static/doc-for-plugin.asciidoc[] // Submitting a Plugin include::static/submitting-a-plugin.asciidoc[] include::static/listing-a-plugin.asciidoc[] include::static/contributing-patch.asciidoc[] include::static/contribute-core.asciidoc[] // Contributing to Logstash - JAVA EDITION :edit_url: include::static/contributing-java-plugin.asciidoc[] // Breaking Changes include::static/breaking-changes.asciidoc[] // Release Notes include::static/releasenotes.asciidoc[] :edit_url: include::static/redirects.asciidoc[] :edit_url!: