mirror of
https://github.com/elastic/logstash.git
synced 2025-04-19 04:15:23 -04:00
90 lines
3.7 KiB
Text
90 lines
3.7 KiB
Text
[role="xpack"]
|
|
[[logstash-centralized-pipeline-management]]
|
|
=== Centralized Pipeline Management
|
|
|
|
The pipeline management feature centralizes the creation and
|
|
management of Logstash configuration pipelines.
|
|
|
|
NOTE: Centralized pipeline management is an {xpack} feature that is not included
|
|
with the basic license. If you want to try all of the features, you can start a
|
|
30-day trial. At the end of the trial period, you can purchase a subscription to
|
|
keep using the full functionality of the {xpack} components. For more
|
|
information, see https://www.elastic.co/subscriptions and
|
|
https://www.elastic.co/guide/en/x-pack/master/license-management.html[License
|
|
Management].
|
|
|
|
From within the pipeline
|
|
management UI in {kib}, you can control multiple Logstash instances. You can
|
|
add, edit, and delete pipeline configurations. On the Logstash side, you simply
|
|
need to enable configuration management and register Logstash to use the
|
|
centrally managed pipeline configurations.
|
|
|
|
The pipeline configurations, along with some metadata, are stored in
|
|
Elasticsearch. Any changes that you make to a pipeline definition in the UI are
|
|
picked up and loaded automatically by all Logstash instances registered to use
|
|
the pipeline. The changes are applied immediately; you do not have to restart
|
|
Logstash to pick up the changes, as long as Logstash is already registered to
|
|
use the pipeline.
|
|
|
|
NOTE: Centralized management is disabled until you configure and enable
|
|
{security}.
|
|
|
|
==== Managing Pipelines
|
|
|
|
Before using the pipeline management UI, you must:
|
|
|
|
* <<configuring-centralized-pipelines, Configure centralized pipeline management>>.
|
|
* If {kib} is protected with basic authentication, make sure your {kib} user has
|
|
the `logstash_admin` role as well as the `logstash_writer` role that you created
|
|
when you <<ls-security,configured Logstash to use basic authentication>>.
|
|
|
|
To centrally manage Logstash pipelines:
|
|
|
|
. Open {kib} in your browser and go to the Management tab. If you've set up
|
|
configuration management correctly, you'll see an area for managing Logstash.
|
|
Click the *Pipelines* link.
|
|
+
|
|
image::management/images/centralized_config.png[]
|
|
|
|
. To add a new pipeline, click the *Add* button and specify values for the
|
|
following fields:
|
|
+
|
|
--
|
|
[horizontal]
|
|
Pipeline ID::
|
|
A name that uniquely identifies the pipeline. This is the ID that you used when
|
|
you
|
|
<<configuring-centralized-pipelines,configured centralized pipeline management>>
|
|
and specified a list of pipeline IDs in the `xpack.management.pipeline.id`
|
|
setting.
|
|
|
|
Description::
|
|
A description of the pipeline configuration. This information is for your use.
|
|
|
|
Pipeline::
|
|
The pipeline configuration. You can treat the editor in the pipeline management
|
|
UI like any other editor. You don't have to worry about whitespace or indentation.
|
|
|
|
image::management/images/new_pipeline.png[]
|
|
--
|
|
|
|
. Click *Save*.
|
|
|
|
The pipeline runs on all Logstash instances that are registered to use the
|
|
pipeline. There is no validation done at the UI level. The UI will save the new
|
|
configuration, and Logstash will attempt to load it. You need to check the local
|
|
Logstash logs for configuration errors. If you're using the Logstash monitoring
|
|
feature in {xpack}, you can also navigate to the Monitoring tab to check the
|
|
status of your Logstash nodes.
|
|
|
|
You can specify multiple pipeline configurations that run in parallel on the
|
|
same Logstash node.
|
|
|
|
If you edit a pipeline configuration and save the changes, Logstash reloads
|
|
the configuration in the background and continues processing events.
|
|
|
|
If you delete a pipeline (for example, `apache`) from the UI, Logstash will
|
|
attempt to stop the pipeline if it's running. Logstash will wait until all
|
|
events have been fully processed by the pipeline. Before deleting a pipeline,
|
|
make sure you understand your data sources because stopping a pipeline may
|
|
lead to data loss.
|