[role="xpack"]
[[logstash-pipeline-viewer]]
=== Pipeline Viewer UI

NOTE: The pipeline viewer is an {xpack} feature that requires a
paid {xpack} license. See the
https://www.elastic.co/subscriptions[Elastic Subscriptions] page for
information about obtaining a license.

The pipeline viewer in {xpack} provides a simple way for you to visualize and
monitor the behavior of complex Logstash pipeline configurations. Within the
pipeline viewer, you can explore a directed acyclic graph (DAG) representation
of the overall pipeline topology, data flow, and branching logic. The diagram
is overlayed with important metrics, like events per second and time spent in
milliseconds, for each plugin in the view.

The diagram includes visual indicators to draw your attention to potential
bottlenecks in the pipeline, making it easy for you to diagnose and fix
problems.

[IMPORTANT]
==========================================================================
When you configure the stages in your Logstash pipeline, make sure you specify
semantic IDs. If you don't specify IDs, Logstash generates them for you.

Using semantic IDs makes it easier to identify the configurations that are
causing bottlenecks. For example, you may have several grok filters running
in your pipeline. If you haven't specified semantic IDs, you won't be able
to tell at a glance which filters are slow. If you specify semantic IDs,
such as `apacheParsingGrok` and `cloudwatchGrok`, you'll know exactly which
grok filters are causing bottlenecks.

==========================================================================

Before using the pipeline viewer, you need to
{logstash-ref}/setup-xpack.html[set up {xpack}] and configure
{xpack-ref}/monitoring-logstash.html[Logstash monitoring]. 

[float]
==== View the pipeline diagram

To view the pipeline diagram:

. In Logstash, start the Logstash pipeline that you want to monitor.
+
Assuming that you've set up Logstash monitoring, Logstash will begin shipping
metrics to the monitoring cluster.

. Navigate to the Monitoring tab in Kibana.
+
You should see a Logstash section. 
+
image::static/docs/images/monitoring-ui.png[Monitoring UI]

. Click the *Pipelines* link under Logstash to see all the pipelines that are
being monitored.
+
Each pipeline is identified by a pipeline ID (`main` by default). For each
pipeline, you'll see a list of all versions of the pipeline stats that were
captured during the specified time range.
+
image::static/docs/images/pipeline-viewer-overview.png[Pipeline Overview]
+
The version information is auto-generated by Logstash. Each time you modify a
pipeline, Logstash generates a new version hash. Viewing different versions
of the pipeline stats allows you see how changes to the pipeline over time
affect throughput and other metrics. Note that Logstash stores multiple versions
of the pipeline stats; it does not store multiple versions of the pipeline
configurations themselves.

. Click a pipeline version in the list to drill down and explore the pipeline
diagram.
+
The diagram shows all the stages feeding data through the pipeline. It also shows
conditional logic.
+
image::static/docs/images/pipeline-diagram.png[Pipeline Diagram]
+
The information displayed on each node varies depending on the plugin type.
+
Here's an example of an *input* node:
+
image::static/docs/images/pipeline-input-detail.png[Input node]
+
The *I* badge indicates that this is an input stage. The node shows:
+
--
* input type - *stdin*
* user-supplied ID - *logfileRead*
* throughput expressed in events per second - *0.7 e/s*

Here's an example of a *filter* node.

image::static/docs/images/pipeline-filter-detail.png[Filter node]

The filter icon indicates that this is a filter stage. The node shows:

* filter type - *sleep*
* user-supplied ID - *caSleep*
* worker usage expressed as the percentage of total execution time - *0%*
* performance - the number of milliseconds spent processing each event - *20.00 ms/e*
* throughput - the number of events sent per second - *0.0 e/s*

Stats that are anomalously slow appear highlighted in the pipeline viewer.
This doesn't necessarily indicate a problem, but it highlights potential
bottle necks so that you can find them quickly.

An *output* node shows the same information as a filter node, but it has an
*O* badge to indicate that it is an output stage:

image::static/docs/images/pipeline-output-detail.png[Output node]
--

. Hover over a node in the diagram, and you'll see only the related nodes that
are ancestors or descendants of the current node. 

. Explore the diagram and look for performance anomalies.