[role="xpack"]
[[monitoring-internal-collection]]
=== Use internal collectors to send monitoring data (Experimental)
experimental[]
++++
<titleabbrev>Internal collection (Experimental)</titleabbrev>
++++

Internal collectors send {ls} monitoring data directly to your _monitoring_ cluster.
<<monitoring-with-metricbeat, {metricbeat} collection>> is available as an alternative. 

IMPORTANT: All Logstash nodes must share the same setup. 
Otherwise, monitoring data might be routed in different ways or to different places.

[[configure-internal-collectors]]
==== Configure {ls} monitoring with internal collectors
experimental[]
++++
<titleabbrev>Configure internal collection</titleabbrev>
++++

To monitor Logstash nodes:

. Specify the location of the _monitoring cluster_. For examples of typical
monitoring architectures, see {ref}/how-monitoring-works.html[How monitoring
works] in the {ref}[Elasticsearch Reference].

. Verify that the `xpack.monitoring.collection.enabled` setting is `true` on the
monitoring cluster. If that setting is `false`, the collection of monitoring data
is disabled in {es}, and data is ignored from all other sources.

. Configure your Logstash nodes to send metrics by setting the
`monitoring.elasticsearch.hosts` in `logstash.yml`. If {security-features}
are enabled, you also need to specify the credentials for the
{ref}/built-in-users.html[built-in `logstash_system` user]. For more
information about these settings, see <<monitoring-settings>>.
+
--
[source,yaml]
--------------------------------------------------
monitoring.elasticsearch.hosts: ["http://es-monitoring-node-1:9200", "http://es-monitoring-node-2:9200"]
monitoring.elasticsearch.username: "logstash_system"
monitoring.elasticsearch.password: "changeme"
--------------------------------------------------

If SSL/TLS is enabled on the monitoring cluster, you must connect through HTTPS.
You can specify a single host as a string, or multiple Elasticsearch hosts as an
array. If multiple URLs are specified, Logstash can round-robin requests to
these monitoring nodes.
--

. If SSL/TLS is enabled on the monitoring {es} cluster, specify the trusted
CA certificates that will be used to verify the identity of the nodes
in the cluster.
+
--
To add a CA certificate to a Logstash node's trusted certificates, you
can specify the location of the PEM encoded certificate with the
`certificate_authority` setting:

[source,yaml]
--------------------------------------------------
monitoring.elasticsearch.ssl.certificate_authority: /path/to/ca.crt
--------------------------------------------------

Alternatively, you can configure trusted certificates using a truststore
(a Java Keystore file that contains the certificates):

[source,yaml]
--------------------------------------------------
monitoring.elasticsearch.ssl.truststore.path: /path/to/file
monitoring.elasticsearch.ssl.truststore.password: password
--------------------------------------------------

Also, optionally, you can set up client certificate using a keystore
(a Java Keystore file that contains the certificate):

[source,yaml]
--------------------------------------------------
monitoring.elasticsearch.ssl.keystore.path: /path/to/file
monitoring.elasticsearch.ssl.keystore.password: password
--------------------------------------------------

Set sniffing to `true` to enable discovery of other nodes of the {es} cluster.
It defaults to `false`.

[source,yaml]
--------------------------------------------------
monitoring.elasticsearch.sniffing: false
--------------------------------------------------

--

. Restart your Logstash nodes.

. To verify your monitoring configuration, point your web browser at your {kib}
host, and select **Monitoring** from the side navigation. Metrics reported from
your Logstash nodes should be visible in the Logstash section. When security is
enabled, you must log in to {kib} as a user who has the `kibana_user` and
`monitoring_user` roles.

include::../settings/monitoring-settings.asciidoc[]


[[internal-collector-components]]
==== How {ls} monitoring with internal collectors works

Monitoring {ls} with internal collectors uses these components:

* <<logstash-monitoring-collectors,Collectors>>
* <<logstash-monitoring-output,Output>>

These pieces live outside of the default Logstash pipeline in a dedicated
monitoring pipeline. This configuration ensures that all data and processing has
a minimal impact on ordinary Logstash processing. 

NOTE: The `elasticsearch` output for Logstash monitoring is configured
exclusively through settings in `logstash.yml`. 

The monitoring {es} cluster should be configured to receive {ls} monitoring
data directly from {ls}. For more information about typical monitoring
architectures, see {ref}/how-monitoring-works.html[How monitoring works] in the
{ref}[Elasticsearch Reference]. 


include::collectors.asciidoc[]
include::monitoring-output.asciidoc[]