mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 22:57:16 -04:00
127 lines
4.8 KiB
Text
127 lines
4.8 KiB
Text
[role="xpack"]
|
|
[[monitoring-internal-collection]]
|
|
=== Use internal collectors to send monitoring data (Experimental)
|
|
experimental[]
|
|
++++
|
|
<titleabbrev>Internal collection (Experimental)</titleabbrev>
|
|
++++
|
|
|
|
Internal collectors send {ls} monitoring data directly to your _monitoring_ cluster.
|
|
<<monitoring-with-metricbeat, {metricbeat} collection>> is available as an alternative.
|
|
|
|
IMPORTANT: All Logstash nodes must share the same setup.
|
|
Otherwise, monitoring data might be routed in different ways or to different places.
|
|
|
|
[[configure-internal-collectors]]
|
|
==== Configure {ls} monitoring with internal collectors
|
|
experimental[]
|
|
++++
|
|
<titleabbrev>Configure internal collection</titleabbrev>
|
|
++++
|
|
|
|
To monitor Logstash nodes:
|
|
|
|
. Specify the location of the _monitoring cluster_. For examples of typical
|
|
monitoring architectures, see {ref}/how-monitoring-works.html[How monitoring
|
|
works] in the {ref}[Elasticsearch Reference].
|
|
|
|
. Verify that the `xpack.monitoring.collection.enabled` setting is `true` on the
|
|
monitoring cluster. If that setting is `false`, the collection of monitoring data
|
|
is disabled in {es}, and data is ignored from all other sources.
|
|
|
|
. Configure your Logstash nodes to send metrics by setting the
|
|
`monitoring.elasticsearch.hosts` in `logstash.yml`. If {security-features}
|
|
are enabled, you also need to specify the credentials for the
|
|
{ref}/built-in-users.html[built-in `logstash_system` user]. For more
|
|
information about these settings, see <<monitoring-settings>>.
|
|
+
|
|
--
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
monitoring.elasticsearch.hosts: ["http://es-monitoring-node-1:9200", "http://es-monitoring-node-2:9200"]
|
|
monitoring.elasticsearch.username: "logstash_system"
|
|
monitoring.elasticsearch.password: "changeme"
|
|
--------------------------------------------------
|
|
|
|
If SSL/TLS is enabled on the monitoring cluster, you must connect through HTTPS.
|
|
You can specify a single host as a string, or multiple Elasticsearch hosts as an
|
|
array. If multiple URLs are specified, Logstash can round-robin requests to
|
|
these monitoring nodes.
|
|
--
|
|
|
|
. If SSL/TLS is enabled on the monitoring {es} cluster, specify the trusted
|
|
CA certificates that will be used to verify the identity of the nodes
|
|
in the cluster.
|
|
+
|
|
--
|
|
To add a CA certificate to a Logstash node's trusted certificates, you
|
|
can specify the location of the PEM encoded certificate with the
|
|
`certificate_authority` setting:
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
monitoring.elasticsearch.ssl.certificate_authority: /path/to/ca.crt
|
|
--------------------------------------------------
|
|
|
|
Alternatively, you can configure trusted certificates using a truststore
|
|
(a Java Keystore file that contains the certificates):
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
monitoring.elasticsearch.ssl.truststore.path: /path/to/file
|
|
monitoring.elasticsearch.ssl.truststore.password: password
|
|
--------------------------------------------------
|
|
|
|
Also, optionally, you can set up client certificate using a keystore
|
|
(a Java Keystore file that contains the certificate):
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
monitoring.elasticsearch.ssl.keystore.path: /path/to/file
|
|
monitoring.elasticsearch.ssl.keystore.password: password
|
|
--------------------------------------------------
|
|
|
|
Set sniffing to `true` to enable discovery of other nodes of the {es} cluster.
|
|
It defaults to `false`.
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
monitoring.elasticsearch.sniffing: false
|
|
--------------------------------------------------
|
|
|
|
--
|
|
|
|
. Restart your Logstash nodes.
|
|
|
|
. To verify your monitoring configuration, point your web browser at your {kib}
|
|
host, and select **Monitoring** from the side navigation. Metrics reported from
|
|
your Logstash nodes should be visible in the Logstash section. When security is
|
|
enabled, you must log in to {kib} as a user who has the `kibana_user` and
|
|
`monitoring_user` roles.
|
|
|
|
include::../settings/monitoring-settings.asciidoc[]
|
|
|
|
|
|
[[internal-collector-components]]
|
|
==== How {ls} monitoring with internal collectors works
|
|
|
|
Monitoring {ls} with internal collectors uses these components:
|
|
|
|
* <<logstash-monitoring-collectors,Collectors>>
|
|
* <<logstash-monitoring-output,Output>>
|
|
|
|
These pieces live outside of the default Logstash pipeline in a dedicated
|
|
monitoring pipeline. This configuration ensures that all data and processing has
|
|
a minimal impact on ordinary Logstash processing.
|
|
|
|
NOTE: The `elasticsearch` output for Logstash monitoring is configured
|
|
exclusively through settings in `logstash.yml`.
|
|
|
|
The monitoring {es} cluster should be configured to receive {ls} monitoring
|
|
data directly from {ls}. For more information about typical monitoring
|
|
architectures, see {ref}/how-monitoring-works.html[How monitoring works] in the
|
|
{ref}[Elasticsearch Reference].
|
|
|
|
|
|
include::collectors.asciidoc[]
|
|
include::monitoring-output.asciidoc[]
|