mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 22:57:16 -04:00
142 lines
5.7 KiB
Text
142 lines
5.7 KiB
Text
[role="xpack"]
|
|
[[monitoring-internal-collection]]
|
|
=== Collect {ls} monitoring data using internal collectors
|
|
++++
|
|
<titleabbrev>Internal collection</titleabbrev>
|
|
++++
|
|
|
|
Using internal collectors for {ls} {monitoring} these components:
|
|
|
|
* <<logstash-monitoring-collectors,Collectors>>
|
|
* <<logstash-monitoring-output,Output>>
|
|
|
|
These pieces live outside of the default Logstash pipeline in a dedicated monitoring
|
|
pipeline. This configuration ensures that all data and processing has a minimal
|
|
impact on ordinary Logstash processing. Existing Logstash features, such as the
|
|
<<plugins-outputs-elasticsearch,`elasticsearch` output>>, can be reused to
|
|
benefit from its retry policies.
|
|
|
|
NOTE: The `elasticsearch` output that is used by {monitoring} for Logstash is
|
|
configured exclusively via settings found in `logstash.yml`. It is not
|
|
configured by using anything from the Logstash configurations that might also be
|
|
using their own separate `elasticsearch` outputs.
|
|
|
|
The {es} cluster that is configured for use with {monitoring} for Logstash is
|
|
expected to be the production cluster. This configuration enables the production
|
|
{es} cluster to add metadata (for example, its cluster UUID) to the Logstash
|
|
monitoring data then route it to the monitoring clusters. For more information
|
|
about typical monitoring architectures, see
|
|
{ref}/how-monitoring-works.html[How monitoring works] in the {ref}[Elasticsearch Reference].
|
|
|
|
include::collectors.asciidoc[]
|
|
include::monitoring-output.asciidoc[]
|
|
|
|
|
|
[[configure-internal-collectors]]
|
|
==== Configure {ls} monitoring with internal collectors
|
|
++++
|
|
<titleabbrev>Configure internal collection</titleabbrev>
|
|
++++
|
|
|
|
To monitor Logstash nodes:
|
|
|
|
. Specify where to send monitoring data. This cluster is often referred to as
|
|
the _production cluster_. For examples of typical monitoring architectures, see
|
|
{ref}/how-monitoring-works.html[How monitoring works].
|
|
+
|
|
--
|
|
IMPORTANT: To visualize Logstash as part of the Elastic Stack (as shown in Step
|
|
6), send metrics to your _production_ cluster. Sending metrics to a dedicated
|
|
monitoring cluster will show the Logstash metrics under the _monitoring_ cluster.
|
|
|
|
--
|
|
|
|
. Verify that the `xpack.monitoring.collection.enabled` setting is `true` on the
|
|
production cluster. If that setting is `false`, the collection of monitoring data
|
|
is disabled in {es} and data is ignored from all other sources.
|
|
|
|
. Configure your Logstash nodes to send metrics by setting the
|
|
`xpack.monitoring.elasticsearch.hosts` in `logstash.yml`. If {security} is
|
|
enabled, you also need to specify the credentials for the
|
|
{ref}/built-in-users.html[built-in `logstash_system` user]. For more
|
|
information about these settings, see <<monitoring-settings>>.
|
|
+
|
|
--
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
xpack.monitoring.elasticsearch.hosts: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"] <1>
|
|
xpack.monitoring.elasticsearch.username: "logstash_system" <2>
|
|
xpack.monitoring.elasticsearch.password: "changeme"
|
|
--------------------------------------------------
|
|
|
|
<1> If SSL/TLS is enabled on the production cluster, you must
|
|
connect through HTTPS. As of v5.2.1, you can specify multiple
|
|
Elasticsearch hosts as an array as well as specifying a single
|
|
host as a string. If multiple URLs are specified, Logstash
|
|
can round-robin requests to these production nodes.
|
|
<2> If {security} is disabled on the production cluster, you can omit these
|
|
`username` and `password` settings.
|
|
--
|
|
|
|
. If SSL/TLS is enabled on the production {es} cluster, specify the trusted
|
|
CA certificates that will be used to verify the identity of the nodes
|
|
in the cluster.
|
|
+
|
|
--
|
|
To add a CA certificate to a Logstash node's trusted certificates, you
|
|
can specify the location of the PEM encoded certificate with the
|
|
`certificate_authority` setting:
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
xpack.monitoring.elasticsearch.ssl.certificate_authority: /path/to/ca.crt
|
|
--------------------------------------------------
|
|
|
|
Alternatively, you can configure trusted certificates using a truststore
|
|
(a Java Keystore file that contains the certificates):
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
xpack.monitoring.elasticsearch.ssl.truststore.path: /path/to/file
|
|
xpack.monitoring.elasticsearch.ssl.truststore.password: password
|
|
--------------------------------------------------
|
|
|
|
Also, optionally, you can set up client certificate using a keystore
|
|
(a Java Keystore file that contains the certificate):
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
|
|
xpack.monitoring.elasticsearch.ssl.keystore.password: password
|
|
--------------------------------------------------
|
|
|
|
Set sniffing to `true` to enable discovery of other nodes of the {es} cluster.
|
|
It defaults to `false`.
|
|
|
|
[source,yaml]
|
|
--------------------------------------------------
|
|
xpack.monitoring.elasticsearch.sniffing: false
|
|
--------------------------------------------------
|
|
|
|
--
|
|
|
|
. Restart your Logstash nodes.
|
|
|
|
. To verify your {monitoring} configuration, point your web browser at your {kib}
|
|
host, and select **Monitoring** from the side navigation. Metrics reported from
|
|
your Logstash nodes should be visible in the Logstash section. When security is
|
|
enabled, to view the monitoring dashboards you must log in to {kib} as a user
|
|
who has the `kibana_user` and `monitoring_user` roles.
|
|
+
|
|
image::images/monitoring-ui.png["Monitoring",link="monitoring/images/monitoring-ui.png"]
|
|
|
|
[float]
|
|
[[monitoring-upgraded-logstash]]
|
|
===== Re-enabling Logstash Monitoring After Upgrading
|
|
|
|
When upgrading from older versions of {xpack}, the built-in `logstash_system`
|
|
user is disabled for security reasons. To resume monitoring,
|
|
change the password and re-enable the logstash_system user.
|
|
|
|
include::../settings/monitoring-settings.asciidoc[]
|
|
|