[role="xpack"]
[[monitoring-with-metricbeat]]
=== Collect {ls} monitoring data with {metricbeat}
[subs="attributes"]
++++
<titleabbrev>{metricbeat} collection</titleabbrev>
++++

You can use {metricbeat} to collect data about {ls} and ship it to the
monitoring cluster. The benefit of Metricbeat collection is that the monitoring
agent remains active even if the {ls} instance does not.

This step requires {ref}/configuring-metricbeat.html[{es} with {metricbeat} monitoring setup].

To collect and ship monitoring data:

. <<disable-default,Disable default collection of monitoring metrics>>
. <<define-cluster__uuid,Specify the target `cluster_uuid`>>
. <<configure-metricbeat,Install and configure {metricbeat} to collect monitoring data>>

Want to use {agent} instead? Refer to <<monitoring-with-elastic-agent>>.

[float]
[[disable-default]]
==== Disable default collection of {ls} monitoring metrics

--
Set the `monitoring.enabled` to `false` in logstash.yml to disable to default monitoring:

[source,yaml]
----------------------------------
monitoring.enabled: false
----------------------------------

--

[float]
[[define-cluster__uuid]]
==== Determine target Elasticsearch cluster
You will need to determine which Elasticsearch cluster that {ls} will bind metrics to in the Stack Monitoring UI by specifying the `cluster_uuid`.
When pipelines contain <<plugins-outputs-elasticsearch,{es} output plugins>>, the `cluster_uuid` is automatically calculated, and the metrics should be bound without any additional settings.

To override automatic values, or if your pipeline does not contain any <<plugins-outputs-elasticsearch,{es} output plugins>>, you can bind the metrics of {ls} to a specific cluster, by defining the target cluster in the `monitoring.cluster_uuid` setting.
in the configuration file (logstash.yml):

[source,yaml]
----------------------------------
monitoring.cluster_uuid: PRODUCTION_ES_CLUSTER_UUID
----------------------------------
Refer to {ref}/cluster-stats.html[{es} cluster stats page] to figure out how to get your cluster `cluster_uuid`.

[float]
[[configure-metricbeat]]
==== Install and configure {metricbeat}

. {metricbeat-ref}/metricbeat-installation-configuration.html[Install {metricbeat}] on the
same server as {ls}. 

. Enable the `logstash-xpack` module in {metricbeat}. +
+
--
To enable the default configuration in the {metricbeat} `modules.d` directory, 
run: 

*deb or rpm:* +

["source","sh",subs="attributes"]
----
metricbeat modules enable logstash-xpack
----

*linux or mac:*

["source","sh",subs="attributes"]
----
./metricbeat modules enable logstash-xpack
----

*win:*

["source","sh",subs="attributes"]
----
PS > .{backslash}metricbeat.exe modules enable logstash-xpack
----

For more information, see 
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and 
{metricbeat-ref}/metricbeat-module-beat.html[beat module]. 
--

. Configure the `logstash-xpack` module in {metricbeat}. +
+
--
The `modules.d/logstash-xpack.yml` file contains these settings:

[source,yaml]
----------------------------------
  - module: logstash
    metricsets:
      - node
      - node_stats
    period: 10s
    hosts: ["localhost:9600"]
    #username: "user"
    #password: "secret"
    xpack.enabled: true
----------------------------------
 
Set the `hosts`, `username`, and `password` to authenticate with {ls}.
For other module settings, it's recommended that you accept the
defaults.

By default, the module collects {ls} monitoring data from
`localhost:9600`. 

To monitor multiple {ls} instances, specify a list of hosts, for example:
[source,yaml]
----------------------------------
hosts: ["http://localhost:9601","http://localhost:9602","http://localhost:9603"]
----------------------------------

**Elastic security.** The Elastic {security-features} are enabled by default. 
You must provide a user ID and password so that {metricbeat} can collect metrics
successfully:

.. Create a user on the production cluster that has the 
`remote_monitoring_collector` {ref}/built-in-roles.html[built-in role]. 

.. Add the `username` and `password` settings to the module configuration 
file (`logstash-xpack.yml`).
--

. Optional: Disable the system module in the {metricbeat}.
+
--
By default, the {metricbeat-ref}/metricbeat-module-system.html[system module] is
enabled. The information it collects, however, is not shown on the
*Stack Monitoring* page in {kib}. Unless you want to use that information for
other purposes, run the following command:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
metricbeat modules disable system
----------------------------------------------------------------------
--

. Identify where to send the monitoring data. +
+
--
TIP: In production environments, we strongly recommend using a separate cluster 
(referred to as the _monitoring cluster_) to store the data. Using a separate 
monitoring cluster prevents production cluster outages from impacting your 
ability to access your monitoring data. It also prevents monitoring activities 
from impacting the performance of your production cluster.

For example, specify the {es} output information in the {metricbeat} 
configuration file (`metricbeat.yml`):

[source,yaml]
----------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>
  
  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"
----------------------------------
<1> In this example, the data is stored on a monitoring cluster with nodes 
`es-mon-1` and `es-mon-2`.

If you configured the monitoring cluster to use encrypted communications, you
must access it via HTTPS. For example, use a `hosts` setting like
`https://es-mon-1:9200`.

IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
cluster that stores the monitoring data must have at least one ingest node.

**Elastic security.** The Elastic {security-features} are enabled by default.
You must provide a user ID and password so that {metricbeat} can send metrics
successfully: 

.. Create a user on the monitoring cluster that has the 
`remote_monitoring_agent` {ref}/built-in-roles.html[built-in role]. 
Alternatively, use the `remote_monitoring_user` 
{ref}/built-in-users.html[built-in user]. 
+
TIP: If you're using index lifecycle management, the remote monitoring user
requires additional privileges to create and read indices. For more
information, see `<<feature-roles>>`.

.. Add the `username` and `password` settings to the {es} output information in 
the {metricbeat} configuration file.

For more information about these configuration options, see 
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}] to begin
collecting monitoring data. 

. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}]. 

Your monitoring setup is complete.
