[role="xpack"]
[[monitoring-with-metricbeat]]
=== Collect {ls} monitoring data with {metricbeat}
[subs="attributes"]
++++
<titleabbrev>{metricbeat} collection</titleabbrev>
++++

You can use {metricbeat} to collect data about {ls} and ship it to the
monitoring cluster. The benefit of Metricbeat collection is that the monitoring
agent remains active even if the {ls} instance does not. 

To collect and ship monitoring data:

. <<disable-default,Disable default collection of monitoring metrics>>
. <<define-cluster__uuid,Specify the target `cluster_uuid` (optional)>>
. <<configure-metricbeat,Install and configure {metricbeat} to collect monitoring data>>

Want to use {agent} instead? Refer to <<monitoring-with-elastic-agent>>.

[float]
[[disable-default]]
==== Disable default collection of {ls} monitoring metrics

--
The `monitoring` setting is in the {ls} configuration file (logstash.yml), but is
commented out: 

[source,yaml]
----------------------------------
monitoring.enabled: false
----------------------------------

Remove the `#` at the beginning of the line to enable the setting.

--

[float]
[[define-cluster__uuid]]
==== Define `cluster_uuid` (optional)
To bind the metrics of {ls} to a specific cluster, optionally define the `monitoring.cluster_uuid`
in the configuration file (logstash.yml):

[source,yaml]
----------------------------------
monitoring.cluster_uuid: PRODUCTION_ES_CLUSTER_UUID
----------------------------------

[float]
[[configure-metricbeat]]
==== Install and configure {metricbeat}

. {metricbeat-ref}/metricbeat-installation-configuration.html[Install {metricbeat}] on the
same server as {ls}. 

. Enable the `logstash-xpack` module in {metricbeat}. +
+
--
To enable the default configuration in the {metricbeat} `modules.d` directory, 
run: 

*deb or rpm:* +

["source","sh",subs="attributes"]
----
metricbeat modules enable logstash-xpack
----

*linux or mac:*

["source","sh",subs="attributes"]
----
./metricbeat modules enable logstash-xpack
----

*win:*

["source","sh",subs="attributes"]
----
PS > .{backslash}metricbeat.exe modules enable logstash-xpack
----

For more information, see 
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and 
{metricbeat-ref}/metricbeat-module-beat.html[beat module]. 
--

. Configure the `logstash-xpack` module in {metricbeat}. +
+
--
The `modules.d/logstash-xpack.yml` file contains these settings:

[source,yaml]
----------------------------------
  - module: logstash
    metricsets:
      - node
      - node_stats
    period: 10s
    hosts: ["localhost:9600"]
    #username: "user"
    #password: "secret"
    xpack.enabled: true
----------------------------------
 
Set the `hosts`, `username`, and `password` to authenticate with {ls}.
For other module settings, it's recommended that you accept the
defaults.

By default, the module collects {ls} monitoring data from
`localhost:9600`. 

To monitor multiple {ls} instances, specify a list of hosts, for example:
[source,yaml]
----------------------------------
hosts: ["http://localhost:9601","http://localhost:9602","http://localhost:9603"]
----------------------------------

**Elastic security.** The Elastic {security-features} are enabled by default. 
You must provide a user ID and password so that {metricbeat} can collect metrics
successfully:

.. Create a user on the production cluster that has the 
`remote_monitoring_collector` {ref}/built-in-roles.html[built-in role]. 

.. Add the `username` and `password` settings to the module configuration 
file (`logstash-xpack.yml`).
--

. Optional: Disable the system module in the {metricbeat}.
+
--
By default, the {metricbeat-ref}/metricbeat-module-system.html[system module] is
enabled. The information it collects, however, is not shown on the
*Stack Monitoring* page in {kib}. Unless you want to use that information for
other purposes, run the following command:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
metricbeat modules disable system
----------------------------------------------------------------------
--

. Identify where to send the monitoring data. +
+
--
TIP: In production environments, we strongly recommend using a separate cluster 
(referred to as the _monitoring cluster_) to store the data. Using a separate 
monitoring cluster prevents production cluster outages from impacting your 
ability to access your monitoring data. It also prevents monitoring activities 
from impacting the performance of your production cluster.

For example, specify the {es} output information in the {metricbeat} 
configuration file (`metricbeat.yml`):

[source,yaml]
----------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>
  
  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"
----------------------------------
<1> In this example, the data is stored on a monitoring cluster with nodes 
`es-mon-1` and `es-mon-2`.

If you configured the monitoring cluster to use encrypted communications, you
must access it via HTTPS. For example, use a `hosts` setting like
`https://es-mon-1:9200`.

IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
cluster that stores the monitoring data must have at least one ingest node.

**Elastic security.** The Elastic {security-features} are enabled by default.
You must provide a user ID and password so that {metricbeat} can send metrics
successfully: 

.. Create a user on the monitoring cluster that has the 
`remote_monitoring_agent` {ref}/built-in-roles.html[built-in role]. 
Alternatively, use the `remote_monitoring_user` 
{ref}/built-in-users.html[built-in user]. 
+
TIP: If you're using index lifecycle management, the remote monitoring user
requires additional privileges to create and read indices. For more
information, see `<<feature-roles>>`.

.. Add the `username` and `password` settings to the {es} output information in 
the {metricbeat} configuration file.

For more information about these configuration options, see 
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}] to begin
collecting monitoring data. 

. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}]. 

Your monitoring setup is complete.
