[role="xpack"]
[[monitoring-metricbeat]]
= Collect {kib} monitoring data with {metricbeat}
[subs="attributes"]
++++
<titleabbrev>Collect monitoring data with {metricbeat}</titleabbrev>
++++

In 6.4 and later, you can use {metricbeat} to collect data about {kib}
and ship it to the monitoring cluster, rather than routing it through the
production cluster as described in <<monitoring-kibana>>.

image::user/monitoring/images/metricbeat.png[Example monitoring architecture]

To learn about monitoring in general, see
{ref}/monitor-elasticsearch-cluster.html[Monitor a cluster].

. Disable the default collection of {kib} monitoring metrics. +
+
--
Add the following setting in the {kib} configuration file (`kibana.yml`):

[source,yaml]
----------------------------------
monitoring.kibana.collection.enabled: false
----------------------------------

Leave the `monitoring.enabled` set to its default value (`true`).
For more information, see
<<monitoring-settings-kb,Monitoring settings in {kib}>>.
--

. <<start-stop,Start {kib}>>.

. Set the `xpack.monitoring.collection.enabled` setting to `true` on
each node in the production cluster. By default, it is disabled (`false`).
+
--
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
node or across the cluster as a dynamic cluster setting. If {es}
{security-features} are enabled, you must have `monitor` cluster privileges to
view the cluster settings and `manage` cluster privileges to change them.

--

** In {kib}:

... Open {kib} in your web browser.
+
--
If you are running {kib} locally, go to `http://localhost:5601/`.

If the Elastic {security-features} are enabled, log in.
--

... In the side navigation, click *Stack Monitoring*. If data collection is
disabled, you are prompted to turn it on.

** From the Console or command line, set `xpack.monitoring.collection.enabled`
to `true` on the production cluster. +
+
--
For example, you can use the following APIs to review and change this setting:

[source,js]
----------------------------------
GET _cluster/settings

PUT _cluster/settings
{
  "persistent": {
    "xpack.monitoring.collection.enabled": true
  }
}
----------------------------------

For more information, see {ref}/monitoring-settings.html[Monitoring settings in {es}]
and {ref}/cluster-update-settings.html[Cluster update settings].
--

. {metricbeat-ref}/metricbeat-installation-configuration.html[Install {metricbeat}] on the
same server as {kib}.

. Enable the {kib} {xpack} module in {metricbeat}. +
+
--
// tag::enable-kibana-module[]
For example, to enable the default configuration in the `modules.d` directory,
run the following command:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
metricbeat modules enable kibana-xpack
----------------------------------------------------------------------

For more information, see
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and
{metricbeat-ref}/metricbeat-module-kibana.html[{kib} module].
// end::enable-kibana-module[]
--

. Configure the {kib} {xpack} module in {metricbeat}. +
+
--
// tag::configure-kibana-module[]
The `modules.d/kibana-xpack.yml` file contains the following settings:

[source,yaml]
----------------------------------
- module: kibana
  metricsets:
    - stats
  period: 10s
  hosts: ["localhost:5601"]
  #basepath: ""
  #username: "user"
  #password: "secret"
  xpack.enabled: true
----------------------------------

By default, the module collects {kib} monitoring metrics from `localhost:5601`.
If that host and port number are not correct, you must update the `hosts`
setting. If you configured {kib} to use encrypted communications, you must
access it via HTTPS. For example, use a `hosts` setting like
`https://localhost:5601`.
// end::configure-kibana-module[]

// tag::remote-monitoring-user[]
If the Elastic {security-features} are enabled, you must also provide a user
ID and password so that {metricbeat} can collect metrics successfully:

.. Create a user on the production cluster that has the
`remote_monitoring_collector` {ref}/built-in-roles.html[built-in role].
Alternatively, use the `remote_monitoring_user`
{ref}/built-in-users.html[built-in user].

.. Add the `username` and `password` settings to the {kib} module configuration
file.
// end::remote-monitoring-user[]
--

. Optional: Disable the system module in {metricbeat}.
+
--
// tag::disable-system-module[]
By default, the {metricbeat-ref}/metricbeat-module-system.html[system module] is
enabled. The information it collects, however, is not shown on the *Monitoring*
page in {kib}. Unless you want to use that information for other purposes, run
the following command:

["source","sh",subs="attributes,callouts"]
----------------------------------------------------------------------
metricbeat modules disable system
----------------------------------------------------------------------
// end::disable-system-module[]
--

. Identify where to send the monitoring data. +
+
--
TIP: In production environments, we strongly recommend using a separate cluster
(referred to as the _monitoring cluster_) to store the data. Using a separate
monitoring cluster prevents production cluster outages from impacting your
ability to access your monitoring data. It also prevents monitoring activities
from impacting the performance of your production cluster.

For example, specify the {es} output information in the {metricbeat}
configuration file (`metricbeat.yml`):

[source,yaml]
----------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #username: "elastic"
  #password: "changeme"
----------------------------------
<1> In this example, the data is stored on a monitoring cluster with nodes
`es-mon-1` and `es-mon-2`.

If you configured the monitoring cluster to use encrypted communications, you
must access it via HTTPS. For example, use a `hosts` setting like
`https://es-mon-1:9200`.

IMPORTANT: The {es} {monitor-features} use ingest pipelines. The
cluster that stores the monitoring data must have at least one node with the
`ingest` role.

If the {es} {security-features} are enabled on the monitoring cluster, you
must provide a valid user ID and password so that {metricbeat} can send metrics
successfully:

.. Create a user on the monitoring cluster that has the
`remote_monitoring_agent` {ref}/built-in-roles.html[built-in role].
Alternatively, use the `remote_monitoring_user`
{ref}/built-in-users.html[built-in user].

.. Add the `username` and `password` settings to the {es} output information in
the {metricbeat} configuration file.

For more information about these configuration options, see
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}].

. <<monitoring-data,View the monitoring data in {kib}>>.
