[DOCS] Merge monitoring sections in Elasticsearch book (#46885)

docs/reference/index.asciidoc

@@ -16,8 +16,6 @@ include::getting-started.asciidoc[]
 
 include::setup.asciidoc[]
 
-include::monitoring/configuring-monitoring.asciidoc[]
-
 include::setup/bootstrap-checks-xes.asciidoc[]
 
 include::upgrade.asciidoc[]

docs/reference/monitoring/collecting-monitoring-data.asciidoc

@@ -1,10 +1,7 @@
 [role="xpack"]
 [testenv="gold"]
 [[collecting-monitoring-data]]
-=== Collecting monitoring data
+== Collecting monitoring data
-++++
-<titleabbrev>Collecting monitoring data</titleabbrev>
-++++
 
 If you enable the Elastic {monitor-features} in your cluster, you can 
 optionally collect metrics about {es}. By default, monitoring is enabled but 

docs/reference/monitoring/collectors.asciidoc

@@ -110,7 +110,7 @@ For more information about the configuration options for the collectors, see
 
 [float]
 [[es-monitoring-stack]]
-=== Collecting data from across the Elastic Stack
+==== Collecting data from across the Elastic Stack
 
 {monitoring} in {es} also receives monitoring data from other parts of the
 Elastic Stack. In this way, it serves as an unscheduled monitoring data

docs/reference/monitoring/configuring-filebeat.asciidoc

@@ -1,7 +1,7 @@
 [role="xpack"]
 [testenv="basic"]
 [[configuring-filebeat]]
-=== Collecting {es} log data with {filebeat}
+== Collecting {es} log data with {filebeat}
 
 [subs="attributes"]
 ++++

docs/reference/monitoring/configuring-metricbeat.asciidoc

@@ -1,7 +1,7 @@
 [role="xpack"]
 [testenv="gold"]
 [[configuring-metricbeat]]
-=== Collecting {es} monitoring data with {metricbeat}
+== Collecting {es} monitoring data with {metricbeat}
 
 [subs="attributes"]
 ++++

docs/reference/monitoring/configuring-monitoring.asciidoc

@@ -1,23 +0,0 @@
-[role="xpack"]
-[testenv="gold"]
-[[configuring-monitoring]]
-== Configuring monitoring in {es}
-++++
-<titleabbrev>Configuring monitoring</titleabbrev>
-++++
-
-If you enable the Elastic {monitor-features} in your cluster, there are two 
-methods to collect metrics about {es}:
-
-* <<collecting-monitoring-data>>
-* <<configuring-metricbeat>>
-
-You can also <<configuring-filebeat,use {filebeat} to collect {es} logs>>. 
-
-To learn about monitoring in general, see 
-{stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. 
-
-include::collecting-monitoring-data.asciidoc[]
-include::configuring-metricbeat.asciidoc[]
-include::configuring-filebeat.asciidoc[]
-include::indices.asciidoc[]

docs/reference/monitoring/exporters.asciidoc

@@ -158,6 +158,3 @@ which is used to determine whether the resource should be replaced. The `version
 field value represents the latest version of {monitoring} that changed the 
 resource. If a resource is edited by someone or something external to 
 {monitoring}, those changes are lost the next time an automatic update occurs. 
-
-include::local-export.asciidoc[]
-include::http-export.asciidoc[]

docs/reference/monitoring/how-monitoring-works.asciidoc

@@ -0,0 +1,39 @@
+[role="xpack"]
+[testenv="basic"]
+[[how-monitoring-works]]
+== How monitoring works
+++++
+<titleabbrev>How it works</titleabbrev>
+++++
+
+Each {es} node, {ls} node, {kib} instance, and Beat is considered unique in the
+cluster based on its persistent UUID, which is written to the
+<<path-settings,`path.data`>> directory when the node or instance starts.
+
+Monitoring documents are just ordinary JSON documents built by monitoring each 
+{stack} component at a specified collection interval. If you want to alter the
+templates for these indices, see <<config-monitoring-indices>>.
+
+Each component in the {stack} is responsible for monitoring itself and then 
+forwarding those documents to the production cluster for both routing and 
+indexing (storage). The routing and indexing processes in {es} are handled by 
+what are called <<es-monitoring-collectors,collectors>> and 
+<<es-monitoring-exporters,exporters>>.
+
+Alternatively, you can use {metricbeat} to collect monitoring data and ship it
+directly to the monitoring cluster. 
+
+To learn how to collect monitoring data, see:
+
+* <<collecting-monitoring-data>>
+* <<configuring-metricbeat>>
+* {kibana-ref}/xpack-monitoring.html[Monitoring {kib}]
+* {logstash-ref}/monitoring-logstash.html[Monitoring {ls}]
+* Monitoring Beats:
+** {auditbeat-ref}/monitoring.html[{auditbeat}]
+** {filebeat-ref}/monitoring.html[{filebeat}]
+** {functionbeat-ref}/monitoring.html[{functionbeat}] 
+** {heartbeat-ref}/monitoring.html[{heartbeat}]
+** {metricbeat-ref}/monitoring.html[{metricbeat}]
+** {packetbeat-ref}/monitoring.html[{packetbeat}]
+** {winlogbeat-ref}/monitoring.html[{winlogbeat}]

docs/reference/monitoring/http-export.asciidoc

@@ -1,7 +1,7 @@
 [role="xpack"]
 [testenv="basic"]
 [[http-exporter]]
-=== HTTP Exporters
+=== HTTP exporters
 
 The `http` exporter is the preferred exporter in {monitoring} because it enables 
 the use of a separate monitoring cluster. As a secondary benefit, it avoids 

docs/reference/monitoring/index.asciidoc

@@ -1,54 +1,39 @@
 [role="xpack"]
 [testenv="basic"]
-[[es-monitoring]]
+[[monitor-elasticsearch-cluster]]
-= Monitoring {es}
+= Monitor a cluster
 
 [partintro]
 --
-The Elastic {monitor-features} enable you to easily monitor the health of 
+The {stack} {monitor-features} provide a way to keep a pulse on the health and
-your {es} cluster. The monitoring metrics are collected from each node and 
+performance of your {es} cluster. 
-stored in {es} indices. 
 
-TIP: In production environments, it is recommended to store the monitoring data 
+* <<monitoring-overview>>
-in a separate _monitoring cluster_. See 
+* <<how-monitoring-works>>
-{stack-ov}/monitoring-production.html[Monitoring in a production environment]. 
+* <<collecting-monitoring-data>>
-
+* <<configuring-metricbeat>>
-Each {es} node is considered unique based on its persistent UUID, which is
+* <<configuring-filebeat>>
-written on first start to its <<path-settings,`path.data`>> directory, which
+* <<config-monitoring-indices>>
-defaults to `./data`.
+* <<es-monitoring-collectors>>
-
+* <<es-monitoring-exporters>>
-All settings associated with monitoring in {es} must be set in either the
-`elasticsearch.yml` file for each node or, where possible, in the dynamic
-cluster settings. For more information, see <<configuring-monitoring>>.
-
-[[es-monitoring-overview]]
-{es} is also at the core of monitoring across the {stack}. In all cases, 
-monitoring documents are just ordinary JSON documents built by monitoring each 
-{stack} component at some collection interval, then indexing those 
-documents into the monitoring cluster. 
-
-Each component in the stack is responsible for monitoring itself and then 
-forwarding those documents to the {es} production cluster for both routing and 
-indexing (storage). The routing and indexing processes in {es} are handled by 
-what are called <<es-monitoring-collectors,collectors>> and 
-<<es-monitoring-exporters,exporters>>. 
-
-Alternatively, in 6.4 and later, you can use {metricbeat} to collect 
-monitoring data about {kib} and ship it directly to the monitoring cluster, 
-rather than routing it through the production cluster. In 6.5 and later, you 
-can also use {metricbeat} to collect and ship data about {es}. 
-
-You can view monitoring data from {kib} where it’s easy to spot issues at a
-glance or delve into the system behavior over time to diagnose operational
-issues. In addition to the built-in status warnings, you can also set up custom
-alerts based on the data in the monitoring indices.
-
-For an introduction to monitoring your {stack}, including Beats, {ls}, and {kib}, 
-see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
 
 --
 
-include::collectors.asciidoc[]
+include::overview.asciidoc[]
-include::exporters.asciidoc[]
+
+include::how-monitoring-works.asciidoc[]
+
+include::collecting-monitoring-data.asciidoc[]
 include::pause-export.asciidoc[]
 
+include::configuring-metricbeat.asciidoc[]
+
+include::configuring-filebeat.asciidoc[]
+
+include::indices.asciidoc[]
+
+include::collectors.asciidoc[]
+
+include::exporters.asciidoc[]
+include::local-export.asciidoc[]
+include::http-export.asciidoc[]

docs/reference/monitoring/indices.asciidoc

@@ -1,7 +1,7 @@
 [role="xpack"]
 [testenv="basic"]
 [[config-monitoring-indices]]
-=== Configuring indices for monitoring
+== Configuring indices for monitoring
 
 <<indices-templates,Index templates>> are used to configure the indices
 that store the monitoring data collected from a cluster.

docs/reference/monitoring/local-export.asciidoc

@@ -1,7 +1,7 @@
 [role="xpack"]
 [testenv="basic"]
 [[local-exporter]]
-=== Local Exporters
+=== Local exporters
 
 The `local` exporter is the default exporter in {monitoring}. It routes data 
 back into the same (local) cluster. In other words, it uses the production 
@@ -56,7 +56,7 @@ For more information about the configuration options for the `local` exporter,
 see <<local-exporter-settings>>.
 
 [[local-exporter-cleaner]]
-==== Cleaner Service
+==== Cleaner service
 
 One feature of the `local` exporter, which is not present in the `http` exporter, 
 is a cleaner service. The cleaner service runs once per day at 01:00 AM UTC on 

docs/reference/monitoring/overview.asciidoc

@@ -0,0 +1,39 @@
+[role="xpack"]
+[[monitoring-overview]]
+== Monitoring overview
+++++
+<titleabbrev>Overview</titleabbrev>
+++++
+
+When you monitor a cluster, you collect data from the {es} nodes, {ls} nodes,
+{kib} instances, and Beats in your cluster. You can also
+<<configuring-filebeat,use {filebeat} to collect {es} logs>>. 
+
+All of the monitoring metrics are stored in {es}, which enables you to easily
+visualize the data from {kib}. By default, the monitoring metrics are stored in
+local indices.
+
+TIP: In production, we strongly recommend using a separate monitoring cluster.
+Using a separate monitoring cluster prevents production cluster outages from
+impacting your ability to access your monitoring data. It also prevents
+monitoring activities from impacting the performance of your production cluster.
+For the same reason, we also recommend using a separate {kib} instance for
+viewing the monitoring data.
+
+You can use {metricbeat} to collect and ship data about {es}, {kib}, {ls}, and
+Beats directly to your monitoring cluster rather than routing it through your
+production cluster. The following diagram illustrates a typical monitoring
+architecture with separate production and monitoring clusters:
+
+image::images/architecture.png[A typical monitoring environment]
+
+If you have the appropriate license, you can route data from multiple production
+clusters to a single monitoring cluster. For more information about the
+differences between various subscription levels, see:
+https://www.elastic.co/subscriptions
+
+IMPORTANT: In general, the monitoring cluster and the clusters being monitored
+should be running the same version of the stack. A monitoring cluster cannot
+monitor production clusters running newer versions of the stack. If necessary,
+the monitoring cluster can monitor production clusters running the latest
+release of the previous major version.

docs/reference/monitoring/pause-export.asciidoc

@@ -1,7 +1,7 @@
 [role="xpack"]
 [testenv="basic"]
 [[pause-export]]
-== Pausing Data Collection
+=== Pausing data collection
 
 To stop generating {monitoring} data in {es}, disable data collection:
 

docs/reference/redirects.asciidoc

@@ -909,3 +909,14 @@ See <<transform-source>>.
 ==== Pivot objects
 
 See <<transform-pivot>>.
+
+[role="exclude",id="configuring-monitoring"]
+=== Configuring monitoring
+
+See <<monitoring-overview>>.
+
+[role="exclude",id="es-monitoring"]
+=== Monitoring {es}
+
+See <<monitor-elasticsearch-cluster>>.
+