github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 01:38:56 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Stack Monitoring] A writeup on SM indices (#127844)

Browse source 
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Mat Schaffer 2022-03-18 09:34:46 +09:00 • committed by GitHub
parent 4fe1f0744e
commit 11b87e3558
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 41 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

40
x-pack/plugins/monitoring/dev_docs/reference/indices.md Normal file
View file

@ -0,0 +1,40 @@
This document provides a general overview of the indices used by the Stack Monitoring UI.
## Index Patterns
Stack Monitoring indices will generally fall into to `.monitoring-*` index pattern with some variation depending on collection mode.
When using [internal collection](data_collection_modes.md#internal-collection), or standalone metricbeat collection prior to 8.0, data will be stored in dated raw indices following the pattern:
- `.monitoring-(component)-7-(date)`
When using [standalone metricbeat collection](data_collection_modes.md#standalone-metricbeat-collection) after 8.0, data will be stored in a data stream with the following pattern:
- `.monitoring-(component)-8-mb`
> Note that the distinction between 7 and 8 is the **data schema** version. These numbers are not expected to track stack release versions.
With the change to [package-driven collection](data_collection_modes.md#package-driven-collection), data will be stored in data streams under the `metrics-*` index pattern.
Additionally, in version 8.0 only, Enterprise Search metrics are published to the `metricbeat-*` index pattern. This was a bug and corrected in 8.1 ([kibana-pr2981](https://github.com/elastic/beats/pull/29871)).
And finally if using the standalone metricbeat modules with `xpack.enabled: false`, they will write data into the `metricbeat-*` index pattern. Stack Monitoring UI was never adapted to read this data, so it would only be available for custom dashboards.
## Mapping sources
The index templates for `.monitoring-*` are shipped with and managed by Elasticsearch itself and can be found in that code repository. For example:
- [monitoring-es.json](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/resources/monitoring-es.json) - for internal collection or standalone metricbeat prior to 8.0
- [monitoring-es-mb.json](https://github.com/elastic/elasticsearch/blob/master/x-pack/plugin/core/src/main/resources/monitoring-es-mb.json) - for standalone metricbeat after 8.0
To verify changes to these templates, either make them in place on a running cluster or run elasticsearch from source.
The `metrics-*` and `metricbeat-*` mappings are managed by metricbeat and elastic agent, **code locations TBD**.
## Aliasing
In order for the Stack Monitoring UI to function across multiple data schemas, aliases are added to newer indices to allow the original queries to continue to work.
For example, see [the alias for es cluster_uuid](https://github.com/elastic/elasticsearch/blob/91379ea21e7d987272ba49e385e74ec55a904d84/x-pack/plugin/core/src/main/resources/monitoring-es-mb.json#L2061-L2064).
In some cases aliases aren't sufficient due to the required query pattern. In these cases we add UI logic to accommodate both old and new schemas.

1
x-pack/plugins/monitoring/readme.md
View file

@ -13,6 +13,7 @@ This plugin provides the Stack Monitoring kibana application.
- [Terminology](dev_docs/reference/terminology.md)
- [Data Collection modes](dev_docs/reference/data_collection_modes.md)
- [Rules and Alerts](dev_docs/reference/rules_alerts.md)
- [Indices](dev_docs/reference/indices.md)
## Tooling
- [Debugging logging](dev_docs/how_to/debug_logging.md)