github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-23 17:28:26 -04:00
Code Issues Projects Releases Packages Wiki Activity

[DOCS]Update rollup tutorial to add steps for index pattern (#67377) (#67667)

Browse source 
* [DOCS]Update rollup tutorial to add steps for index pattern

* Make edits suggested by reviewers
# Conflicts:
#	docs/management/rollups/create_and_manage_rollups.asciidoc
This commit is contained in:
Melori Arellano 2020-05-28 12:39:57 -06:00 • committed by GitHub
parent 131368a621
commit c38c4c994f
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 56 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

BIN
docs/images/management-create-rollup-bar-chart.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 265 KiB

BIN
docs/images/management-rollup-index-pattern.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 124 KiB

99
docs/management/rollups/create_and_manage_rollups.asciidoc
View file

@ -3,12 +3,12 @@
== Rollup jobs
A rollup job is a periodic task that aggregates data from indices specified
by an index pattern and rolls it into a new index. Rollup indices are a good way to
compactly store months or years of historical
A rollup job is a periodic task that aggregates data from indices specified
by an index pattern and rolls it into a new index. Rollup indices are a good way to
compactly store months or years of historical
data for use in visualizations and reports.
You’ll find *Rollup Jobs* under *Management > Elasticsearch*. With this UI,
You’ll find *Rollup Jobs* under *Management > Elasticsearch*. With this UI,
you can:
* <<create-and-manage-rollup-job, Create a rollup job>>
@ -17,22 +17,22 @@ you can:
[role="screenshot"]
image::images/management_rollup_list.png[][List of currently active rollup jobs]
Before using this feature, you should be familiar with how rollups work.
{ref}/xpack-rollup.html[Rolling up historical data] is a good source for more detailed information.
Before using this feature, you should be familiar with how rollups work.
{ref}/xpack-rollup.html[Rolling up historical data] is a good source for more detailed information.
[float]
[[create-and-manage-rollup-job]]
=== Create a rollup job
{kib} makes it easy for you to create a rollup job by walking you through
the process. You fill in the name, data flow, and how often you want to roll
up the data. Then you define a date histogram aggregation for the rollup job
and optionally terms, histogram, and metrics aggregations.
{kib} makes it easy for you to create a rollup job by walking you through
the process. You fill in the name, data flow, and how often you want to roll
up the data. Then you define a date histogram aggregation for the rollup job
and optionally terms, histogram, and metrics aggregations.
When defining the index pattern, you must enter a name that is different than
the output rollup index. Otherwise, the job
will attempt to capture the data in the rollup index. For example, if your index pattern is `metricbeat-*`,
you can name your rollup index `rollup-metricbeat`, but not `metricbeat-rollup`.
When defining the index pattern, you must enter a name that is different than
the output rollup index. Otherwise, the job
will attempt to capture the data in the rollup index. For example, if your index pattern is `metricbeat-*`,
you can name your rollup index `rollup-metricbeat`, but not `metricbeat-rollup`.
[role="screenshot"]
image::images/management_create_rollup_job.png[][Wizard that walks you through creation of a rollup job]
@ -41,38 +41,38 @@ image::images/management_create_rollup_job.png[][Wizard that walks you through c
[[manage-rollup-job]]
=== Start, stop, and delete rollup jobs
Once you’ve saved a rollup job, you’ll see it the *Rollup Jobs* overview page,
where you can drill down for further investigation. The *Manage* menu in
Once you’ve saved a rollup job, you’ll see it the *Rollup Jobs* overview page,
where you can drill down for further investigation. The *Manage* menu in
the lower right enables you to start, stop, and delete the rollup job.
You must first stop a rollup job before deleting it.
[role="screenshot"]
image::images/management_rollup_job_details.png[][Rollup job details]
You can’t change a rollup job after you’ve created it. To select additional fields
or redefine terms, you must delete the existing job, and then create a new one
with the updated specifications. Be sure to use a different name for the new rollup
job&mdash;reusing the same name can lead to problems with mismatched job configurations.
You can read more at {ref}/rollup-job-config.html[rollup job configuration].
You can’t change a rollup job after you’ve created it. To select additional fields
or redefine terms, you must delete the existing job, and then create a new one
with the updated specifications. Be sure to use a different name for the new rollup
job&mdash;reusing the same name can lead to problems with mismatched job configurations.
You can read more at {ref}/rollup-job-config.html[rollup job configuration].
[float]
=== Try it: Create and visualize rolled up data
This example creates a rollup job to capture log data from sample web logs.
This example creates a rollup job to capture log data from sample web logs.
To follow along, add the <<add-sample-data, sample web logs data set>>.
In this example, you want data that is older than 7 days in the target index pattern `kibana_sample_data_logs`
to roll up once a day into the index `rollup_logstash`. You’ll bucket the
rolled up data on an hourly basis, using 60m for the time bucket configuration.
to roll up once a day into the index `rollup_logstash`. You’ll bucket the
rolled up data on an hourly basis, using 60m for the time bucket configuration.
This allows for more granular queries, such as 2h and 12h.
[float]
==== Create the rollup job
As you walk through the *Create rollup job* UI, enter the data shown in
the table below. The terms, histogram, and metrics fields reflect
the key information to retain in the rolled up data: where visitors are from (geo.src),
what operating system they are using (machine.os.keyword),
As you walk through the *Create rollup job* UI, enter the data shown in
the table below. The terms, histogram, and metrics fields reflect
the key information to retain in the rolled up data: where visitors are from (geo.src),
what operating system they are using (machine.os.keyword),
and how much data is being sent (bytes).
|===
@ -118,31 +118,44 @@ and how much data is being sent (bytes).
|===
You can now use the rolled up data for analysis at a fraction of the storage cost
of the original index. The original data can live side by side with the new
You can now use the rolled up data for analysis at a fraction of the storage cost
of the original index. The original data can live side by side with the new
rollup index, or you can remove or archive it using <<creating-index-lifecycle-policies,Index Lifecycle Management>>.
[float]
==== Visualize the rolled up data
Your next step is to visualize your rolled up data in a vertical bar chart.
Your next step is to visualize your rolled up data in a vertical bar chart.
Most visualizations support rolled up data, with the exception of Timelion, TSVB, and Vega visualizations.
Using the information from the example rollup configuration described above,
you can use `rollup_logstash` to match the rolled up index pattern,
and `kibana_sample_data_logs` to match the index pattern for raw data.
The notation for a combination index pattern with both raw and rolled up data
is `rollup_logstash,kibana_sample_data_logs`.
. Create the rollup index pattern in *Management > Index Patterns* so you can
select your rolled up data for visualizations. Click *Create index pattern*, and select *Rollup index pattern* from the dropdown.
+
[role="screenshot"]
image::images/management-rollup-index-pattern.png[][Create rollup index pattern]
. Enter *rollup_logstash,kibana_sample_logs* as your *Index Pattern* and `@timestamp`
as the *Time Filter field name*.
+
The notation for a combination index pattern with both raw and rolled up data
is `rollup_logstash,kibana_sample_data_logs`. In this index pattern, `rollup_logstash`
matches the rolled up index pattern and `kibana_sample_data_logs` matches the index
pattern for raw data.
. Go to *Visualize* and create a vertical bar chart. Choose `rollup_logstash,kibana_sample_data_logs`
as your source to see both the raw and rolled up data.
+
[role="screenshot"]
image::images/management-create-rollup-bar-chart.png[][Create visualization of rolled up data]
. Look at the data in your visualization.
+
[role="screenshot"]
image::images/management_rollup_job_vis.png[][Visualization of rolled up data]
You can then create a dashboard that contains visualizations of the rolled up
data, raw data, or both. See <<visualize-rollup-data, Using rolled up data in a visualization>>
for more information.
. Optionally, create a dashboard that contains visualizations of the rolled up
data, raw data, or both.
+
[role="screenshot"]
image::images/management_rollup_job_dashboard.png[][Dashboard with rolled up data]