mirror of
https://github.com/elastic/kibana.git
synced 2025-06-27 18:51:07 -04:00
935f904f32
## Summary
Adds note to the [Format data
fields](https://www.elastic.co/guide/en/kibana/current/managing-data-views.html#managing-fields)
section that default formatters use the [`meta.unit`
field](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-field-meta.html)
for displaying time units, bytes, and percentages.
Relates to: #174973
294 lines
10 KiB
Text
294 lines
10 KiB
Text
[[managing-data-views]]
|
|
== Manage data views
|
|
|
|
:frontmatter-description: Conceptual and step-by-step procedures for using runtime fields, scripted fields, and field formatters.
|
|
:frontmatter-tags-products: [kibana]
|
|
|
|
To customize the data fields in your data view,
|
|
you can add runtime fields to the existing documents,
|
|
add scripted fields to compute data on the fly, and change how {kib} displays the data fields.
|
|
|
|
[float]
|
|
[[runtime-fields]]
|
|
=== Explore your data with runtime fields
|
|
|
|
Runtime fields are fields that you add to documents after you've ingested your data, and are evaluated at query time. With runtime fields, you allow for a smaller index and faster ingest time so that you can use less resources and reduce your operating costs.
|
|
You can use runtime fields anywhere data views are used, for example, you can explore runtime fields in *Discover* and create visualizations with runtime fields for your dashboard.
|
|
|
|
With runtime fields, you can:
|
|
|
|
* Define fields for a specific use case without modifying the underlying schema.
|
|
|
|
* Override the returned values from index fields.
|
|
|
|
* Start working on your data without understanding the structure.
|
|
|
|
* Add fields to existing documents without reindexing your data.
|
|
|
|
WARNING: Runtime fields can impact {kib} performance. When you run a query, {es} uses the fields you index first to shorten the response time.
|
|
Index the fields that you commonly search for and filter on, such as `timestamp`, then use runtime fields to limit the number of fields {es} uses to calculate values.
|
|
|
|
For detailed information on how to use runtime fields with {es}, refer to {ref}/runtime.html[Runtime fields].
|
|
|
|
[float]
|
|
[[create-runtime-fields]]
|
|
==== Add runtime fields
|
|
|
|
To add runtime fields to your data views, open the data view you want to change,
|
|
then define the field values by emitting a single value using
|
|
the {ref}/modules-scripting-painless.html[Painless scripting language].
|
|
You can also add runtime fields in <<add-field-in-discover,*Discover*>> and <<change-the-fields,*Lens*>>.
|
|
|
|
. Open the main menu, then click *Stack Management > Data Views*.
|
|
|
|
. Select the data view that you want to add the runtime field to, then click *Add field*.
|
|
|
|
. Enter the field *Name*, then select the *Type*.
|
|
|
|
. Select *Set custom label*, then enter the label you want to display where the data view is used,
|
|
such as *Discover*.
|
|
|
|
. Select *Set value*, then define the script. The script must match the *Type*, or the data view fails anywhere it is used.
|
|
|
|
. To help you define the script, use the *Preview*:
|
|
|
|
* To view the other available fields, use the *Document ID* arrows.
|
|
|
|
* To filter the fields list, enter the keyword in *Filter fields*.
|
|
|
|
* To pin frequently used fields to the top of the list, hover over the field,
|
|
then click image:images/stackManagement-indexPatterns-pinRuntimeField-7.15.png[Icon to pin field to the top of the list].
|
|
|
|
. Click *Create field*.
|
|
|
|
[float]
|
|
[[runtime-field-examples]]
|
|
==== Runtime field examples
|
|
|
|
Try the runtime field examples on your own using the <<gs-get-data-into-kibana,*Sample web logs*>> data.
|
|
|
|
[float]
|
|
[[simple-hello-world-example]]
|
|
==== Return a keyword value
|
|
|
|
Return `Hello World!`:
|
|
|
|
[source,text]
|
|
----
|
|
emit("Hello World!");
|
|
----
|
|
|
|
image:management/images/runtime_field.png[Runtime field with keyword type]
|
|
|
|
[float]
|
|
[[perform-a-calculation-on-a-single-field]]
|
|
==== Perform a calculation on a single field
|
|
|
|
Calculate kilobytes from bytes:
|
|
|
|
[source,text]
|
|
----
|
|
emit(doc['bytes'].value / 1024)
|
|
----
|
|
|
|
[float]
|
|
[[return-substring]]
|
|
==== Return a substring
|
|
|
|
Return the string that appears after the last slash in the URL:
|
|
|
|
[source,text]
|
|
----
|
|
def path = doc["url.keyword"].value;
|
|
if (path != null) {
|
|
int lastSlashIndex = path.lastIndexOf('/');
|
|
if (lastSlashIndex > 0) {
|
|
emit(path.substring(lastSlashIndex+1));
|
|
return;
|
|
}
|
|
}
|
|
emit("");
|
|
----
|
|
|
|
[float]
|
|
[[composite-runtime-field]]
|
|
==== Return multiple fields with a composite runtime field
|
|
|
|
A single runtime field can also produce multiple subfields when the type `Composite` is selected. The script editor provides default types that can be customized for each subfields.
|
|
|
|
Return `keyword` and `double` type subfields. Note that the first argument for `emit` is the name of the subfield.
|
|
|
|
[source,text]
|
|
----
|
|
emit('subfield_a', 'Hello');
|
|
emit('subfield_b', 42);
|
|
----
|
|
|
|
image:management/images/runtime_field_composite.png[Runtime field with composite type]
|
|
|
|
[float]
|
|
[[replace-nulls-with-blanks]]
|
|
==== Replace nulls with blanks
|
|
Replace `null` values with `None`:
|
|
|
|
[source,text]
|
|
----
|
|
def source = doc['referer'].value;
|
|
if (source != null) {
|
|
emit(source);
|
|
return;
|
|
}
|
|
else {
|
|
emit("None");
|
|
}
|
|
----
|
|
|
|
Specify the operating system condition:
|
|
|
|
[source,text]
|
|
----
|
|
def source = doc['machine.os.keyword'].value;
|
|
if (source != "") {
|
|
emit(source);
|
|
}
|
|
else {
|
|
emit("None");
|
|
}
|
|
----
|
|
|
|
[float]
|
|
[[manage-runtime-fields]]
|
|
==== Manage runtime fields
|
|
|
|
Edit the settings for runtime fields, or remove runtime fields from data views.
|
|
|
|
. Open the main menu, then click *Stack Management > Data Views*.
|
|
|
|
. Select the data view that contains the runtime field you want to manage, then open the runtime field edit options or delete the runtime field.
|
|
|
|
[float]
|
|
[[scripted-fields]]
|
|
=== Add scripted fields to data views
|
|
|
|
deprecated::[7.13,Use {ref}/runtime.html[runtime fields] instead of scripted fields. Runtime fields support Painless scripts and provide greater flexibility.]
|
|
|
|
Scripted fields compute data on the fly from the data in your {es} indices. The data is shown on
|
|
the Discover tab as part of the document data, and you can use scripted fields in your visualizations. You query scripted fields with the <<kuery-query, {kib} query language>>, and can filter them using the filter bar. The scripted field values are computed at query time, so they aren't indexed and cannot be searched using the {kib} default
|
|
query language.
|
|
|
|
WARNING: Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on
|
|
{kib} performance. Keep in mind that there's no built-in validation of a scripted field. If your scripts are
|
|
buggy, you'll get exceptions whenever you try to view the dynamically generated data.
|
|
|
|
When you define a scripted field in {kib}, you have a choice of the {ref}/modules-scripting-expression.html[Lucene expressions] or the
|
|
{ref}/modules-scripting-painless.html[Painless] scripting language.
|
|
|
|
You can reference any single value numeric field in your expressions, for example:
|
|
|
|
----
|
|
doc['field_name'].value
|
|
----
|
|
|
|
For more information on scripted fields and additional examples, refer to
|
|
https://www.elastic.co/blog/using-painless-kibana-scripted-fields[Using Painless in {kib} scripted fields]
|
|
|
|
[float]
|
|
[[create-scripted-field]]
|
|
==== Create scripted fields
|
|
|
|
Create and add scripted fields to your data views.
|
|
|
|
. Open the main menu, then click *Stack Management > Data Views*.
|
|
|
|
. Select the data view you want to add a scripted field to.
|
|
|
|
. Select the *Scripted fields* tab, then click *Add scripted field*.
|
|
|
|
. Enter a *Name* for the scripted field, then enter the *Script* you want to use to compute a value on the fly from your index data.
|
|
|
|
. Click *Create field*.
|
|
|
|
For more information about scripted fields in {es}, refer to {ref}/modules-scripting.html[Scripting].
|
|
|
|
[float]
|
|
[[update-scripted-field]]
|
|
==== Manage scripted fields
|
|
|
|
. Open the main menu, then click *Stack Management > Data Views*.
|
|
|
|
. Select the data view that contains the scripted field you want to manage.
|
|
|
|
. Select the *Scripted fields* tab, then open the scripted field edit options or delete the scripted field.
|
|
|
|
WARNING: Built-in validation is unsupported for scripted fields. When your scripts contain errors, you receive
|
|
exceptions when you view the dynamically generated data.
|
|
|
|
[float]
|
|
[[managing-fields]]
|
|
=== Format data fields
|
|
|
|
{kib} uses the same field types as {es}, however, some {es} field types are unsupported in {kib}.
|
|
To customize how {kib} displays data fields, use the formatting options.
|
|
|
|
. Open the main menu, then click *Stack Management > Data Views*.
|
|
|
|
. Click the data view that contains the field you want to change.
|
|
|
|
. Find the field, then open the edit options (image:management/index-patterns/images/edit_icon.png[Data field edit icon]).
|
|
|
|
. Select *Set custom label*, then enter a *Custom label* for the field.
|
|
|
|
. Select *Set format*, then enter the *Format* for the field.
|
|
|
|
NOTE: For numeric fields the default field formatters are based on the `meta.unit` field. The unit is associated with a {ref}/api-conventions.html#time-units[time unit], percent, or byte. The convention for percents is to use value 1 to mean 100%.
|
|
|
|
[float]
|
|
[[string-field-formatters]]
|
|
==== String field formatters
|
|
|
|
String fields support *String* and *Url* formatters.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-date]]
|
|
==== Date field formatters
|
|
|
|
Date fields support *Date*, *String*, and *Url* formatters.
|
|
|
|
The *Date* formatter enables you to choose the display format of date stamps using the https://momentjs.com/[moment.js]
|
|
standard format definitions.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-geopoint]]
|
|
==== Geographic point field formatters
|
|
|
|
Geographic point fields support the *String* formatter.
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
[float]
|
|
[[field-formatters-numeric]]
|
|
==== Number field formatters
|
|
|
|
Numeric fields support *Bytes*, *Color*, *Duration*, *Histogram*, *Number*, *Percentage*, *String*, and *Url* formatters.
|
|
|
|
The *Bytes*, *Number*, and *Percentage* formatters enable you to choose the display formats of numbers in the field using
|
|
the <<numeral, Elastic numeral pattern>> syntax that {kib} maintains.
|
|
|
|
The *Histogram* formatter is used only for the {ref}/histogram.html[histogram field type]. When you use the *Histogram* formatter,
|
|
you can apply the *Bytes*, *Number*, or *Percentage* format to aggregated data.
|
|
|
|
include::field-formatters/url-formatter.asciidoc[]
|
|
|
|
include::field-formatters/string-formatter.asciidoc[]
|
|
|
|
include::field-formatters/duration-formatter.asciidoc[]
|
|
|
|
include::field-formatters/color-formatter.asciidoc[]
|