kibana/docs/management/manage-data-views.asciidoc

[[managing-data-views]]
== Manage data views
:keywords: administrator, data view, data views, management, runtime fields, runtime fields in Kibana, scripted fields, field formatters, data fields, index pattern, index patterns
:description: Conceptual and step-by-step procedures for using runtime fields, scripted fields, and field formatters.

To customize the data fields in your data view,
you can add runtime fields to the existing documents,
add scripted fields to compute data on the fly, and change how {kib} displays the data fields.

[float]
[[runtime-fields]]
=== Explore your data with runtime fields

Runtime fields are fields that you add to documents after you've ingested your data, and are evaluated at query time. With runtime fields, you allow for a smaller index and faster ingest time so that you can use less resources and reduce your operating costs.
You can use runtime fields anywhere data views are used, for example, you can explore runtime fields in *Discover* and create visualizations with runtime fields for your dashboard.

With runtime fields, you can:

* Define fields for a specific use case without modifying the underlying schema.

* Override the returned values from index fields.

* Start working on your data without understanding the structure.

* Add fields to existing documents without reindexing your data.

WARNING: Runtime fields can impact {kib} performance. When you run a query, {es} uses the fields you index first to shorten the response time.
Index the fields that you commonly search for and filter on, such as `timestamp`, then use runtime fields to limit the number of fields {es} uses to calculate values.

For detailed information on how to use runtime fields with {es}, refer to {ref}/runtime.html[Runtime fields].

[float]
[[create-runtime-fields]]
==== Add runtime fields

To add runtime fields to your data views, open the data view you want to change,
then define the field values by emitting a single value using
the {ref}/modules-scripting-painless.html[Painless scripting language].
You can also add runtime fields in <<add-field-in-discover,*Discover*>> and <<change-the-fields,*Lens*>>.

. Open the main menu, then click *Stack Management > Data Views*.

. Select the data view that you want to add the runtime field to, then click *Add field*.

. Enter the field *Name*, then select the *Type*.

. Select *Set custom label*, then enter the label you want to display where the data view is used,
such as *Discover*.

. Select *Set value*, then define the script. The script must match the *Type*, or the data view fails anywhere it is used.

. To help you define the script, use the *Preview*:

* To view the other available fields, use the *Document ID* arrows.

* To filter the fields list, enter the keyword in *Filter fields*.

* To pin frequently used fields to the top of the list, hover over the field,
then click image:images/stackManagement-indexPatterns-pinRuntimeField-7.15.png[Icon to pin field to the top of the list].

. Click *Create field*.

[float]
[[runtime-field-examples]]
==== Runtime field examples

Try the runtime field examples on your own using the <<gs-get-data-into-kibana,*Sample web logs*>> data.

[float]
[[simple-hello-world-example]]
==== Return a keyword value

Return `Hello World!`:

[source,text]
----
emit("Hello World!");
----

image:management/images/runtime_field.png[Runtime field with keyword type]

[float]
[[perform-a-calculation-on-a-single-field]]
==== Perform a calculation on a single field

Calculate kilobytes from bytes:

[source,text]
----
emit(doc['bytes'].value / 1024)
----

[float]
[[return-substring]]
==== Return a substring

Return the string that appears after the last slash in the URL:

[source,text]
----
def path = doc["url.keyword"].value;
if (path != null) {
    int lastSlashIndex = path.lastIndexOf('/');
    if (lastSlashIndex > 0) {
        emit(path.substring(lastSlashIndex+1));
    return;
    }
}
emit("");
----

[float]
[[composite-runtime-field]]
==== Return multiple fields with a composite runtime field

A single runtime field can also produce multiple subfields when the type `Composite` is selected. The script editor provides default types that can be customized for each subfields.

Return `keyword` and `double` type subfields. Note that the first argument for `emit` is the name of the subfield.

[source,text]
----
emit('subfield_a', 'Hello');
emit('subfield_b', 42);
----

image:management/images/runtime_field_composite.png[Runtime field with composite type]

[float]
[[replace-nulls-with-blanks]]
==== Replace nulls with blanks
Replace `null` values with `None`:

[source,text]
----
def source = doc['referer'].value;
if (source != null) {
	emit(source);
	return;
}
else {
	emit("None");
}
----

Specify the operating system condition:

[source,text]
----
def source = doc['machine.os.keyword'].value;
if (source != "") {
	emit(source);
}
else {
	emit("None");
}
----

[float]
[[manage-runtime-fields]]
==== Manage runtime fields

Edit the settings for runtime fields, or remove runtime fields from data views.

. Open the main menu, then click *Stack Management > Data Views*.

. Select the data view that contains the runtime field you want to manage, then open the runtime field edit options or delete the runtime field.

[float]
[[scripted-fields]]
=== Add scripted fields to data views

deprecated::[7.13,Use {ref}/runtime.html[runtime fields] instead of scripted fields. Runtime fields support Painless scripts and provide greater flexibility.]

Scripted fields compute data on the fly from the data in your {es} indices. The data is shown on
the Discover tab as part of the document data, and you can use scripted fields in your visualizations. You query scripted fields with the <<kuery-query, {kib} query language>>, and can filter them using the filter bar. The scripted field values are computed at query time, so they aren't indexed and cannot be searched using the {kib} default
query language.

WARNING: Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on
{kib} performance. Keep in mind that there's no built-in validation of a scripted field. If your scripts are
buggy, you'll get exceptions whenever you try to view the dynamically generated data.

When you define a scripted field in {kib}, you have a choice of the {ref}/modules-scripting-expression.html[Lucene expressions] or the
{ref}/modules-scripting-painless.html[Painless] scripting language.

You can reference any single value numeric field in your expressions, for example:

----
doc['field_name'].value
----

For more information on scripted fields and additional examples, refer to
https://www.elastic.co/blog/using-painless-kibana-scripted-fields[Using Painless in {kib} scripted fields]

[float]
[[create-scripted-field]]
==== Create scripted fields

Create and add scripted fields to your data views.

. Open the main menu, then click *Stack Management > Data Views*.

. Select the data view you want to add a scripted field to.

. Select the *Scripted fields* tab, then click *Add scripted field*.

. Enter a *Name* for the scripted field, then enter the *Script* you want to use to compute a value on the fly from your index data.

. Click *Create field*.

For more information about scripted fields in {es}, refer to {ref}/modules-scripting.html[Scripting].

[float]
[[update-scripted-field]]
==== Manage scripted fields

. Open the main menu, then click *Stack Management > Data Views*.

. Select the data view that contains the scripted field you want to manage.

. Select the *Scripted fields* tab, then open the scripted field edit options or delete the scripted field.

WARNING: Built-in validation is unsupported for scripted fields. When your scripts contain errors, you receive
exceptions when you view the dynamically generated data.

[float]
[[managing-fields]]
=== Format data fields

{kib} uses the same field types as {es}, however, some {es} field types are unsupported in {kib}.
To customize how {kib} displays data fields, use the formatting options.

. Open the main menu, then click *Stack Management > Data Views*.

. Click the data view that contains the field you want to change.

. Find the field, then open the edit options (image:management/index-patterns/images/edit_icon.png[Data field edit icon]).

. Select *Set custom label*, then enter a *Custom label* for the field.

. Select *Set format*, then enter the *Format* for the field.

[float]
[[string-field-formatters]]
==== String field formatters

String fields support *String* and *Url* formatters.

include::field-formatters/string-formatter.asciidoc[]

include::field-formatters/url-formatter.asciidoc[]

[float]
[[field-formatters-date]]
==== Date field formatters

Date fields support *Date*, *String*, and *Url* formatters.

The *Date* formatter enables you to choose the display format of date stamps using the https://momentjs.com/[moment.js]
standard format definitions.

include::field-formatters/string-formatter.asciidoc[]

include::field-formatters/url-formatter.asciidoc[]

[float]
[[field-formatters-geopoint]]
==== Geographic point field formatters

Geographic point fields support the *String* formatter.

include::field-formatters/string-formatter.asciidoc[]

[float]
[[field-formatters-numeric]]
==== Number field formatters

Numeric fields support *Bytes*, *Color*, *Duration*, *Histogram*, *Number*, *Percentage*, *String*, and *Url* formatters.

The *Bytes*, *Number*, and *Percentage* formatters enable you to choose the display formats of numbers in the field using
the <<numeral, Elastic numeral pattern>> syntax that {kib} maintains.

The *Histogram* formatter is used only for the {ref}/histogram.html[histogram field type]. When you use the *Histogram* formatter,
you can apply the *Bytes*, *Number*, or *Percentage* format to aggregated data.

include::field-formatters/url-formatter.asciidoc[]

include::field-formatters/string-formatter.asciidoc[]

include::field-formatters/duration-formatter.asciidoc[]

include::field-formatters/color-formatter.asciidoc[]