[[managing-index-patterns]]
== Manage index pattern data fields

To customize the data fields in your index pattern, you can add runtime fields to the existing documents, add scrited fields to compute data on the fly, and change how {kib} displays the data fields. 

[float]
[[runtime-fields]]
=== Explore your data with runtime fields 

Runtime fields are fields that you add to documents after you've ingested, and are evaluated at query time. With runtime fields, you allow for a smaller index and faster ingest time so that you can use less resources and reduce your operating costs. You can use runtime fields anywhere index patterns are used.

When you use runtime fields, you can:

* Define fields for a specific use without modifying the underlying schema. 

* Override the returned values from index fields.

* Start working on your data without first understanding the structure. 

* Add fields to existing documents without reindexing your data. 

* Explore runtime field data in *Discover*.

* Create visualizations with runtime field data using *Lens*, *Maps*, and *TSVB*.

WARNING: Runtime fields can impact {kib} performance. When you run a query, {es} uses the fields you index first to shorten the response time. 
Index the fields that you commonly search for and filter on, such as `timestamp`, then use runtime fields to limit the number of fields {es} uses to calculate values. 

For more information, refer to {ref}/runtime.html[Runtime fields].

[float]
[[create-runtime-fields]]
==== Create runtime fields

Create runtime fields in your index patterns, or create runtime fields in *Discover* and *Lens*. 

. Open the main menu, then click *Stack Management > Index Patterns*.

. Select the index pattern you want to add the runtime field to, then click *Add field*.

. Enter a *Name* for the runtime field, then select the field *Type*.

. Select *Set value*, then define the field value by emitting a single value using the {ref}/modules-scripting-painless.html[Painless scripting language].
+
The script must match the field *Type*, or the script fails.

. Click *Create field*.
+
For information on how to create runtime fields in *Discover*, refer to <<add-field-in-discover,Add a field>>.
+
For information on how to create runtime fields in *Lens*, refer to <<change-the-fields,Change the fields list>>.

[float]
[[runtime-field-examples]]
==== Runtime field examples

Try the runtime field examples on your own using the *Sample web logs* data index pattern. 

[float]
[[simple-hello-world-example]]
==== Return a keyword value

To return `Hello World!` value:

[source,text]
----
emit("Hello World!");
----

[float]
[[perform-a-calculation-on-a-single-field]]
===== Perform a calculation on a single field

Calculate kilobytes from bytes:

[source,text]
----
emit(doc['bytes'].value / 1024)
----

[float]
[[return-substring]]
===== Return a substring

Return the string that appears after the last slash in the URL:

[source,text]
----
def path = doc["url.keyword"].value;
if (path != null) {
    int lastSlashIndex = path.lastIndexOf('/');
    if (lastSlashIndex > 0) {
        emit(path.substring(lastSlashIndex+1));
    return;
    }
}
emit("");
----

[float]
[[replace-nulls-with-blanks]]
===== Replace nulls with blanks

Replace null values with none values:

[source,text]
----
def source = doc['referer'].value;
if (source != null) {
	emit(source);
	return;
}
else { 
	emit("None");
}
----

Specify operating system condition:

[source,text]
----
def source = doc['machine.os.keyword'].value;
if (source != "") {
	emit(source);
}
else { 
	emit("None");
}
----

[float]
[[manage-runtime-fields]]
==== Manage runtime fields

Edit the settings for runtime fields, or remove runtime fields from index patterns. 

. Open the main menu, then click *Stack Management > Index Patterns*.

. Select the index pattern that contains the runtime field you want to manage, then open the runtime field edit options or delete the runtime field.

[float]
[[scripted-fields]]
=== Add scripted fields to index patterns

deprecated::[7.13,Use {ref}/runtime.html[runtime fields] instead of scripted fields. Runtime fields support Painless scripts and provide greater flexibility.]

Scripted fields compute data on the fly from the data in your {es} indices. The data is shown on
the Discover tab as part of the document data, and you can use scripted fields in your visualizations. You query scripted fields with the <<kuery-query, {kib} query language>>, and can filter them using the filter bar. The scripted field values are computed at query time, so they aren't indexed and cannot be searched using the {kib} default
query language.

WARNING: Computing data on the fly with scripted fields can be very resource intensive and can have a direct impact on
{kib} performance. Keep in mind that there's no built-in validation of a scripted field. If your scripts are
buggy, you'll get exceptions whenever you try to view the dynamically generated data.

When you define a scripted field in {kib}, you have a choice of the {ref}/modules-scripting-expression.html[Lucene expressions] or the
{ref}/modules-scripting-painless.html[Painless] scripting language.

You can reference any single value numeric field in your expressions, for example:

----
doc['field_name'].value
----

For more information on scripted fields and additional examples, refer to
https://www.elastic.co/blog/using-painless-kibana-scripted-fields[Using Painless in {kib} scripted fields]

[float]
[[create-scripted-field]]
==== Create scripted fields

Create and add scripted fields to your index patterns.

. Open the main menu, then click *Stack Management > Index Patterns*.

. Select the index pattern you want to add a scripted field to.

. Select the *Scripted fields* tab, then click *Add scripted field*.

. Enter a *Name* for the scripted field, then enter the *Script* you want to use to compute a value on the fly from your index data.

. Click *Create field*.

For more information about scripted fields in {es}, refer to {ref}/modules-scripting.html[Scripting].

[float]
[[update-scripted-field]]
==== Manage scripted fields

. Open the main menu, then click *Stack Management > Index Patterns*.

. Select the index pattern that contains the scripted field you want to manage.

. Select the *Scripted fields* tab, then open the scripted field edit options or delete the scripted field.

WARNING: Built-in validation is unsupported for scripted fields. When your scripts contain errors, you receive
exceptions when you view the dynamically generated data.

[float]
[[managing-fields]]
=== Format data fields

{kib} uses the same field types as {es}, however, some {es} field types are unsupported in {kib}.
To customize how {kib} displays data fields, use the formatting options.

. Open the main menu, then click *Stack Management > Index Patterns*.

. Click the index pattern that contains the field you want to change.

. Find the field, then open the edit options (image:management/index-patterns/images/edit_icon.png[Data field edit icon]).

. Select *Set custom label*, then enter a *Custom label* for the field.

. Select *Set format*, then enter the *Format* for the field.

[float]
[[string-field-formatters]]
==== String field formatters

String fields support *String* and *Url* formatters.

include::field-formatters/string-formatter.asciidoc[]

include::field-formatters/url-formatter.asciidoc[]

[float]
[[field-formatters-date]]
==== Date field formatters

Date fields support *Date*, *String*, and *Url* formatters.

The *Date* formatter enables you to choose the display format of date stamps using the https://momentjs.com/[moment.js]
standard format definitions.

include::field-formatters/string-formatter.asciidoc[]

include::field-formatters/url-formatter.asciidoc[]

[float]
[[field-formatters-geopoint]]
==== Geographic point field formatters

Geographic point fields support the *String* formatter.

include::field-formatters/string-formatter.asciidoc[]

[float]
[[field-formatters-numeric]]
==== Number field formatters

Numeric fields support *Bytes*, *Color*, *Duration*, *Histogram*, *Number*, *Percentage*, *String*, and *Url* formatters.

The *Bytes*, *Number*, and *Percentage* formatters enable you to choose the display formats of numbers in the field using
the <<numeral, Elastic numeral pattern>> syntax that {kib} maintains.

The *Histogram* formatter is used only for the {ref}/histogram.html[histogram field type]. When you use the *Histogram* formatter,
you can apply the *Bytes*, *Number*, or *Percentage* format to aggregated data.

include::field-formatters/url-formatter.asciidoc[]

include::field-formatters/string-formatter.asciidoc[]

include::field-formatters/duration-formatter.asciidoc[]

include::field-formatters/color-formatter.asciidoc[]