Runtime fields to optionally ignore script errors (#92380)

Currently Elasticsearch always returns a shard failure once a runtime error arises from using a runtime field, the exception being script-less runtime fields. This also means that execution of the query for that shard stops, which is okay for development and exploration. In a production scenario, however, it is often desirable to ignore runtime errors and continue with the query execution.

This change adds a new a new on_script_error parameter to runtime field definitions similar to the already existing
parameter for index-time scripted fields. When `on_script_error` is set to `continue`, errors from script execution are effectively ignored. This means affected documents don't show up in query results, but also don't prevent other matches from the same shard. Runtime fields accessed through the fields API don't return values on errors, aggregations will ignore documents that throw errors.

Note that this change affects scripted runtime fields only, while leaving default behaviour untouched. Also, ignored errors are not reported back to users for now.

Relates to #72143
This commit is contained in:
Christoph Büscher 2022-12-23 09:29:12 +01:00 committed by GitHub
parent 3bbf202843
commit 8067f01d48
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
78 changed files with 2184 additions and 303 deletions

View file

@ -53,7 +53,7 @@ combined use less resources and reduce your operating costs.
Runtime fields can replace many of the ways you can use scripting with the
`_search` API. How you use a runtime field is impacted by the number of
documents that the included script runs against. For example, if you're using
the `fields` parameter on the `_search` API to
the `fields` parameter on the `_search` API to
<<runtime-retrieving-fields,retrieve the values of a runtime field>>, the script
runs only against the top hits just like script fields do.
@ -77,7 +77,7 @@ If you move a script from any of these sections in a search request to a
runtime field that is computing values from the same number of documents, the
performance should be about the same. The performance for these features is
largely dependent upon the calculations that the included script is running and
how many documents the script runs against.
how many documents the script runs against.
[discrete]
[[runtime-compromises]]
@ -88,9 +88,9 @@ the runtime script.
To balance search performance and flexibility, index fields that you'll
frequently search for and filter on, such as a timestamp. {es} automatically
uses these indexed fields first when running a query, resulting in a fast
response time. You can then use runtime fields to limit the number of fields
that {es} needs to calculate values for. Using indexed fields in tandem with
uses these indexed fields first when running a query, resulting in a fast
response time. You can then use runtime fields to limit the number of fields
that {es} needs to calculate values for. Using indexed fields in tandem with
runtime fields provides flexibility in the data that you index and how you
define queries for other fields.
@ -111,7 +111,7 @@ You map runtime fields by adding a `runtime` section under the mapping
definition and defining
<<modules-scripting-using,a Painless script>>. This script has access to the
entire context of a document, including the original `_source` via `params._source`
and any mapped fields plus their values. At query time, the script runs and
and any mapped fields plus their values. At query time, the script runs and
generates values for each scripted field that is required for the query.
.Emitting runtime field values
@ -227,6 +227,16 @@ with `params._source` (such as `params._source.day_of_week`). For simplicity,
defining a runtime field in the mapping definition without a script is the
recommended option, whenever possible.
[[runtime-errorhandling]]
==== Ignoring script errors on runtime fields
Scripts can throw errors at runtime, e.g. on accessing missing or invalid values
in documents or because of performing invalid operations. The `on_script_error`
parameter can be used to control error behaviour when this happens. Setting this
parameter to `continue` will have the effect of silently ignoring all errors on
this runtime field. The default `fail` value will cause a shard failure which
gets reported in the search response.
[[runtime-updating-scripts]]
==== Updating and removing runtime fields
@ -932,14 +942,14 @@ can define runtime fields in the
decide to index a runtime field for greater performance, just move the full
runtime field definition (including the script) to the context of an index
mapping. {es} automatically uses these indexed fields to drive queries,
resulting in a fast response time. This capability means you can write a
resulting in a fast response time. This capability means you can write a
script only once, and apply it to any context that supports runtime fields.
NOTE: Indexing a `composite` runtime field is currently not supported.
You can then use runtime fields to limit the number of fields that {es} needs
to calculate values for. Using indexed fields in tandem with runtime fields
provides flexibility in the data that you index and how you define queries for
You can then use runtime fields to limit the number of fields that {es} needs
to calculate values for. Using indexed fields in tandem with runtime fields
provides flexibility in the data that you index and how you define queries for
other fields.
IMPORTANT: After indexing a runtime field, you cannot update the included
@ -1417,9 +1427,9 @@ GET my-index-000001/_search
[[runtime-examples-grok-composite]]
==== Define a composite runtime field
You can also define a _composite_ runtime field to emit multiple fields from a
single script. You can define a set of typed subfields and emit a map of
values. At search time, each subfield retrieves the value associated with
You can also define a _composite_ runtime field to emit multiple fields from a
single script. You can define a set of typed subfields and emit a map of
values. At search time, each subfield retrieves the value associated with
their name in the map. This means that you only need to specify your grok
pattern one time and can return multiple values:
@ -1467,11 +1477,11 @@ GET my-index-000001/_search
----
// TEST[continued]
The API returns the following result. Because `http` is a `composite` runtime
The API returns the following result. Because `http` is a `composite` runtime
field, the response includes each of the sub-fields under `fields`, including
any associated values that match the query. Without building your data structure
any associated values that match the query. Without building your data structure
in advance, you can search and explore your data in meaningful ways to
experiment and determine which fields to index.
experiment and determine which fields to index.
[source,console-result]
----