mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-06-28 17:34:17 -04:00
Runtime fields to optionally ignore script errors (#92380)
Currently Elasticsearch always returns a shard failure once a runtime error arises from using a runtime field, the exception being script-less runtime fields. This also means that execution of the query for that shard stops, which is okay for development and exploration. In a production scenario, however, it is often desirable to ignore runtime errors and continue with the query execution. This change adds a new a new on_script_error parameter to runtime field definitions similar to the already existing parameter for index-time scripted fields. When `on_script_error` is set to `continue`, errors from script execution are effectively ignored. This means affected documents don't show up in query results, but also don't prevent other matches from the same shard. Runtime fields accessed through the fields API don't return values on errors, aggregations will ignore documents that throw errors. Note that this change affects scripted runtime fields only, while leaving default behaviour untouched. Also, ignored errors are not reported back to users for now. Relates to #72143
This commit is contained in:
parent
3bbf202843
commit
8067f01d48
78 changed files with 2184 additions and 303 deletions
|
@ -53,7 +53,7 @@ combined use less resources and reduce your operating costs.
|
|||
Runtime fields can replace many of the ways you can use scripting with the
|
||||
`_search` API. How you use a runtime field is impacted by the number of
|
||||
documents that the included script runs against. For example, if you're using
|
||||
the `fields` parameter on the `_search` API to
|
||||
the `fields` parameter on the `_search` API to
|
||||
<<runtime-retrieving-fields,retrieve the values of a runtime field>>, the script
|
||||
runs only against the top hits just like script fields do.
|
||||
|
||||
|
@ -77,7 +77,7 @@ If you move a script from any of these sections in a search request to a
|
|||
runtime field that is computing values from the same number of documents, the
|
||||
performance should be about the same. The performance for these features is
|
||||
largely dependent upon the calculations that the included script is running and
|
||||
how many documents the script runs against.
|
||||
how many documents the script runs against.
|
||||
|
||||
[discrete]
|
||||
[[runtime-compromises]]
|
||||
|
@ -88,9 +88,9 @@ the runtime script.
|
|||
|
||||
To balance search performance and flexibility, index fields that you'll
|
||||
frequently search for and filter on, such as a timestamp. {es} automatically
|
||||
uses these indexed fields first when running a query, resulting in a fast
|
||||
response time. You can then use runtime fields to limit the number of fields
|
||||
that {es} needs to calculate values for. Using indexed fields in tandem with
|
||||
uses these indexed fields first when running a query, resulting in a fast
|
||||
response time. You can then use runtime fields to limit the number of fields
|
||||
that {es} needs to calculate values for. Using indexed fields in tandem with
|
||||
runtime fields provides flexibility in the data that you index and how you
|
||||
define queries for other fields.
|
||||
|
||||
|
@ -111,7 +111,7 @@ You map runtime fields by adding a `runtime` section under the mapping
|
|||
definition and defining
|
||||
<<modules-scripting-using,a Painless script>>. This script has access to the
|
||||
entire context of a document, including the original `_source` via `params._source`
|
||||
and any mapped fields plus their values. At query time, the script runs and
|
||||
and any mapped fields plus their values. At query time, the script runs and
|
||||
generates values for each scripted field that is required for the query.
|
||||
|
||||
.Emitting runtime field values
|
||||
|
@ -227,6 +227,16 @@ with `params._source` (such as `params._source.day_of_week`). For simplicity,
|
|||
defining a runtime field in the mapping definition without a script is the
|
||||
recommended option, whenever possible.
|
||||
|
||||
[[runtime-errorhandling]]
|
||||
==== Ignoring script errors on runtime fields
|
||||
|
||||
Scripts can throw errors at runtime, e.g. on accessing missing or invalid values
|
||||
in documents or because of performing invalid operations. The `on_script_error`
|
||||
parameter can be used to control error behaviour when this happens. Setting this
|
||||
parameter to `continue` will have the effect of silently ignoring all errors on
|
||||
this runtime field. The default `fail` value will cause a shard failure which
|
||||
gets reported in the search response.
|
||||
|
||||
[[runtime-updating-scripts]]
|
||||
==== Updating and removing runtime fields
|
||||
|
||||
|
@ -932,14 +942,14 @@ can define runtime fields in the
|
|||
decide to index a runtime field for greater performance, just move the full
|
||||
runtime field definition (including the script) to the context of an index
|
||||
mapping. {es} automatically uses these indexed fields to drive queries,
|
||||
resulting in a fast response time. This capability means you can write a
|
||||
resulting in a fast response time. This capability means you can write a
|
||||
script only once, and apply it to any context that supports runtime fields.
|
||||
|
||||
NOTE: Indexing a `composite` runtime field is currently not supported.
|
||||
|
||||
You can then use runtime fields to limit the number of fields that {es} needs
|
||||
to calculate values for. Using indexed fields in tandem with runtime fields
|
||||
provides flexibility in the data that you index and how you define queries for
|
||||
You can then use runtime fields to limit the number of fields that {es} needs
|
||||
to calculate values for. Using indexed fields in tandem with runtime fields
|
||||
provides flexibility in the data that you index and how you define queries for
|
||||
other fields.
|
||||
|
||||
IMPORTANT: After indexing a runtime field, you cannot update the included
|
||||
|
@ -1417,9 +1427,9 @@ GET my-index-000001/_search
|
|||
|
||||
[[runtime-examples-grok-composite]]
|
||||
==== Define a composite runtime field
|
||||
You can also define a _composite_ runtime field to emit multiple fields from a
|
||||
single script. You can define a set of typed subfields and emit a map of
|
||||
values. At search time, each subfield retrieves the value associated with
|
||||
You can also define a _composite_ runtime field to emit multiple fields from a
|
||||
single script. You can define a set of typed subfields and emit a map of
|
||||
values. At search time, each subfield retrieves the value associated with
|
||||
their name in the map. This means that you only need to specify your grok
|
||||
pattern one time and can return multiple values:
|
||||
|
||||
|
@ -1467,11 +1477,11 @@ GET my-index-000001/_search
|
|||
----
|
||||
// TEST[continued]
|
||||
|
||||
The API returns the following result. Because `http` is a `composite` runtime
|
||||
The API returns the following result. Because `http` is a `composite` runtime
|
||||
field, the response includes each of the sub-fields under `fields`, including
|
||||
any associated values that match the query. Without building your data structure
|
||||
any associated values that match the query. Without building your data structure
|
||||
in advance, you can search and explore your data in meaningful ways to
|
||||
experiment and determine which fields to index.
|
||||
experiment and determine which fields to index.
|
||||
|
||||
[source,console-result]
|
||||
----
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue