diff --git a/docs/changelog/92380.yaml b/docs/changelog/92380.yaml new file mode 100644 index 000000000000..0fc266e5c630 --- /dev/null +++ b/docs/changelog/92380.yaml @@ -0,0 +1,5 @@ +pr: 92380 +summary: Add option to allow script errors on runtime fields +area: Search +type: enhancement +issues: [] diff --git a/docs/reference/mapping/runtime.asciidoc b/docs/reference/mapping/runtime.asciidoc index fd5f1a01cb71..20793fd9b17a 100644 --- a/docs/reference/mapping/runtime.asciidoc +++ b/docs/reference/mapping/runtime.asciidoc @@ -53,7 +53,7 @@ combined use less resources and reduce your operating costs. Runtime fields can replace many of the ways you can use scripting with the `_search` API. How you use a runtime field is impacted by the number of documents that the included script runs against. For example, if you're using -the `fields` parameter on the `_search` API to +the `fields` parameter on the `_search` API to <>, the script runs only against the top hits just like script fields do. @@ -77,7 +77,7 @@ If you move a script from any of these sections in a search request to a runtime field that is computing values from the same number of documents, the performance should be about the same. The performance for these features is largely dependent upon the calculations that the included script is running and -how many documents the script runs against. +how many documents the script runs against. [discrete] [[runtime-compromises]] @@ -88,9 +88,9 @@ the runtime script. To balance search performance and flexibility, index fields that you'll frequently search for and filter on, such as a timestamp. {es} automatically -uses these indexed fields first when running a query, resulting in a fast -response time. You can then use runtime fields to limit the number of fields -that {es} needs to calculate values for. Using indexed fields in tandem with +uses these indexed fields first when running a query, resulting in a fast +response time. You can then use runtime fields to limit the number of fields +that {es} needs to calculate values for. Using indexed fields in tandem with runtime fields provides flexibility in the data that you index and how you define queries for other fields. @@ -111,7 +111,7 @@ You map runtime fields by adding a `runtime` section under the mapping definition and defining <>. This script has access to the entire context of a document, including the original `_source` via `params._source` -and any mapped fields plus their values. At query time, the script runs and +and any mapped fields plus their values. At query time, the script runs and generates values for each scripted field that is required for the query. .Emitting runtime field values @@ -227,6 +227,16 @@ with `params._source` (such as `params._source.day_of_week`). For simplicity, defining a runtime field in the mapping definition without a script is the recommended option, whenever possible. +[[runtime-errorhandling]] +==== Ignoring script errors on runtime fields + +Scripts can throw errors at runtime, e.g. on accessing missing or invalid values +in documents or because of performing invalid operations. The `on_script_error` +parameter can be used to control error behaviour when this happens. Setting this +parameter to `continue` will have the effect of silently ignoring all errors on +this runtime field. The default `fail` value will cause a shard failure which +gets reported in the search response. + [[runtime-updating-scripts]] ==== Updating and removing runtime fields @@ -932,14 +942,14 @@ can define runtime fields in the decide to index a runtime field for greater performance, just move the full runtime field definition (including the script) to the context of an index mapping. {es} automatically uses these indexed fields to drive queries, -resulting in a fast response time. This capability means you can write a +resulting in a fast response time. This capability means you can write a script only once, and apply it to any context that supports runtime fields. NOTE: Indexing a `composite` runtime field is currently not supported. -You can then use runtime fields to limit the number of fields that {es} needs -to calculate values for. Using indexed fields in tandem with runtime fields -provides flexibility in the data that you index and how you define queries for +You can then use runtime fields to limit the number of fields that {es} needs +to calculate values for. Using indexed fields in tandem with runtime fields +provides flexibility in the data that you index and how you define queries for other fields. IMPORTANT: After indexing a runtime field, you cannot update the included @@ -1417,9 +1427,9 @@ GET my-index-000001/_search [[runtime-examples-grok-composite]] ==== Define a composite runtime field -You can also define a _composite_ runtime field to emit multiple fields from a -single script. You can define a set of typed subfields and emit a map of -values. At search time, each subfield retrieves the value associated with +You can also define a _composite_ runtime field to emit multiple fields from a +single script. You can define a set of typed subfields and emit a map of +values. At search time, each subfield retrieves the value associated with their name in the map. This means that you only need to specify your grok pattern one time and can return multiple values: @@ -1467,11 +1477,11 @@ GET my-index-000001/_search ---- // TEST[continued] -The API returns the following result. Because `http` is a `composite` runtime +The API returns the following result. Because `http` is a `composite` runtime field, the response includes each of the sub-fields under `fields`, including -any associated values that match the query. Without building your data structure +any associated values that match the query. Without building your data structure in advance, you can search and explore your data in meaningful ways to -experiment and determine which fields to index. +experiment and determine which fields to index. [source,console-result] ---- diff --git a/modules/lang-painless/src/main/java/org/elasticsearch/painless/action/PainlessExecuteAction.java b/modules/lang-painless/src/main/java/org/elasticsearch/painless/action/PainlessExecuteAction.java index a65d18a762ec..1c50b3558dcd 100644 --- a/modules/lang-painless/src/main/java/org/elasticsearch/painless/action/PainlessExecuteAction.java +++ b/modules/lang-painless/src/main/java/org/elasticsearch/painless/action/PainlessExecuteAction.java @@ -49,6 +49,7 @@ import org.elasticsearch.index.Index; import org.elasticsearch.index.IndexService; import org.elasticsearch.index.mapper.DateFieldMapper; import org.elasticsearch.index.mapper.DocumentMapper; +import org.elasticsearch.index.mapper.OnScriptError; import org.elasticsearch.index.mapper.ParsedDocument; import org.elasticsearch.index.mapper.SourceToParse; import org.elasticsearch.index.query.AbstractQueryBuilder; @@ -557,7 +558,8 @@ public class PainlessExecuteAction extends ActionType booleans = new ArrayList<>(); @@ -571,7 +573,8 @@ public class PainlessExecuteAction extends ActionType dates = new ArrayList<>(); @@ -584,7 +587,8 @@ public class PainlessExecuteAction extends ActionType doubles = new ArrayList<>(); @@ -597,7 +601,8 @@ public class PainlessExecuteAction extends ActionType points = new ArrayList<>(); @@ -615,7 +620,8 @@ public class PainlessExecuteAction extends ActionType ips = new ArrayList<>(); @@ -634,7 +640,8 @@ public class PainlessExecuteAction extends ActionType longs = new ArrayList<>(); @@ -647,7 +654,8 @@ public class PainlessExecuteAction extends ActionType keywords = new ArrayList<>(); @@ -660,7 +668,8 @@ public class PainlessExecuteAction extends ActionType= 18); + on_script_error: continue + rtf_strict_error: + type: boolean + script: | + if(doc['age'].value <= 0) throw new Exception("invalid age"); + emit(doc['age'].value >=18); + on_script_error: fail + properties: + age: + type: integer + + - do: + bulk: + index: testindex + refresh: true + body: | + {"index":{}} + {"age": 14} + {"index":{}} + {"age": 20} + {"index":{}} + {"age": -1} + +--- +"Query rtf with on_script_error continue": + - do: + search: + index: testindex + body: + query: + match: + rtf: + true + fields: [ age, rtf ] + - match: { hits.total.value: 1 } + - match: { hits.hits.0.fields.age: [ 20 ] } + - match: { hits.hits.0.fields.rtf: [ true ] } + +--- +"Query rtf with on_script_error fail": + - do: + catch: /type=script_exception, reason=runtime error/ + search: + index: testindex + body: + query: + match: + rtf_strict_error: true + fields: [ age, rtf_strict_error ] + +--- +"Aggregate on rtf with on_script_error continue": + - do: + search: + index: testindex + body: + aggs: + rtf: + terms: + field: rtf + order: { "_key": "asc" } + - length: { aggregations.rtf.buckets: 2 } + - match: { aggregations.rtf.buckets.0.key_as_string: "false" } + - match: { aggregations.rtf.buckets.1.key_as_string: "true" } +--- +"Aggregate on rtf with on_script_error fail": + - do: + catch: /type=script_exception, reason=runtime error/ + search: + index: testindex + body: + aggs: + rtf: + terms: + field: rtf_strict_error + +--- +"Fields retrieval with ignoring error": + - do: + search: + index: testindex + body: + query: { match_all: { } } + fields: [ age, rtf ] + sort: { "age": "desc" } + - match: { hits.total.value: 3 } + - match: { hits.hits.0.fields.age: [ 20 ] } + - match: { hits.hits.0.fields.rtf: [ true ] } + - match: { hits.hits.1.fields.age: [ 14 ] } + - match: { hits.hits.1.fields.rtf: [ false ] } + - match: { hits.hits.2.fields.age: [ -1 ] } + - is_false: hits.hits.2.fields.rtf + +--- +"Fields retrieval with failing on error": + - do: + catch: /type=script_exception, reason=runtime error/ + search: + index: testindex + body: + query: { match_all: { } } + fields: [ age, rtf_strict_error ] + +--- +"Sorting with ignoring error": + - do: + search: + index: testindex + body: + query: { match_all: { } } + fields: [ age ] + sort: rtf + - match: { hits.total.value: 3 } + - match: { hits.hits.0.fields.age: [ 14 ] } + - match: { hits.hits.1.fields.age: [ 20 ] } + - match: { hits.hits.2.fields.age: [ -1 ] } + +--- +"Sorting with with failing on error": + - do: + catch: /type=script_exception, reason=runtime error/ + search: + index: testindex + body: + query: { match_all: { } } + fields: [ age ] + sort: rtf_strict_error + +--- +"Query search time rtf with on_script_error continue": + - do: + search: + index: testindex + body: + query: + match: + rtf_search: true + fields: [ age, rtf_search ] + runtime_mappings: + rtf_search: + type: boolean + script: | + if(doc['age'].value < 0) throw new Exception("invalid age"); + emit(doc['age'].value >= 18); + on_script_error: continue + + - match: { hits.total.value: 1 } + - match: { hits.hits.0.fields.age: [ 20 ] } + - match: { hits.hits.0.fields.rtf_search: [ true ] } + +--- +"Query search time rtf with on_script_error fail": + - do: + catch: /type=script_exception, reason=runtime error/ + search: + index: testindex + body: + query: + match: + rtf_search: true + fields: [ age, rtf_search ] + runtime_mappings: + rtf_search: + type: boolean + script: | + if(doc['age'].value < 0) throw new Exception("invalid age"); + emit(doc['age'].value >= 18); + on_script_error: fail diff --git a/server/src/main/java/org/elasticsearch/index/mapper/AbstractScriptFieldType.java b/server/src/main/java/org/elasticsearch/index/mapper/AbstractScriptFieldType.java index aa0e0c17a52b..684adb9ac124 100644 --- a/server/src/main/java/org/elasticsearch/index/mapper/AbstractScriptFieldType.java +++ b/server/src/main/java/org/elasticsearch/index/mapper/AbstractScriptFieldType.java @@ -215,7 +215,7 @@ abstract class AbstractScriptFieldType extends MappedFieldType { abstract static class Builder extends RuntimeField.Builder { private final ScriptContext scriptContext; - final FieldMapper.Parameter