mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-25 07:37:19 -04:00
319 lines
7.4 KiB
Text
319 lines
7.4 KiB
Text
[[foreach-processor]]
|
|
=== Foreach processor
|
|
++++
|
|
<titleabbrev>Foreach</titleabbrev>
|
|
++++
|
|
|
|
Runs an ingest processor on each element of an array or object.
|
|
|
|
All ingest processors can run on array or object elements. However, if the
|
|
number of elements is unknown, it can be cumbersome to process each one in the
|
|
same way.
|
|
|
|
The `foreach` processor lets you specify a `field` containing array or object
|
|
values and a `processor` to run on each element in the field.
|
|
|
|
[[foreach-options]]
|
|
.Foreach Options
|
|
[options="header"]
|
|
|======
|
|
| Name | Required | Default | Description
|
|
| `field` | yes | - | Field containing array or object
|
|
values.
|
|
| `processor` | yes | - | Ingest processor to run on each
|
|
element.
|
|
| `ignore_missing` | no | false | If `true`, the processor silently
|
|
exits without changing the document if the `field` is `null` or missing.
|
|
include::common-options.asciidoc[]
|
|
|======
|
|
|
|
[discrete]
|
|
[[foreach-keys-values]]
|
|
==== Access keys and values
|
|
|
|
When iterating through an array or object, the `foreach` processor stores the
|
|
current element's value in the `_ingest._value` <<access-ingest-metadata,ingest
|
|
metadata>> field. `_ingest._value` contains the entire element value, including
|
|
any child fields. You can access child field values using dot notation on the
|
|
`_ingest._value` field.
|
|
|
|
When iterating through an object, the `foreach` processor also stores the
|
|
current element's key as a string in `_ingest._key`.
|
|
|
|
You can access and change `_ingest._key` and `_ingest._value` in the
|
|
`processor`. For an example, see the <<foreach-object-ex, object
|
|
example>>.
|
|
|
|
[discrete]
|
|
[[foreach-failure-handling]]
|
|
==== Failure handling
|
|
|
|
If the `foreach` processor fails to process an element and no `on_failure`
|
|
processor is specified, the `foreach` processor silently exits. This leaves the
|
|
entire array or object value unchanged.
|
|
|
|
[discrete]
|
|
[[foreach-examples]]
|
|
==== Examples
|
|
|
|
The following examples show how you can use the `foreach` processor with
|
|
different data types and options:
|
|
|
|
* <<foreach-array-ex>>
|
|
* <<foreach-array-objects-ex>>
|
|
* <<foreach-object-ex>>
|
|
* <<failure-handling-ex>>
|
|
|
|
[discrete]
|
|
[[foreach-array-ex]]
|
|
===== Array
|
|
|
|
Assume the following document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"values" : ["foo", "bar", "baz"]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
When this `foreach` processor operates on this sample document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach" : {
|
|
"field" : "values",
|
|
"processor" : {
|
|
"uppercase" : {
|
|
"field" : "_ingest._value"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
Then the document will look like this after processing:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"values" : ["FOO", "BAR", "BAZ"]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
[discrete]
|
|
[[foreach-array-objects-ex]]
|
|
===== Array of objects
|
|
|
|
Assume the following document:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"persons" : [
|
|
{
|
|
"id" : "1",
|
|
"name" : "John Doe"
|
|
},
|
|
{
|
|
"id" : "2",
|
|
"name" : "Jane Doe"
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
In this case, the `id` field needs to be removed,
|
|
so the following `foreach` processor is used:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach" : {
|
|
"field" : "persons",
|
|
"processor" : {
|
|
"remove" : {
|
|
"field" : "_ingest._value.id"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
After processing the result is:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"persons" : [
|
|
{
|
|
"name" : "John Doe"
|
|
},
|
|
{
|
|
"name" : "Jane Doe"
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
For another array of objects example, refer to the
|
|
<<attachment-with-arrays,attachment processor documentation>>.
|
|
|
|
[discrete]
|
|
[[foreach-object-ex]]
|
|
===== Object
|
|
|
|
You can also use the `foreach` processor on object fields. For example,
|
|
the following document contains a `products` field with object values.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"products" : {
|
|
"widgets" : {
|
|
"total_sales" : 50,
|
|
"unit_price": 1.99,
|
|
"display_name": ""
|
|
},
|
|
"sprockets" : {
|
|
"total_sales" : 100,
|
|
"unit_price": 9.99,
|
|
"display_name": "Super Sprockets"
|
|
},
|
|
"whizbangs" : {
|
|
"total_sales" : 200,
|
|
"unit_price": 19.99,
|
|
"display_name": "Wonderful Whizbangs"
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
The following `foreach` processor changes the value of `products.display_name`
|
|
to uppercase.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach": {
|
|
"field": "products",
|
|
"processor": {
|
|
"uppercase": {
|
|
"field": "_ingest._value.display_name"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
When run on the document, the `foreach` processor returns:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"products" : {
|
|
"widgets" : {
|
|
"total_sales" : 50,
|
|
"unit_price" : 1.99,
|
|
"display_name" : ""
|
|
},
|
|
"sprockets" : {
|
|
"total_sales" : 100,
|
|
"unit_price" : 9.99,
|
|
"display_name" : "SUPER SPROCKETS"
|
|
},
|
|
"whizbangs" : {
|
|
"total_sales" : 200,
|
|
"unit_price" : 19.99,
|
|
"display_name" : "WONDERFUL WHIZBANGS"
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
The following `foreach` processor sets each element's key to the
|
|
value of `products.display_name`. If `products.display_name` contains an empty string,
|
|
the processor deletes the element.
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach": {
|
|
"field": "products",
|
|
"processor": {
|
|
"set": {
|
|
"field": "_ingest._key",
|
|
"value": "{{_ingest._value.display_name}}"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
When run on the previous document, the `foreach` processor returns:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"products" : {
|
|
"Wonderful Whizbangs" : {
|
|
"total_sales" : 200,
|
|
"unit_price" : 19.99,
|
|
"display_name" : "Wonderful Whizbangs"
|
|
},
|
|
"Super Sprockets" : {
|
|
"total_sales" : 100,
|
|
"unit_price" : 9.99,
|
|
"display_name" : "Super Sprockets"
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
[discrete]
|
|
[[failure-handling-ex]]
|
|
===== Failure handling
|
|
|
|
The wrapped processor can have a `on_failure` definition.
|
|
For example, the `id` field may not exist on all person objects.
|
|
Instead of failing the index request, you can use an `on_failure`
|
|
block to send the document to the 'failure_index' index for later inspection:
|
|
|
|
[source,js]
|
|
--------------------------------------------------
|
|
{
|
|
"foreach" : {
|
|
"field" : "persons",
|
|
"processor" : {
|
|
"remove" : {
|
|
"field" : "_value.id",
|
|
"on_failure" : [
|
|
{
|
|
"set" : {
|
|
"field": "_index",
|
|
"value": "failure_index"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// NOTCONSOLE
|
|
|
|
In this example, if the `remove` processor does fail, then
|
|
the array elements that have been processed thus far will
|
|
be updated.
|