mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-06-30 02:13:33 -04:00
c18078e11e
* Add docs for geo_grid ingest processor Adds docs for https://github.com/elastic/elasticsearch/pull/93370 * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Update docs/reference/ingest/processors/geo-grid.asciidoc Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co> * Consistent GeoJSON case --------- Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
250 lines
9.2 KiB
Text
250 lines
9.2 KiB
Text
[role="xpack"]
|
|
[[ingest-geo-grid-processor]]
|
|
=== Geo-grid processor
|
|
++++
|
|
<titleabbrev>Geo-grid</titleabbrev>
|
|
++++
|
|
|
|
Converts geo-grid definitions of grid tiles or cells to regular bounding boxes or polygons which describe their shape.
|
|
This is useful if there is a need to interact with the tile shapes as spatially indexable fields.
|
|
For example the `geotile` field value `"4/8/3"` could be indexed as a string field, but that would not enable
|
|
any spatial operations on it.
|
|
Instead, convert it to the value
|
|
`"POLYGON ((0.0 40.979898069620134, 22.5 40.979898069620134, 22.5 55.77657301866769, 0.0 55.77657301866769, 0.0 40.979898069620134))"`,
|
|
which can be indexed as a <<geo-shape,`geo_shape`>> field.
|
|
|
|
[[geo-grid-processor-options]]
|
|
.geo_grid processor options
|
|
[options="header"]
|
|
|======
|
|
| Name | Required | Default | Description
|
|
| `field` | yes | - | The field to interpret as a geo-tile. The field format is determined by the `tile_type`.
|
|
| `tile_type` | yes | - | Three tile formats are understood: `geohash`, `geotile` and `geohex`.
|
|
| `target_field` | no | `field` | The field to assign the polygon shape to, by default `field` is updated in-place.
|
|
| `parent_field` | no | - | If specified and a parent tile exists, save that tile address to this field.
|
|
| `children_field` | no | - | If specified and children tiles exist, save those tile addresses to this field as an array of strings.
|
|
| `non_children_field` | no | - | If specified and intersecting non-child tiles exist, save their addresses to this field as an array of strings.
|
|
| `precision_field` | no | - | If specified, save the tile precision (zoom) as an integer to this field.
|
|
| `ignore_missing` | no | - | If `true` and `field` does not exist, the processor quietly exits without modifying the document.
|
|
| `target_format` | no | "GeoJSON" | Which format to save the generated polygon in. Either `WKT` or `GeoJSON`.
|
|
include::common-options.asciidoc[]
|
|
|======
|
|
|
|
To demonstrate the usage of this ingest processor, consider an index called `geocells`
|
|
with a mapping for a field `geocell` of type `geo_shape`.
|
|
In order to populate that index using `geotile` and `geohex` fields, define
|
|
two ingest processors:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT geocells
|
|
{
|
|
"mappings": {
|
|
"properties": {
|
|
"geocell": {
|
|
"type": "geo_shape"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
PUT _ingest/pipeline/geotile2shape
|
|
{
|
|
"description": "translate rectangular z/x/y geotile to bounding box",
|
|
"processors": [
|
|
{
|
|
"geo_grid": {
|
|
"field": "geocell",
|
|
"tile_type": "geotile"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
|
|
PUT _ingest/pipeline/geohex2shape
|
|
{
|
|
"description": "translate H3 cell to polygon",
|
|
"processors": [
|
|
{
|
|
"geo_grid": {
|
|
"field": "geocell",
|
|
"tile_type": "geohex",
|
|
"target_format": "wkt"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
|
|
These two pipelines can be used to index documents into the `geocells` index.
|
|
The `geocell` field will be the string version of either a rectangular tile with format `z/x/y` or an H3 cell address,
|
|
depending on which ingest processor we use when indexing the document.
|
|
The resulting geometry will be represented and indexed as a <<geo-shape,`geo_shape`>> field in either
|
|
http://geojson.org[GeoJSON] or the https://docs.opengeospatial.org/is/12-063r5/12-063r5.html[Well-Known Text] format.
|
|
|
|
==== Example: Rectangular geotile with envelope in GeoJSON
|
|
|
|
In this example a `geocell` field with a value defined in `z/x/y` format is indexed as a
|
|
http://geojson.org[GeoJSON Envelope] since the ingest-processor above was defined with default `target_format`.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT geocells/_doc/1?pipeline=geotile2shape
|
|
{
|
|
"geocell": "4/8/5"
|
|
}
|
|
|
|
GET geocells/_doc/1
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
The response shows how the ingest-processor has replaced the `geocell` field with an indexable `geo_shape`:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"_index": "geocells",
|
|
"_id": "1",
|
|
"_version": 1,
|
|
"_seq_no": 0,
|
|
"_primary_term": 1,
|
|
"found": true,
|
|
"_source": {
|
|
"geocell": {
|
|
"type": "Envelope",
|
|
"coordinates": [
|
|
[ 0.0, 55.77657301866769 ],
|
|
[ 22.5, 40.979898069620134 ]
|
|
]
|
|
}
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[s/"_version": \d+/"_version": $body._version/ s/"_seq_no": \d+/"_seq_no": $body._seq_no/ s/"_primary_term": 1/"_primary_term": $body._primary_term/]
|
|
|
|
image:images/spatial/geogrid_tile.png[Kibana map with showing the geotile at 4/8/5 and its four child cells]
|
|
|
|
==== Example: Hexagonal geohex with polygon in WKT format
|
|
|
|
In this example a `geocell` field with an H3 string address is indexed as a
|
|
https://docs.opengeospatial.org/is/12-063r5/12-063r5.html[WKT Polygon], since this ingest processor explicitly
|
|
defined the `target_format`.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT geocells/_doc/1?pipeline=geohex2shape
|
|
{
|
|
"geocell": "811fbffffffffff"
|
|
}
|
|
|
|
GET geocells/_doc/1
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
The response shows how the ingest-processor has replaced the `geocell` field with an indexable `geo_shape`:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"_index": "geocells",
|
|
"_id": "1",
|
|
"_version": 1,
|
|
"_seq_no": 0,
|
|
"_primary_term": 1,
|
|
"found": true,
|
|
"_source": {
|
|
"geocell": "POLYGON ((1.1885095294564962 49.470279179513454, 2.0265689212828875 45.18424864858389, 7.509948452934623 43.786609335802495, 12.6773177459836 46.40695743262768, 12.345747342333198 50.55427505169064, 6.259687012061477 51.964770150370896, 3.6300085578113794 50.610463307239115, 1.1885095294564962 49.470279179513454))"
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[s/"_version": \d+/"_version": $body._version/ s/"_seq_no": \d+/"_seq_no": $body._seq_no/ s/"_primary_term": 1/"_primary_term": $body._primary_term/]
|
|
|
|
image:images/spatial/geogrid_h3.png[Kibana map with showing an H3 cell, and its seven child cells]
|
|
|
|
==== Example: Enriched tile details
|
|
|
|
As described in <<geo-grid-processor-options,geo_grid processor options>>,
|
|
there are many other fields that can be set, which will enrich the information available.
|
|
For example, with H3 tiles there are 7 child tiles, but only the first is fully contained by the parent.
|
|
The remaining six are only partially overlapping the parent, and there exist a further six non-child tiles
|
|
that overlap the parent.
|
|
This can be investigated by adding parent and child additional fields to the ingest-processor:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT _ingest/pipeline/geohex2shape
|
|
{
|
|
"description": "translate H3 cell to polygon with enriched fields",
|
|
"processors": [
|
|
{
|
|
"geo_grid": {
|
|
"description": "Ingest H3 cells like '811fbffffffffff' and create polygons",
|
|
"field": "geocell",
|
|
"tile_type": "geohex",
|
|
"target_format": "wkt",
|
|
"target_field": "shape",
|
|
"parent_field": "parent",
|
|
"children_field": "children",
|
|
"non_children_field": "nonChildren",
|
|
"precision_field": "precision"
|
|
}
|
|
}
|
|
]
|
|
}
|
|
--------------------------------------------------
|
|
|
|
Index the document to see a different result:
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
PUT geocells/_doc/1?pipeline=geohex2shape
|
|
{
|
|
"geocell": "811fbffffffffff"
|
|
}
|
|
|
|
GET geocells/_doc/1
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
|
|
The response from this index request:
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"_index": "geocells",
|
|
"_id": "1",
|
|
"_version": 1,
|
|
"_seq_no": 0,
|
|
"_primary_term": 1,
|
|
"found": true,
|
|
"_source": {
|
|
"parent": "801ffffffffffff",
|
|
"geocell": "811fbffffffffff",
|
|
"precision": 1,
|
|
"shape": "POLYGON ((1.1885095294564962 49.470279179513454, 2.0265689212828875 45.18424864858389, 7.509948452934623 43.786609335802495, 12.6773177459836 46.40695743262768, 12.345747342333198 50.55427505169064, 6.259687012061477 51.964770150370896, 3.6300085578113794 50.610463307239115, 1.1885095294564962 49.470279179513454))",
|
|
"children": [
|
|
"821f87fffffffff",
|
|
"821f8ffffffffff",
|
|
"821f97fffffffff",
|
|
"821f9ffffffffff",
|
|
"821fa7fffffffff",
|
|
"821faffffffffff",
|
|
"821fb7fffffffff"
|
|
],
|
|
"nonChildren": [
|
|
"821ea7fffffffff",
|
|
"82186ffffffffff",
|
|
"82396ffffffffff",
|
|
"821f17fffffffff",
|
|
"821e37fffffffff",
|
|
"82194ffffffffff"
|
|
]
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TESTRESPONSE[s/"_version": \d+/"_version": $body._version/ s/"_seq_no": \d+/"_seq_no": $body._seq_no/ s/"_primary_term": 1/"_primary_term": $body._primary_term/]
|
|
|
|
This additional information will then enable, for example, creating a visualization of the H3 cell,
|
|
its children and its intersecting non-children cells.
|
|
|
|
image:images/spatial/geogrid_h3_children.png[Kibana map with three H3 layers: cell, children and intersecting non-children]
|