[DOCS] Overhaul search template docs (#72583)

Changes:

* Adds a tutorial for search templates.
* Adds reference docs for the render search template API.
* Improves parameter documentation for the multi search template API.
* Removes duplicate examples from the search template API, multi search API, and create stored script API docs.
* Splits the source files for the search template API and the multi search template API docs.
This commit is contained in:
James Rodewig 2021-06-30 16:03:49 -04:00 committed by GitHub
parent 8bbba5f8ff
commit 3d1cb4944d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
13 changed files with 1054 additions and 829 deletions

View file

@ -82,34 +82,14 @@ Contains the script or search template, its parameters, and its language.
`source`::
(Required, string or object)
Script or search template.
For scripts, a string containing the script.
+
For search templates, an object containing the search template. The object
supports the same parameters as the <<search-search,search API>>'s request body.
Also supports https://mustache.github.io/[Mustache] variables. See
<<search-template>>.
`params`::
(Optional, object)
Parameters for the script or search template.
====
[[create-stored-script-api-example]]
==== {api-examples-title}
The following request stores a search template. Search templates must use a
`lang` of `mustache`.
[source,console]
----
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"from": "{{from}}{{^from}}0{{/from}}",
"size": "{{size}}{{^size}}10{{/size}}",
"query": {
"match": {
"content": "{{query_string}}"
}
}
}
}
}
----

View file

@ -35,8 +35,9 @@ exception of the <<search-explain,explain API>>.
[[search-template-apis]]
=== Search templates
* <<search-template>>
* <<search-template-api>>
* <<multi-search-template>>
* <<render-search-template-api>>
[discrete]
[[eql-search-apis]]
@ -60,7 +61,11 @@ include::search/scroll-api.asciidoc[]
include::search/clear-scroll-api.asciidoc[]
include::search/search-template.asciidoc[]
include::search/search-template-api.asciidoc[]
include::search/multi-search-template-api.asciidoc[]
include::search/render-search-template-api.asciidoc[]
include::search/search-shards.asciidoc[]

View file

@ -0,0 +1,163 @@
[[multi-search-template]]
=== Multi search template API
++++
<titleabbrev>Multi search template</titleabbrev>
++++
Runs multiple <<run-multiple-templated-searches,templated searches>> with a single
request.
////
[source,console]
----
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
},
"params": {
"query_string": "My query string"
}
}
}
PUT my-index/_doc/1?refresh
{
"message": "hello world"
}
----
// TESTSETUP
////
[source,console]
----
GET my-index/_msearch/template
{ }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}
----
// TEST[s/my-other-search-template/my-search-template/]
[[multi-search-template-api-request]]
==== {api-request-title}
`GET <target>/_msearch/template`
`GET _msearch/template`
`POST <target>/_msearch/template`
`POST _msearch/template`
[[multi-search-template-api-prereqs]]
==== {api-prereq-title}
* If the {es} {security-features} are enabled, you must have the `read`
<<privileges-list-indices,index privilege>> for the target data stream, index,
or alias. For cross-cluster search, see <<cross-cluster-configuring>>.
[[multi-search-template-api-path-params]]
==== {api-path-parms-title}
`<target>`::
(Optional, string) Comma-separated list of data streams, indices, and aliases to
search. Supports wildcards (`*`). To search all data streams and indices, omit
this parameter or use `*`.
[[multi-search-template-api-query-params]]
==== {api-query-parms-title}
`ccs_minimize_roundtrips`::
(Optional, Boolean) If `true`, network round-trips are minimized for
cross-cluster search requests. Defaults to `true`.
`max_concurrent_searches`::
(Optional, integer) Maximum number of concurrent searches the API can run.
Defaults to +max(1, (# of <<data-node,data nodes>> *
min(<<search-threadpool,search thread pool size>>, 10)))+.
`rest_total_hits_as_int`::
(Optional, Boolean) If `true`, the response returns `hits.total` as an integer.
If false, it returns `hits.total` as an object. Defaults to `false`.
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=search_type]
`typed_keys`::
(Optional, Boolean) If `true`, the response prefixes aggregation and suggester
names with their respective types. Defaults to `false`.
[role="child_attributes"]
[[multi-search-template-api-request-body]]
==== {api-request-body-title}
The request body must be newline-delimited JSON (NDJSON) in the following
format:
[source,js]
----
<header>\n
<body>\n
<header>\n
<body>\n
----
// NOTCONSOLE
Each `<header>` and `<body>` pair represents a search request.
The `<header>` supports the same parameters as the <<search-multi-search,multi
search API>>'s `<header>`. The `<body>` supports the same parameters as the
<<search-search-api-request-body,search template API>>'s request body.
include::multi-search.asciidoc[tag=header-params]
`<body>`::
(Request, object) Parameters for the search.
+
=====
include::search-template-api.asciidoc[tag=body-params]
=====
[[multi-search-template-api-response-codes]]
==== {api-response-codes-title}
The API returns a `400` status code only if the request itself fails. If one or
more searches in the request fail, the API returns a `200` status code with an
`error` object for each failed search in the response.
[[multi-search-template-api-response-body]]
==== {api-response-body-title}
`responses`::
(array of objects) Results for each search, returned in the order submitted.
Each object uses the same properties as the <<search-search,search API>>'s
response.
+
If a search fails, the response includes an `error` object containing an error
message.
[[multi-search-template-api-curl-requests]]
==== curl requests
If a providing text file or text input to `curl`, use the `--data-binary` flag
instead of `-d` to preserve newlines.
[source,sh]
----
$ cat requests
{ "index": "my-index" }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ "index": "my-other-index" }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}
$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
----
// NOTCONSOLE

View file

@ -168,9 +168,10 @@ respective types in the response.
The request body contains a newline-delimited list of search `<header>` and
search `<body>` objects.
// tag::header-params[]
`<header>`::
(Required, object)
Contains parameters used to limit or change the subsequent search body request.
Parameters used to limit or change the search.
+
This object is required for each search body but can be empty (`{}`) or a blank
line.
@ -235,6 +236,7 @@ Documents are scored using global term and document frequencies across all
shards. This is usually slower but more accurate.
--
====
// end::header-params[]
`<body>`::
(Optional, object)
@ -335,76 +337,6 @@ all search requests.
See <<url-access-control>>
[[template-msearch]]
==== Template support
Much like described in <<search-template>> for the _search resource, _msearch
also provides support for templates. Submit them like follows for inline
templates:
[source,console]
-----------------------------------------------
GET _msearch/template
{"index" : "my-index-000001"}
{ "source" : "{ \"query\": { \"match\": { \"message\" : \"{{keywords}}\" } } } }", "params": { "query_type": "match", "keywords": "some message" } }
{"index" : "my-index-000001"}
{ "source" : "{ \"query\": { \"match_{{template}}\": {} } }", "params": { "template": "all" } }
-----------------------------------------------
// TEST[setup:my_index]
You can also create search templates:
[source,console]
------------------------------------------
POST /_scripts/my_template_1
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
}
}
}
}
------------------------------------------
// TEST[setup:my_index]
[source,console]
------------------------------------------
POST /_scripts/my_template_2
{
"script": {
"lang": "mustache",
"source": {
"query": {
"term": {
"{{field}}": "{{value}}"
}
}
}
}
}
------------------------------------------
// TEST[continued]
You can use search templates in a _msearch:
[source,console]
-----------------------------------------------
GET _msearch/template
{"index" : "main"}
{ "id": "my_template_1", "params": { "query_string": "some message" } }
{"index" : "main"}
{ "id": "my_template_2", "params": { "field": "user", "value": "test" } }
-----------------------------------------------
// TEST[continued]
[[multi-search-partial-responses]]
==== Partial responses

View file

@ -205,7 +205,7 @@ GET /my-index-000001/_rank_eval
<3> a reference to a previously defined template
<4> the parameters to use to fill the template
It is also possible to use <<pre-registered-templates,templates already stored>> in the cluster state by referencing their id in the templates section.
You can also use a <<create-search-template,stored search template>>.
[source,js]
--------------------------------

View file

@ -0,0 +1,89 @@
[[render-search-template-api]]
=== Render search template API
++++
<titleabbrev>Render search template</titleabbrev>
++++
Renders a <<search-template,search
template>> as a <<search-search,search request body>>.
////
[source,console]
----
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
},
"params": {
"query_string": "My query string"
}
}
}
----
// TESTSETUP
////
[source,console]
----
POST _render/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 20,
"size": 10
}
}
----
[[render-search-template-api-request]]
==== {api-request-title}
`GET _render/template`
`GET _render/template/<template-id>`
`POST _render/template`
`POST _render/template/<template-id>`
[[render-search-template-api-prereqs]]
==== {api-prereq-title}
* If the {es} {security-features} are enabled, you must have the `read`
<<privileges-list-indices,index privilege>> for at least one index pattern.
[[render-search-template-api-path-params]]
==== {api-path-parms-title}
`<template-id>`::
(Required*, string) ID of the search template to render. If no `source` is
specified, this or the `id` request body parameter is required.
[[render-search-template-api-request-body]]
==== {api-request-body-title}
`id`::
(Required*, string) ID of the search template to render. If no `source` is
specified, this or the `<template-id>` request path parameter is required. If
you specify both this parameter and the `<template-id>` parameter, the API uses
only `<template-id>`.
`params`::
(Optional, object) Key-value pairs used to replace Mustache variables in the
template. The key is the variable name. The value is the variable value.
`source`::
(Required*, object) An inline search template. Supports the same parameters as
the <<search-search,search API>>'s request body. These parameters also support
https://mustache.github.io/[Mustache] variables. If no `id` or `<templated-id>`
is specified, this parameter is required.

View file

@ -0,0 +1,148 @@
[[search-template-api]]
=== Search template API
++++
<titleabbrev>Search template</titleabbrev>
++++
Runs a search with a <<search-template,search template>>.
////
[source,console]
----
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
},
"params": {
"query_string": "My query string"
}
}
}
PUT my-index/_doc/1?refresh
{
"message": "hello world"
}
----
// TESTSETUP
////
[source,console]
----
GET my-index/_search/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 0,
"size": 10
}
}
----
[[search-template-api-request]]
==== {api-request-title}
`GET <target>/_search/template`
`GET _search/template`
`POST <target>/_search/template`
`POST _search/template`
[[search-template-api-prereqs]]
==== {api-prereq-title}
* If the {es} {security-features} are enabled, you must have the `read`
<<privileges-list-indices,index privilege>> for the target data stream, index,
or alias. For cross-cluster search, see <<cross-cluster-configuring>>.
[[search-template-api-path-params]]
==== {api-path-parms-title}
`<target>`::
(Optional, string) Comma-separated list of data streams, indices, and aliases to
search. Supports wildcards (`*`). To search all data streams and indices, omit
this parameter or use `*`.
[[search-template-api-query-params]]
==== {api-query-parms-title}
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
+
Defaults to `true`.
`ccs_minimize_roundtrips`::
(Optional, Boolean) If `true`, network round-trips are minimized for
cross-cluster search requests. Defaults to `true`.
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
`explain`::
(Optional, Boolean) If `true`, the response includes additional details about
score computation as part of a hit. Defaults to `false`.
`ignore_throttled`::
(Optional, Boolean) If `true`, specified concrete, expanded, or aliased indices
are not included in the response when throttled. Defaults to `true`.
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=preference]
`rest_total_hits_as_int`::
(Optional, Boolean) If `true`, the response returns `hits.total` as an integer.
If false, it returns `hits.total` as an object. Defaults to `false`.
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=routing]
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=scroll]
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=search_type]
`typed_keys`::
(Optional, Boolean) If `true`, the response prefixes aggregation and suggester
names with their respective types. Defaults to `false`.
[[search-template-api-request-body]]
==== {api-request-body-title}
// tag::body-params[]
`explain`::
(Optional, Boolean) If `true`, returns detailed information about score
calculation as part of each hit. Defaults to `false`.
// end::body-params[]
+
If you specify both this and the `explain` query parameter, the API uses only
the query parameter.
// tag::body-params[]
`id`::
(Required*, string) ID of the search template to use. If no `source` is
specified, this parameter is required.
`params`::
(Optional, object) Key-value pairs used to replace Mustache variables in the
template. The key is the variable name. The value is the variable value.
`profile`::
(Optional, Boolean) If `true`, the query execution is profiled. Defaults to
`false`.
`source`::
(Required*, object) An inline search template. Supports the same parameters as
the <<search-search,search API>>'s request body. Also supports
https://mustache.github.io/[Mustache] variables.
+
If no `id` is specified, this parameter is required.
// end::body-params[]

View file

@ -1,725 +0,0 @@
[[search-template]]
=== Search template API
++++
<titleabbrev>Search template</titleabbrev>
++++
Allows you to use the mustache language to pre render search requests.
[source,console]
------------------------------------------
GET _search/template
{
"source" : {
"query": { "match" : { "{{my_field}}" : "{{my_value}}" } },
"size" : "{{my_size}}"
},
"params" : {
"my_field" : "message",
"my_value" : "foo",
"my_size" : 5
}
}
------------------------------------------
// TEST[setup:my_index]
[[search-template-api-request]]
==== {api-request-title}
`GET _search/template`
[[search-template-api-prereqs]]
==== {api-prereq-title}
* If the {es} {security-features} are enabled, you must have the `read`
<<privileges-list-indices,index privilege>> for the target data stream, index,
or alias. For cross-cluster search, see <<cross-cluster-configuring>>.
[[search-template-api-desc]]
==== {api-description-title}
The `/_search/template` endpoint allows you to use the mustache language to pre-
render search requests, before they are executed and fill existing templates
with template parameters.
For more information on how Mustache templating and what kind of templating you
can do with it check out the https://mustache.github.io/mustache.5.html[online
documentation of the mustache project].
NOTE: The mustache language is implemented in {es} as a sandboxed scripting
language, hence it obeys settings that may be used to enable or disable scripts
per type and context as described in the
<<allowed-script-types-setting, scripting docs>>.
[[search-template-api-path-params]]
==== {api-path-parms-title}
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index]
[[search-template-api-query-params]]
==== {api-query-parms-title}
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
+
Defaults to `true`.
`ccs_minimize_roundtrips`::
(Optional, Boolean) If `true`, network round-trips are minimized for
cross-cluster search requests. Defaults to `true`.
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
`explain`::
(Optional, Boolean) If `true`, the response includes additional details about
score computation as part of a hit. Defaults to `false`.
`ignore_throttled`::
(Optional, Boolean) If `true`, specified concrete, expanded or aliased indices
are not included in the response when throttled. Defaults to `true`.
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=preference]
`profile`::
(Optional, Boolean) If `true`, the query execution is profiled. Defaults
to `false`.
`rest_total_hits_as_int`::
(Optional, Boolean) If `true`, `hits.total` are rendered as an integer in
the response. Defaults to `false`.
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=routing]
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=scroll]
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=search_type]
`typed_keys`::
(Optional, Boolean) If `true`, aggregation and suggester names are
prefixed by their respective types in the response. Defaults to `false`.
[[search-template-api-request-body]]
==== {api-request-body-title}
The API request body must contain the search definition template and its parameters.
[[search-template-api-example]]
==== {api-examples-title}
[[pre-registered-templates]]
===== Store a search template
To store a search template, use the <<create-stored-script-api,create stored
script API>>. Specify `mustache` as the `lang`.
[source,console]
------------------------------------------
POST _scripts/<templateid>
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"title": "{{query_string}}"
}
}
}
}
}
------------------------------------------
// TEST[continued]
//////////////////////////
The API returns the following result if the template has been successfully
created:
[source,console-result]
--------------------------------------------------
{
"acknowledged" : true
}
--------------------------------------------------
//////////////////////////
To retrieve the template, use the <<get-stored-script-api,get stored script
API>>.
[source,console]
------------------------------------------
GET _scripts/<templateid>
------------------------------------------
// TEST[continued]
The API returns:
[source,console-result]
------------------------------------------
{
"script" : {
"lang" : "mustache",
"source" : """{"query":{"match":{"title":"{{query_string}}"}}}""",
"options": {
"content_type" : "application/json;charset=utf-8"
}
},
"_id": "<templateid>",
"found": true
}
------------------------------------------
To delete the template, use the <<delete-stored-script-api,delete stored script
API>>.
[source,console]
------------------------------------------
DELETE _scripts/<templateid>
------------------------------------------
// TEST[continued]
[[use-registered-templates]]
===== Using a stored search template
To use a stored template at search time send the following request:
[source,console]
------------------------------------------
GET _search/template
{
"id": "<templateid>", <1>
"params": {
"query_string": "search for these words"
}
}
------------------------------------------
// TEST[catch:missing]
<1> Name of the stored template script.
[[_validating_templates]]
==== Validating a search template
A template can be rendered in a response with given parameters by using the
following request:
[source,console]
------------------------------------------
GET _render/template
{
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
"params": {
"statuses" : {
"status": [ "pending", "published" ]
}
}
}
------------------------------------------
The API returns the rendered template:
[source,console-result]
------------------------------------------
{
"template_output": {
"query": {
"terms": {
"status": [ <1>
"pending",
"published"
]
}
}
}
}
------------------------------------------
<1> `status` array has been populated with values from the `params` object.
Stored templates can also be rendered by calling the following request:
[source,js]
------------------------------------------
GET _render/template/<template_name>
{
"params": {
"..."
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-explain-parameter]]
===== Using the explain parameter
You can use the `explain` parameter when running a template:
[source,console]
------------------------------------------
GET _search/template
{
"id": "my_template",
"params": {
"status": [ "pending", "published" ]
},
"explain": true
}
------------------------------------------
// TEST[catch:missing]
[[search-template-profile-parameter]]
===== Profiling
You can use the `profile` parameter when running a template:
[source,console]
------------------------------------------
GET _search/template
{
"id": "my_template",
"params": {
"status": [ "pending", "published" ]
},
"profile": true
}
------------------------------------------
// TEST[catch:missing]
[[search-template-query-string-single]]
===== Filling in a query string with a single value
[source,console]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"term": {
"message": "{{query_string}}"
}
}
},
"params": {
"query_string": "search for these words"
}
}
------------------------------------------
// TEST[setup:my_index]
[[search-template-converting-to-json]]
===== Converting parameters to JSON
The `{{#toJson}}parameter{{/toJson}}` function can be used to convert parameters
like maps and array to their JSON representation:
[source,console]
------------------------------------------
GET _search/template
{
"source": "{ \"query\": { \"terms\": {{#toJson}}statuses{{/toJson}} }}",
"params": {
"statuses" : {
"status": [ "pending", "published" ]
}
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query": {
"terms": {
"status": [
"pending",
"published"
]
}
}
}
------------------------------------------
// NOTCONSOLE
A more complex example substitutes an array of JSON objects:
[source,console]
------------------------------------------
GET _search/template
{
"source": "{\"query\":{\"bool\":{\"must\": {{#toJson}}clauses{{/toJson}} }}}",
"params": {
"clauses": [
{ "term": { "user" : "foo" } },
{ "term": { "user" : "bar" } }
]
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query": {
"bool": {
"must": [
{
"term": {
"user": "foo"
}
},
{
"term": {
"user": "bar"
}
}
]
}
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-concatenate-array]]
===== Concatenating array of values
The `{{#join}}array{{/join}}` function can be used to concatenate the
values of an array as a comma delimited string:
[source,console]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"match": {
"emails": "{{#join}}emails{{/join}}"
}
}
},
"params": {
"emails": [ "username@email.com", "lastname@email.com" ]
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query" : {
"match" : {
"emails" : "username@email.com,lastname@email.com"
}
}
}
------------------------------------------
// NOTCONSOLE
The function also accepts a custom delimiter:
[source,console]
------------------------------------------
GET _search/template
{
"source": {
"query": {
"range": {
"born": {
"gte" : "{{date.min}}",
"lte" : "{{date.max}}",
"format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
}
}
}
},
"params": {
"date": {
"min": "2016",
"max": "31/12/2017",
"formats": ["dd/MM/yyyy", "yyyy"]
}
}
}
------------------------------------------
which is rendered as:
[source,js]
------------------------------------------
{
"query": {
"range": {
"born": {
"gte": "2016",
"lte": "31/12/2017",
"format": "dd/MM/yyyy||yyyy"
}
}
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-default-values]]
===== Default values
A default value is written as `{{var}}{{^var}}default{{/var}}` for instance:
[source,js]
------------------------------------------
{
"source": {
"query": {
"range": {
"line_no": {
"gte": "{{start}}",
"lte": "{{end}}{{^end}}20{{/end}}"
}
}
}
},
"params": { ... }
}
------------------------------------------
// NOTCONSOLE
When `params` is `{ "start": 10, "end": 15 }` this query would be rendered as:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "15"
}
}
}
------------------------------------------
// NOTCONSOLE
But when `params` is `{ "start": 10 }` this query would use the default value
for `end`:
[source,js]
------------------------------------------
{
"range": {
"line_no": {
"gte": "10",
"lte": "20"
}
}
}
------------------------------------------
// NOTCONSOLE
[[search-template-conditional-clauses]]
===== Conditional clauses
Conditional clauses cannot be expressed using the JSON form of the template.
Instead, the template *must* be passed as a string. For instance, let's say
we wanted to run a `match` query on the `line` field, and optionally wanted
to filter by line numbers, where `start` and `end` are optional.
The `params` would look like:
[source,js]
------------------------------------------
{
"params": {
"text": "words to search for",
"line_no": { <1>
"start": 10,
"end": 20
}
}
}
------------------------------------------
// NOTCONSOLE
<1> The `line_no`, `start`, and `end` parameters are optional.
When written as a query, the template would include invalid JSON, such as
section markers like `{{#line_no}}`:
[source,js]
------------------------------------------
{
"query": {
"bool": {
"must": {
"match": {
"line": "{{text}}" <1>
}
},
"filter": {
{{#line_no}} <2>
"range": {
"line_no": {
{{#start}} <3>
"gte": "{{start}}" <4>
{{#end}},{{/end}} <5>
{{/start}}
{{#end}} <6>
"lte": "{{end}}" <7>
{{/end}}
}
}
{{/line_no}}
}
}
}
}
------------------------------------------
// NOTCONSOLE
<1> Fill in the value of param `text`
<2> Include the `range` filter only if `line_no` is specified
<3> Include the `gte` clause only if `line_no.start` is specified
<4> Fill in the value of param `line_no.start`
<5> Add a comma after the `gte` clause only if `line_no.start`
AND `line_no.end` are specified
<6> Include the `lte` clause only if `line_no.end` is specified
<7> Fill in the value of param `line_no.end`
Because search templates cannot include invalid JSON, you can pass the same
query as a string instead:
[source,js]
--------------------
"source": "{\"query\":{\"bool\":{\"must\":{\"match\":{\"line\":\"{{text}}\"}},\"filter\":{{{#line_no}}\"range\":{\"line_no\":{{{#start}}\"gte\":\"{{start}}\"{{#end}},{{/end}}{{/start}}{{#end}}\"lte\":\"{{end}}\"{{/end}}}}{{/line_no}}}}}}"
--------------------
// NOTCONSOLE
[[search-template-encode-urls]]
===== Encoding URLs
The `{{#url}}value{{/url}}` function can be used to encode a string value
in a HTML encoding form as defined in by the
https://www.w3.org/TR/html4/[HTML specification].
As an example, it is useful to encode a URL:
[source,console]
------------------------------------------
GET _render/template
{
"source": {
"query": {
"term": {
"http_access_log": "{{#url}}{{host}}/{{page}}{{/url}}"
}
}
},
"params": {
"host": "https://www.elastic.co/",
"page": "learn"
}
}
------------------------------------------
The previous query will be rendered as:
[source,console-result]
------------------------------------------
{
"template_output": {
"query": {
"term": {
"http_access_log": "https%3A%2F%2Fwww.elastic.co%2F%2Flearn"
}
}
}
}
------------------------------------------
[[multi-search-template]]
=== Multi search template API
++++
<titleabbrev>Multi search template</titleabbrev>
++++
Allows to execute several search template requests.
[[multi-search-template-api-request]]
==== {api-request-title}
`GET _msearch/template`
[[multi-search-template-api-prereqs]]
==== {api-prereq-title}
* If the {es} {security-features} are enabled, you must have the `read`
<<privileges-list-indices,index privilege>> for the target data stream, index,
or alias. For cross-cluster search, see <<cross-cluster-configuring>>.
[[multi-search-template-api-desc]]
==== {api-description-title}
Allows to execute several search template requests within the same API using the
`_msearch/template` endpoint.
The format of the request is similar to the <<search-multi-search, Multi
Search API>> format:
[source,js]
--------------------------------------------------
header\n
body\n
header\n
body\n
--------------------------------------------------
// NOTCONSOLE
The header part supports the same `index`, `search_type`, `preference`, and
`routing` options as the Multi Search API.
The body includes a search template body request and supports inline, stored and
file templates.
[[multi-search-template-api-example]]
==== {api-examples-title}
[source,js]
--------------------------------------------------
$ cat requests
{"index": "test"}
{"source": {"query": {"match": {"user" : "{{username}}" }}}, "params": {"username": "john"}} <1>
{"source": {"query": {"{{query_type}}": {"name": "{{name}}" }}}, "params": {"query_type": "match_phrase_prefix", "name": "Smith"}}
{"index": "_all"}
{"id": "template_1", "params": {"query_string": "search for these words" }} <2>
$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch/template --data-binary "@requests"; echo
--------------------------------------------------
// NOTCONSOLE
// Not converting to console because this shows how curl works
<1> Inline search template request
<2> Search template request based on a stored template
The response returns a `responses` array, which includes the search template
response for each search template request matching its order in the original
multi search template request. If there was a complete failure for that specific
search template request, an object with `error` message will be returned in
place of the actual search response.

View file

@ -0,0 +1,632 @@
[[search-template]]
== Search templates
A search template is a stored search you can run with different variables.
If you use {es} as a search backend, you can pass user input from a search bar
as parameters for a search template. This lets you run searches without exposing
{es}'s query syntax to your users.
If you use {es} for a custom application, search templates let you change
your searches without modifying your app's code.
[discrete]
[[create-search-template]]
=== Create a search template
To create or update a search template, use the <<create-stored-script-api,create
stored script API>>.
The request's `source` supports the same parameters as the
<<search-search-api-request-body,search API>>'s request body. `source` also
supports https://mustache.github.io/[Mustache] variables, typically enclosed in
double curly brackets: `{{my-var}}`. When you run a templated search, {es}
replaces these variables with values from `params`.
Search templates must use a `lang` of `mustache`.
The following request creates a search template with an `id` of
`my-search-template`.
[source,console]
----
PUT _scripts/my-search-template
{
"script": {
"lang": "mustache",
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
},
"params": {
"query_string": "My query string"
}
}
}
----
{es} stores search templates as Mustache <<modules-scripting,scripts>> in the
cluster state. {es} compiles search templates in the `template` script context.
Settings that limit or disable scripts also affect search templates.
[discrete]
[[validate-search-template]]
=== Validate a search template
[[_validating_templates]]
To test a template with different `params`, use the
<<render-search-template-api,render search template API>>.
[source,console]
----
POST _render/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 20,
"size": 10
}
}
----
// TEST[continued]
When rendered, the template outputs a <<search-search-api-request-body,search
request body>>.
[source,console-result]
----
{
"template_output": {
"query": {
"match": {
"message": "hello world"
}
},
"from": "20",
"size": "10"
}
}
----
You can also use the API to test inline templates.
[source,console]
----
POST _render/template
{
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}",
"size": "{{size}}"
},
"params": {
"query_string": "hello world",
"from": 20,
"size": 10
}
}
----
// TEST[continued]
[discrete]
[[run-templated-search]]
=== Run a templated search
To run a search with a search template, use the <<search-template-api,search
template API>>. You can specify different `params` with each request.
////
[source,console]
----
PUT my-index/_doc/1?refresh
{
"message": "hello world"
}
----
// TEST[continued]
////
[source,console]
----
GET my-index/_search/template
{
"id": "my-search-template",
"params": {
"query_string": "hello world",
"from": 0,
"size": 10
}
}
----
// TEST[continued]
The response uses the same properties as the <<search-search,search API>>'s
response.
[source,console-result]
----
{
"took": 36,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.5753642,
"hits": [
{
"_index": "my-index",
"_id": "1",
"_score": 0.5753642,
"_source": {
"message": "hello world"
}
}
]
}
}
----
// TESTRESPONSE[s/"took": 36/"took": "$body.took"/]
[discrete]
[[run-multiple-templated-searches]]
=== Run multiple templated searches
To run multiple templated searches with a single request, use the
<<multi-search-template,multi search template API>>. These requests often have
less overhead and faster speeds than multiple individual searches.
[source,console]
----
GET my-index/_msearch/template
{ }
{ "id": "my-search-template", "params": { "query_string": "hello world", "from": 0, "size": 10 }}
{ }
{ "id": "my-other-search-template", "params": { "query_type": "match_all" }}
----
// TEST[continued]
// TEST[s/my-other-search-template/my-search-template/]
[discrete]
[[get-search-templates]]
=== Get search templates
To retrieve a search template, use the <<get-stored-script-api,get stored
script API>>.
[source,console]
----
GET _scripts/my-search-template
----
// TEST[continued]
To get a list of all search templates and other stored scripts, use the
<<cluster-state,cluster state API>>.
[source,console]
----
GET _cluster/state/metadata?pretty&filter_path=metadata.stored_scripts
----
// TEST[continued]
[discrete]
[[delete-search-template]]
=== Delete a search template
To delete a search template, use the <<delete-stored-script-api,delete stored
script API>>.
[source,console]
----
DELETE _scripts/my-search-template
----
// TEST[continued]
[discrete]
[[search-template-set-default-values]]
=== Set default values
To set a default value for a variable, use the following syntax:
[source,mustache]
----
{{my-var}}{{^my-var}}default value{{/my-var}}
----
If a templated search doesn't specify a value in its `params`, the search uses
the default value instead. For example, the following template sets defaults
for `from` and `size`.
[source,console]
----
POST _render/template
{
"source": {
"query": {
"match": {
"message": "{{query_string}}"
}
},
"from": "{{from}}{{^from}}0{{/from}}",
"size": "{{size}}{{^size}}10{{/size}}"
},
"params": {
"query_string": "hello world"
}
}
----
[discrete]
[[search-template-url-encode-strings]]
=== URL encode strings
Use the `{{#url}}` function to URL encode a string.
[source,console]
----
POST _render/template
{
"source": {
"query": {
"term": {
"url.full": "{{#url}}{{host}}/{{page}}{{/url}}"
}
}
},
"params": {
"host": "http://example.com",
"page": "hello-world"
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output": {
"query": {
"term": {
"url.full": "http%3A%2F%2Fexample.com%2Fhello-world"
}
}
}
}
----
[discrete]
[[search-template-concatenate-values]]
=== Concatenate values
Use the `{{#join}}` function to concatenate array values as a comma-delimited
string. For example, the following template concatenates two email addresses.
[source,console]
----
POST _render/template
{
"source": {
"query": {
"match": {
"user.group.emails": "{{#join}}emails{{/join}}"
}
}
},
"params": {
"emails": [ "user1@example.com", "user_one@example.com" ]
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output": {
"query": {
"match": {
"user.group.emails": "user1@example.com,user_one@example.com"
}
}
}
}
----
You can also specify a custom delimiter.
[source,console]
----
POST _render/template
{
"source": {
"query": {
"range": {
"user.effective.date": {
"gte": "{{date.min}}",
"lte": "{{date.max}}",
"format": "{{#join delimiter='||'}}date.formats{{/join delimiter='||'}}"
}
}
}
},
"params": {
"date": {
"min": "2098",
"max": "06/05/2099",
"formats": ["dd/MM/yyyy", "yyyy"]
}
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output": {
"query": {
"range": {
"user.effective.date": {
"gte": "2098",
"lte": "06/05/2099",
"format": "dd/MM/yyyy||yyyy"
}
}
}
}
}
----
[discrete]
[[search-template-convert-json]]
=== Convert to JSON
Use the `{{#toJson}}` function to convert a variable value to its JSON
representation.
For example, the following template uses `{{#toJson}}` to pass an array. To
ensure the request body is valid JSON, the `source` is written in the string
format.
[source,console]
----
POST _render/template
{
"source": "{ \"query\": { \"terms\": { \"tags\": {{#toJson}}tags{{/toJson}} }}}",
"params": {
"tags": [
"prod",
"es01"
]
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output": {
"query": {
"terms": {
"tags": [
"prod",
"es01"
]
}
}
}
}
----
You can also use `{{#toJson}}` to pass objects.
[source,console]
----
POST _render/template
{
"source": "{ \"query\": {{#toJson}}my_query{{/toJson}} }",
"params": {
"my_query": {
"match_all": { }
}
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output" : {
"query" : {
"match_all" : { }
}
}
}
----
You can also pass an array of objects.
[source,console]
----
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"must\": {{#toJson}}clauses{{/toJson}} }}}",
"params": {
"clauses": [
{
"term": {
"user.id": "kimchy"
}
},
{
"term": {
"url.domain": "example.com"
}
}
]
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output": {
"query": {
"bool": {
"must": [
{
"term": {
"user.id": "kimchy"
}
},
{
"term": {
"url.domain": "example.com"
}
}
]
}
}
}
}
----
[discrete]
[[search-template-use-conditions]]
=== Use conditions
To create if conditions, use the following syntax:
[source,mustache]
----
{{#condition}}content{{/condition}}
----
If the condition variable is `true`, {es} displays its content. For example, the
following template searches data from the past year if `year_scope` is `true`.
[source,console]
----
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"filter\": [ {{#year_scope}} { \"range\": { \"@timestamp\": { \"gte\": \"now-1y/d\", \"lt\": \"now/d\" } } }, {{/year_scope}} { \"term\": { \"user.id\": \"{{user_id}}\" }}]}}}",
"params": {
"year_scope": true,
"user_id": "kimchy"
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output" : {
"query" : {
"bool" : {
"filter" : [
{
"range" : {
"@timestamp" : {
"gte" : "now-1y/d",
"lt" : "now/d"
}
}
},
{
"term" : {
"user.id" : "kimchy"
}
}
]
}
}
}
}
----
If `year_scope` is `false`, the template searches data from any time period.
[source,console]
----
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"filter\": [ {{#year_scope}} { \"range\": { \"@timestamp\": { \"gte\": \"now-1y/d\", \"lt\": \"now/d\" } } }, {{/year_scope}} { \"term\": { \"user.id\": \"{{user_id}}\" }}]}}}",
"params": {
"year_scope": false,
"user_id": "kimchy"
}
}
----
The template renders as:
[source,console-result]
----
{
"template_output" : {
"query" : {
"bool" : {
"filter" : [
{
"term" : {
"user.id" : "kimchy"
}
}
]
}
}
}
}
----
To create if-else conditions, use the following syntax:
[source,mustache]
----
{{#condition}}if content{{/condition}} {{^condition}}else content{{/condition}}
----
For example, the following template searches data from the past year if
`year_scope` is `true`. Otherwise, it searches data from the past day.
[source,console]
----
POST _render/template
{
"source": "{ \"query\": { \"bool\": { \"filter\": [ { \"range\": { \"@timestamp\": { \"gte\": {{#year_scope}} \"now-1y/d\" {{/year_scope}} {{^year_scope}} \"now-1d/d\" {{/year_scope}} , \"lt\": \"now/d\" }}}, { \"term\": { \"user.id\": \"{{user_id}}\" }}]}}}",
"params": {
"year_scope": true,
"user_id": "kimchy"
}
}
----

View file

@ -524,4 +524,5 @@ include::retrieve-selected-fields.asciidoc[]
include::search-across-clusters.asciidoc[]
include::search-multiple-indices.asciidoc[]
include::search-shard-routing.asciidoc[]
include::search-template.asciidoc[]
include::sort-search-results.asciidoc[]

View file

@ -206,8 +206,8 @@ The response contains suggestions scored by the most likely spelling correction
Checks each suggestion against the specified `query` to prune suggestions
for which no matching docs exist in the index. The collate query for a
suggestion is run only on the local shard from which the suggestion has
been generated from. The `query` must be specified and it can be templated,
see <<search-template,search templates>> for more information.
been generated from. The `query` must be specified and it can be templated.
See <<search-template>>.
The current suggestion is automatically made available as the `{{suggestion}}`
variable, which should be used in your query. You can still specify
your own template `params` -- the `suggestion` value will be added to the

View file

@ -1,7 +1,7 @@
{
"render_search_template":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/current/search-template.html#_validating_templates",
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/current/render-search-template-api.html",
"description":"Allows to use the Mustache language to pre-render a search definition."
},
"stability":"stable",

View file

@ -96,7 +96,7 @@ The following table lists all available settings for the search
The search {watcher-transform} support mustache <<templates, templates>>. This
can either be as part of the body definition or alternatively point to an
existing template (either defined in a file or
<<pre-registered-templates,registered>> as a script in Elasticsearch).
<<create-search-template,stored>> as a script in Elasticsearch).
For example, the following snippet shows a search that refers to the scheduled
time of the watch: