Currently pinned queries require either the `ids` or `docs` parameter.
`docs` allows pinning documents from specific indices. However for
`docs` the `_index` field is always required:
```
GET test/_search
{
"query": {
"pinned": {
"organic": {
"query_string": {
"query": "something"
}
},
"docs": [
{ "_id": "1" }
]
}
}
}
```
returns an error:
```
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[10:22] [pinned] failed to parse field [docs]",
"line": 10,
"col": 22
}
],
"type": "parsing_exception",
"reason": "[10:22] [pinned] failed to parse field [docs]",
"line": 10,
"col": 22,
"caused_by": {
"type": "x_content_parse_exception",
"reason": "[10:22] [pinned] failed to parse field [docs]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "Required [_index]"
}
}
},
"status": 400
}
```
The proposal here is to make `_index` optional. I don't think we have a
strong requirement for making `_index` required, when it was initially
introduced in https://github.com/elastic/elasticsearch/pull/74873, we
mostly wanted the ability to pin docs from specific indices.
Making `_index` optional can give more flexibility to use a combination
of pinned documents from specific indices or just document ids. This
change can also help with pinned query rules. Currently pinned query
rules can accept either `ids` or `docs`. If multiple pinned query rules
match and they use a combination of `ids` and `docs`, we cannot build a
pinned query and we would need to return an error. This is because a
pinned query cannot accept both `ids` and `docs`. By making `_index`
optional we would no longer need to return an error when pinned query
rules use a combination of `ids` and `docs`, because we can easily
translate `ids` in `docs`.
The following pinned queries would be equivalent:
```
GET test/_search
{
"query": {
"pinned": {
"organic": {
"query_string": {
"query": "something"
}
},
"docs": [
{ "_id": "1" }
]
}
}
}
GET test/_search
{
"query": {
"pinned": {
"organic": {
"query_string": {
"query": "something"
}
},
"ids": [1]
}
}
}
```
The scores should be consistent when using a combination of _docs that
might use _index or not - see example
<details> <summary>Example </summary>
```
PUT test-1/_doc/1 { "title": "doc 1" }
PUT test-1/_doc/2 { "title": "doc 2" }
PUT test-2/_doc/1 { "title": "doc 1" }
PUT test-2/_doc/3 { "title": "lalala" }
POST test-1,test-2/_search { "query": { "pinned": {
"organic": { "query_string": { "query": "lalala"
} }, "docs": [ { "_id": "2", "_index": "test-1" },
{ "_id": "1" } ] } } }
```
response:
```
{ "took": 1, "timed_out": false, "_shards": { "total": 2,
"successful": 2, "skipped": 0, "failed": 0 }, "hits": {
"total": { "value": 4, "relation": "eq" },
"max_score": 1.7014124e+38, "hits": [ { "_index":
"test-1", "_id": "2", "_score": 1.7014124e+38,
"_source": { "title": "doc 2" } }, {
"_index": "test-1", "_id": "1", "_score": 1.7014122e+38,
// same score as doc with id 1 from test-2 "_source": {
"title": "doc 1" } }, { "_index": "test-2",
"_id": "1", "_score": 1.7014122e+38, // same score as doc with
id 1 from test-1 "_source": { "title": "doc 1"
} }, { "_index": "test-2", "_id": "3",
"_score": 0.8025915, // organic result "_source": {
"title": "lalala" } } ] } }
```
</details>
For query rules, if we have two query rules that both match and use a
combination of `ids` and `pinned`:
```
PUT _query_rules/test-ruleset
{
"ruleset_id": "test-ruleset",
"rules": [
{
"rule_id": "1",
"type": "pinned",
"criteria": [
{
"type": "exact",
"metadata": "query_string",
"value": "country"
}
],
"actions": {
"docs": [
{ "_index": "singers", "_id": "1" }
]
}
},
{
"rule_id": "2",
"type": "pinned",
"criteria": [
{
"type": "exact",
"metadata": "query_string",
"value": "country"
}
],
"actions": {
"ids": [
2
]
}
}
]
}
```
and the following query:
```
POST singers/_search
{
"query": {
"rule_query": {
"organic": {
"query_string": {
"default_field": "name",
"query": "country"
}
},
"match_criteria": {
"query_string": "country"
},
"ruleset_id": "test-ruleset"
}
}
}
```
then this would get translated into the following pinned query:
```
POST singers/_search
{
"query": {
"pinned": {
"organic": {
"query_string": {
"default_field": "name",
"query": "country"
}
},
"docs": [
{ "_index": "singers", "_id": "1" },
{"_id": 2 }
]
}
}
}
```
I think we can also simplify the pinned query rule so that it only
receives `docs`:
```
PUT _query_rules/test-ruleset
{
"ruleset_id": "test-ruleset",
"rules": [
{
"rule_id": "1",
"type": "pinned",
"criteria": [
{
"type": "exact",
"metadata": "query_string",
"value": "country"
}
],
"actions": {
"docs": [
{ "_id": "1" },
{ "_id": "2", "_index": "singers" }
]
}
}
]
}
```
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.
Relates to #79309, #31619
The current `ids` option doesn't allow pinning a specific document in a
single index when searching over multiple indices. This introduces a
`documents` option, which is an array of `_id` and `_index`
fields to allow index-specific pins.
Closes https://github.com/elastic/elasticsearch/issues/67855.
Search enhancement: - new query type allows selected documents to be promoted above any "organic” search results.
This is the first feature in a new module `search-business-rules` which will house licensed (non OSS) logic for rewriting queries according to business rules.
The PinnedQueryBuilder class offers a new `pinned` query in the DSL that takes an array of promoted IDs and an “organic” query and ensures the documents with the promoted IDs rank higher than the organic matches.
Closes#44074
