When originally added, the knn query didn't apply `top-k` restrictions
to the query. Instead it would allow the resulting `num_candidate` to be
combined with sibling queries without restricting to `top-size` results
ahead of time.
This honestly is confusing behavior and leads to some bugs in understand
how it all works.
This commit addresses this by eagerly gathering only `size` results when
`k==null` before combining with other queries.
To achieve the previous behavior, this can be done directly by setting
`k==num_candidates` in the query.
* Term Stats documentation
* Update docs/reference/reranking/learning-to-rank-model-training.asciidoc
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
* Fix query example.
---------
Co-authored-by: István Zoltán Szabó <istvan.szabo@elastic.co>
Since https://github.com/apache/lucene-solr/pull/620, intervals disjunctions are automatically rewritten to handle cases where minimizations can miss valid matches.
This change updates the documentation to take this behaviour into account (users don't need to manually pull intervals disjunctions to the top anymore).
* Cleanup: Remove pinned IDs from applied rules in favor of single applied docs
* Add support for query rules of type exclude, to exclude specified documents from result sets
* Support exluded documents that specify the _index as well as the _id
* Cleanup
* Update docs/changelog/111420.yaml
* Update docs
* Spotless
* PR feedback - docs updates
* Apply PR feedback
* PR feedback
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
indices.query.bool.max_clause_count is set automatically and does
not default to 4096 as before. This remove mentions of 4096
from query documentations.
Relates to PR#91811
Introduce an optional k param for knn query
If k is not set, knn query has the previous behaviour:
- `num_candidates` docs is collected from each shard. This `num_candidates` docs
are used for combining with results with other queries and aggregations on each shard.
- docs from all shards are merged to produce the top global `size` results
If k is set, the behaviour instead is following:
- `k` docs is collected from each shard. This `k` docs are used for
combining results with other queries and aggregations on each shard.
- similarly, docs from all shards are merged to produce the top global `size`
results.
Having `k` param makes it more intuitive for users to address their needs.
They also don't need to care and can skip `num_candidates` param for this query
as it is of more internal details to tune how knn search operates.
Closes#108473
* Remove `es-test-dir` book-scoped variable
* Remove `plugins-examples-dir` book-scoped variable
* Remove `:dependencies-dir:` and `:xes-repo-dir:` book-scoped variables
- In `index.asciidoc`, two variables (`:dependencies-dir:` and `:xes-repo-dir:`) were removed.
- In `sql/index.asciidoc`, the `:sql-tests:` path was updated to fuller path
- In `esql/index.asciidoc`, the `:esql-tests:` path was updated idem
* Replace `es-repo-dir` with `es-ref-dir`
* Move `:include-xpack: true` to few files that use it, remove from index.asciidoc
* Add modelId and modelText to KnnVectorQueryBuilder
Use QueryVectorBuilder within KnnVectorQueryBuilder to make it
possible to perform knn queries also when a query vector is not
immediately available. Supplying a text_embedding query_vector_builder
with model_text and model_id instead of the query_vector will result
in the generation of a query_vector by calling inference on the
specified model_id with the supplied model_text (during query
rewrite). This is consistent with the way query vectors are built
from model_id / model_text in KnnSearchBuilder (DFS phase).
This enhancement adds a new abstraction to the _search API called "retriever." A
retriever is something that returns top hits. This adds three initial retrievers called
"standard", "knn", and "rrf". The retrievers use a parser-only approach where they
are parsed and then translated into a SearchSourceBuilder to execute the actual
search.
---------
Co-authored-by: Mayya Sharipova <mayya.sharipova@elastic.co>
With this commit we amend the docs for the `exists` query to clarify
that it works with either `index` *or* `doc_values` set to `true` in the
mapping. Only if both are disabled, the `exists` query won't work.
Improvement includes:
1. Remove reference to Lucene queries (this information is not necessary
for Elastic users, and can be outdated)
2. For `span_field_masking` include a node to use
"require_field_match" : false parameter for highlighters to work.
Closes#101804
Return matched_queries for named queries in Percolator.
In a response, each hit together with
a `_percolator_document_slot` field will contain
`_percolator_document_slot_<slotNumber>_matched_queries` fields that will show
which sub-queries matched each percolated document.
Closes#10163
