Add an option to allow reranking to fail, and the docs to pass through as-is.
Exposing the error to users and adding documentation is a later piece of work.
Features added before 8.18 can be removed, starting with 9.0. But first they need to be marked as assumed, so existing code knows they could be removed in later builds.
This updates the gradle wrapper to 8.12
We addressed deprecation warnings due to the update that includes:
- Fix change in TestOutputEvent api
- Fix deprecation in groovy syntax
- Use latest ospackage plugin containing our fix
- Remove project usages at execution time
- Fix deprecated project references in repository-old-versions
This pull request introduces a new retriever called `rescorer`, which leverages the `rescore` functionality of the search request.
The `rescorer` retriever re-scores only the top documents retrieved by its child retriever, offering fine-tuned scoring capabilities.
All rescorers supported in the `rescore` section of a search request are available in this retriever, and the same format is used to define the rescore configuration.
<details>
<summary>Example:</summary>
```yaml
- do:
search:
index: test
body:
retriever:
rescorer:
rescore:
window_size: 10
query:
rescore_query:
rank_feature:
field: "features.second_stage"
linear: { }
query_weight: 0
retriever:
standard:
query:
rank_feature:
field: "features.first_stage"
linear: { }
size: 2
```
</details>
Closes#118327
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
* Added query name to inference field metadata
* Fix build error
* Added query builder service
* Add query builder service to query rewrite context
* Updated match query to support querying semantic text fields
* Fix build error
* Fix NPE
* Update the POC to rewrite to a bool query when combined inference and non-inference fields
* Separate clause for each inference index (to avoid inference ID clashes)
* Simplify query builder service concept to a single default inference query
* Rename QueryBuilderService, remove query name from inference metadata
* Fix too many rewrite rounds error by injecting booleans in constructors for match query builder and semantic text
* Fix test compilation errors
* Fix tests
* Add yaml test for semantic match
* Add NodeFeature
* Fix license headers
* Spotless
* Updated getClass comparison in MatchQueryBuilder
* Cleanup
* Add Mock Inference Query Builder Service
* Spotless
* Cleanup
* Update docs/changelog/117839.yaml
* Update changelog
* Replace the default inference query builder with a query rewrite interceptor
* Cleanup
* Some more cleanup/renames
* Some more cleanup/renames
* Spotless
* Checkstyle
* Convert List<QueryRewriteInterceptor> to Map keyed on query name, error on query name collisions
* PR feedback - remove check on QueryRewriteContext class only
* PR feedback
* Remove intercept flag from MatchQueryBuilder and replace with wrapper
* Move feature to test feature
* Ensure interception happens only once
* Rename InterceptedQueryBuilderWrapper to AbstractQueryBuilderWrapper
* Add lenient field to SemanticQueryBuilder
* Clean up yaml test
* Add TODO comment
* Add comment
* Spotless
* Rename AbstractQueryBuilderWrapper back to InterceptedQueryBuilderWrapper
* Spotless
* Didn't mean to commit that
* Remove static class wrapping the InterceptedQueryBuilderWrapper
* Make InterceptedQueryBuilderWrapper part of QueryRewriteInterceptor
* Refactor the interceptor to be an internal plugin that cannot be used outside inference plugin
* Fix tests
* Spotless
* Minor cleanup
* C'mon spotless
* Test spotless
* Cleanup InternalQueryRewriter
* Change if statement to assert
* Simplify template of InterceptedQueryBuilderWrapper
* Change constructor of InterceptedQueryBuilderWrapper
* Refactor InterceptedQueryBuilderWrapper to extend QueryBuilder
* Cleanup
* Add test
* Spotless
* Rename rewrite to interceptAndRewrite in QueryRewriteInterceptor
* DOESN'T WORK - for testing
* Add comment
* Getting closer - match on single typed fields works now
* Deleted line by mistake
* Checkstyle
* Fix over-aggressive IntelliJ Refactor/Rename
* And another one
* Move SemanticMatchQueryRewriteInterceptor.SEMANTIC_MATCH_QUERY_REWRITE_INTERCEPTION_SUPPORTED to Test feature
* PR feedback
* Require query name with no default
* PR feedback & update test
* Add rewrite test
* Update server/src/main/java/org/elasticsearch/index/query/InnerHitContextBuilder.java
Co-authored-by: Mike Pellegrini <mike.pellegrini@elastic.co>
---------
Co-authored-by: Mike Pellegrini <mike.pellegrini@elastic.co>
The most relevant ES changes that upgrading to Lucene 10 requires are:
- use the appropriate IOContext
- Scorer / ScorerSupplier breaking changes
- Regex automaton are no longer determinized by default
- minimize moved to test classes
- introduce Elasticsearch900Codec
- adjust slicing code according to the added support for intra-segment concurrency
- disable intra-segment concurrency in tests
- adjust accessor methods for many Lucene classes that became a record
- adapt to breaking changes in the analysis area
Co-authored-by: Christoph Büscher <christophbuescher@posteo.de>
Co-authored-by: Mayya Sharipova <mayya.sharipova@elastic.co>
Co-authored-by: ChrisHegarty <chegar999@gmail.com>
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: Armin Braun <me@obrown.io>
Co-authored-by: Panagiotis Bailis <pmpailis@gmail.com>
Co-authored-by: Benjamin Trent <4357155+benwtrent@users.noreply.github.com>
This PR ensures that all retriever applies the rewrite to all their rewriteable.
Rewriting eagerly at the retriever level ensures that we don't rewrite the same query multiple times
when compound retrievers are used.
