Add new int8_flat and flat vector index types (#104872)

This adds two new vector index types:  - flat   - int8_flat

Both store the vectors in a flat space and search is brute-force over
the vectors in the index.   For the regular `flat` index, this can be
considered syntactic sugar that allows `knn` queries without having to
put indices within HNSW. 

For `int8_flat`, this allows float vectors to be stored in a flat
manner, but also automatically quantized.
This commit is contained in:
Benjamin Trent 2024-02-05 12:56:13 -05:00 committed by GitHub
parent 4376bdb2f1
commit 43362d5de5
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
13 changed files with 1093 additions and 6 deletions

View file

@ -424,6 +424,10 @@ module org.elasticsearch.server {
org.elasticsearch.index.codec.bloomfilter.ES87BloomFilterPostingsFormat,
org.elasticsearch.index.codec.postings.ES812PostingsFormat;
provides org.apache.lucene.codecs.DocValuesFormat with ES87TSDBDocValuesFormat;
provides org.apache.lucene.codecs.KnnVectorsFormat
with
org.elasticsearch.index.codec.vectors.ES813FlatVectorFormat,
org.elasticsearch.index.codec.vectors.ES813Int8FlatVectorFormat;
exports org.elasticsearch.cluster.routing.allocation.shards
to