github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-04-25 07:37:19 -04:00
Code Issues Projects Releases Packages Wiki Activity

[DOCS] Fixes formatting issue on dense vector reference page. (#127214)

Browse source
This commit is contained in:
István Zoltán Szabó 2025-04-23 11:24:17 +02:00 • committed by GitHub
parent 0ea2a748d4
commit 1e7c6abaf6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 4 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

16
docs/reference/elasticsearch/mapping-reference/dense-vector.md
View file

@ -6,7 +6,6 @@ mapped_pages:
# Dense vector field type [dense-vector]
The `dense_vector` field type stores dense vectors of numeric values. Dense vector fields are primarily used for [k-nearest neighbor (kNN) search](docs-content://deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md).
The `dense_vector` type does not support aggregations or sorting.
@ -46,7 +45,6 @@ PUT my-index/_doc/2
Unlike most other data types, dense vectors are always single-valued. It is not possible to store multiple values in one `dense_vector` field.
::::
## Index vectors for kNN search [index-vectors-knn-search]
A *k-nearest neighbor* (kNN) search finds the *k* nearest vectors to a query vector, as measured by a similarity metric.
@ -78,7 +76,6 @@ PUT my-index-2
Indexing vectors for approximate kNN search is an expensive process. It can take substantial time to ingest documents that contain vector fields with `index` enabled. See [k-nearest neighbor (kNN) search](docs-content://deploy-manage/production-guidance/optimize-performance/approximate-knn-search.md) to learn more about the memory requirements.
::::
You can disable indexing by setting the `index` parameter to `false`:
```console
@ -98,7 +95,6 @@ PUT my-index-2
{{es}} uses the [HNSW algorithm](https://arxiv.org/abs/1603.09320) to support efficient kNN search. Like most kNN algorithms, HNSW is an approximate method that sacrifices result accuracy for improved speed.
## Automatically quantize vectors for kNN search [dense-vector-quantization]
The `dense_vector` type supports quantization to reduce the memory footprint required when [searching](docs-content://solutions/search/vector/knn.md#approximate-knn) `float` vectors. The three following quantization strategies are supported:
@ -117,17 +113,14 @@ Quantized vectors can use [oversampling and rescoring](docs-content://solutions/
Quantization will continue to keep the raw float vector values on disk for reranking, reindexing, and quantization improvements over the lifetime of the data. This means disk usage will increase by ~25% for `int8`, ~12.5% for `int4`, and ~3.1% for `bbq` due to the overhead of storing the quantized and raw vectors.
::::
::::{note}
`int4` quantization requires an even number of vector dimensions.
::::
::::{note}
`bbq` quantization only supports vector dimensions that are greater than 64.
::::
Here is an example of how to create a byte-quantized index:
```console
@ -188,7 +181,6 @@ PUT my-byte-quantized-index
}
```
## Parameters for dense vector fields [dense-vector-params]
The following mapping parameters are accepted:
@ -210,7 +202,6 @@ $$$dense-vector-element-type$$$
::::
`dims`
: (Optional, integer) Number of vector dimensions. Can’t exceed `4096`. If `dims` is not specified, it will be set to the length of the first vector added to the field.
@ -228,7 +219,6 @@ $$$dense-vector-similarity$$$
`bit` vectors only support `l2_norm` as their similarity metric.
::::
::::{dropdown} Valid values for `similarity`
`l2_norm`
: Computes similarity based on the L2 distance (also known as Euclidean distance) between the vectors. The document `_score` is computed as `1 / (1 + l2_norm(query, vector)^2)`.
@ -242,7 +232,6 @@ For `bit` vectors, instead of using `l2_norm`, the `hamming` distance between th
When `element_type` is `byte`, all vectors must have the same length including both document and query vectors or results will be inaccurate. The document `_score` is computed as `0.5 + (dot_product(query, vector) / (32768 * dims))` where `dims` is the number of dimensions per vector.
`cosine`
: Computes the cosine similarity. During indexing {{es}} automatically normalizes vectors with `cosine` similarity to unit length. This allows to internally use `dot_product` for computing similarity, which is more efficient. Original un-normalized vectors can be still accessed through scripts. The document `_score` is computed as `(1 + cosine(query, vector)) / 2`. The `cosine` similarity does not allow vectors with zero magnitude, since cosine is not defined in this case.
@ -251,12 +240,10 @@ For `bit` vectors, instead of using `l2_norm`, the `hamming` distance between th
::::
::::{note}
Although they are conceptually related, the `similarity` parameter is different from [`text`](/reference/elasticsearch/mapping-reference/text.md) field [`similarity`](/reference/elasticsearch/mapping-reference/similarity.md) and accepts a distinct set of options.
::::
$$$dense-vector-index-options$$$
`index_options`
@ -267,7 +254,6 @@ $$$dense-vector-index-options$$$
::::{dropdown} Properties of `index_options`
`type`
: (Required, string) The type of kNN algorithm to use. Can be either any of:
* `hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) for scalable approximate kNN search. This supports all `element_type` values.
* `int8_hnsw` - The default index type for float vectors. This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 4x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
* `int4_hnsw` - This utilizes the [HNSW algorithm](https://arxiv.org/abs/1603.09320) in addition to automatically scalar quantization for scalable approximate kNN search with `element_type` of `float`. This can reduce the memory footprint by 8x at the cost of some accuracy. See [Automatically quantize vectors for kNN search](#dense-vector-quantization).
@ -277,7 +263,6 @@ $$$dense-vector-index-options$$$
* `int4_flat` - This utilizes a brute-force search algorithm in addition to automatically half-byte scalar quantization. Only supports `element_type` of `float`.
* `bbq_flat` - This utilizes a brute-force search algorithm in addition to automatically binary quantization. Only supports `element_type` of `float`.
`m`
: (Optional, integer) The number of neighbors each node will be connected to in the HNSW graph. Defaults to `16`. Only applicable to `hnsw`, `int8_hnsw`, `int4_hnsw` and `bbq_hnsw` index types.
@ -287,6 +272,7 @@ $$$dense-vector-index-options$$$
`confidence_interval`
: (Optional, float) Only applicable to `int8_hnsw`, `int4_hnsw`, `int8_flat`, and `int4_flat` index types. The confidence interval to use when quantizing the vectors. Can be any value between and including `0.90` and `1.0` or exactly `0`. When the value is `0`, this indicates that dynamic quantiles should be calculated for optimized quantization. When between `0.90` and `1.0`, this value restricts the values used when calculating the quantization thresholds. For example, a value of `0.95` will only use the middle 95% of the values when calculating the quantization thresholds (e.g. the highest and lowest 2.5% of values will be ignored). Defaults to `1/(dims + 1)` for `int8` quantized vectors and `0` for `int4` for dynamic quantile calculation.
`rescore_vector`
: (Optional, object) An optional section that configures automatic vector rescoring on knn queries for the given field. Only applicable to quantized index types.
:::::{dropdown} Properties of `rescore_vector`