elasticsearch/dev-tools
Chris Hegarty 6b52d7837b
Add an optimised int8 vector distance function for aarch64. (#106133)
This commit adds an optimised int8 vector distance implementation for aarch64. Additional platforms like, say, x64, will be added as a follow-up.

The vector distance implementation outperforms Lucene's Pamana Vector implementation for binary comparisons by approx 5x (depending on the number of dimensions). It does so by means of compiler intrinsics built into a separate native library and link by Panama's FFI. Comparisons are performed on off-heap mmap'ed vector data.

The implementation is currently only used during merging of scalar quantized segments, through a custom format ES814HnswScalarQuantizedVectorsFormat, but its usage will likely be expanded over time.

Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
Co-authored-by: Lorenzo Dematté <lorenzo.dematte@elastic.co>
Co-authored-by: Mark Vieira <portugee@gmail.com>
Co-authored-by: Ryan Ernst <ryan@iernst.net>
2024-04-12 08:44:21 +01:00
..
atomic_push.sh Improve atomic push script to support worktrees (#68873) 2021-02-11 10:13:15 -08:00
delete_branch_jobs.sh Add convenience script for pruning old dev branch CI jobs 2022-08-24 09:59:38 -07:00
prepare_release_update_documentation.py Update sources with new SSPL+Elastic-2.0 license headers 2021-02-02 16:10:53 -08:00
publish_zstd_binaries.sh Add an optimised int8 vector distance function for aarch64. (#106133) 2024-04-12 08:44:21 +01:00
zstd.Dockerfile Add an optimised int8 vector distance function for aarch64. (#106133) 2024-04-12 08:44:21 +01:00