github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-06-28 09:28:55 -04:00
Code Issues Projects Releases Packages Wiki Activity
elasticsearch/qa/ccs-common-rest
History
Exact
Chris Hegarty 19550a838f
Add dense vector off-heap stats to Node stats and Index stats APIs (#126704) 
This change enhances the dense_vector section of the Nodes stats and Index stats APIs so that they report the desired size of off-heap memory for all indexed vectors. The dense_vector section of the Custer stats API remains unchanged.

The retrieval mechanism and structure of the new stats is the same across the various three stats APIs, but more fine-grained information is disclosed as when moving from Cluster -> Node -> Index API.

For Node stats, we aggregate the total byte sizes for all vectors, categorised by the data type. For example:

"dense_vector" : {
  "value_count" : 5,
  "off_heap" : {
    "total_size_in_bytes" : 27,
    "total_veb_size_in_bytes" : 3,
    "total_vec_size_in_bytes" : 23,
    "total_veq_size_in_bytes" : 0,
    "total_vex_size_in_bytes" : 1
  }
}
Index stats: same as Node stats with included field break down . For example:

"dense_vector" : {
  "value_count" : 5,
  "off_heap" : {
    "total_size_in_bytes" : 27,
    "total_veb_size_in_bytes" : 3,
    "total_vec_size_in_bytes" : 23,
    "total_veq_size_in_bytes" : 0,
    "total_vex_size_in_bytes" : 1,
    "fielddata" : {
      "bar" : {
        "veb_size_in_bytes" : 3,
        "vec_size_in_bytes" : 14,
        "vex_size_in_bytes" : 1
      },
      "foo" : {
        "vec_size_in_bytes" : 9
      }
    }
  }
The implementation accesses the actual statistics through reflection. This will be completely removed when Lucene exposes this, which is expected in Lucene 10.3
2025-04-23 15:04:44 +01:00
..
src/yamlRestTest Add feature flag for subobjects auto (#114616) 2024-10-12 18:55:27 +02:00
build.gradle Add dense vector off-heap stats to Node stats and Index stats APIs (#126704) 2025-04-23 15:04:44 +01:00