github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-04-25 15:47:23 -04:00
Code Issues Projects Releases Packages Wiki Activity

Add service for computing the optimal number of shards for data streams (#105498)

Browse source 
This adds the `DataStreamAutoShardingService` that will compute the
optimal number of shards for a data stream and return a recommendation
as to when to apply it (a time interval we call cool down which is 0
when the auto sharding recommendation can be applied immediately).

This also introduces a `DataStreamAutoShardingEvent` object that will be
stored in the data stream metadata to indicate the last auto sharding
event that was applied to a data stream and its cluster state
representation looks like so:

```
"auto_sharding": {
 "trigger_index_name": ".ds-logs-nginx-2024.02.12-000002",
 "target_number_of_shards": 3,
 "event_timestamp": 1707739707954
}
```

The auto sharding service is not used in this PR, so the auto sharding
event will not be stored in the data stream metadata, but the required
infrastructure to configure it is in place.
This commit is contained in:
Andrei Dan 2024-03-06 10:12:08 +00:00 • committed by GitHub
parent fa735a9b77
commit 882b92ab60
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
25 changed files with 1681 additions and 119 deletions
Download patch file Download diff file Expand all files Collapse all files

1
server/src/main/java/module-info.java
View file

@ -381,6 +381,7 @@ module org.elasticsearch.server {
opens org.elasticsearch.common.logging to org.apache.logging.log4j.core;
exports org.elasticsearch.action.datastreams.lifecycle;
exports org.elasticsearch.action.datastreams.autosharding;
exports org.elasticsearch.action.downsample;
exports org.elasticsearch.plugins.internal
to