github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-04-24 15:17:30 -04:00
Code Issues Projects Releases Packages Wiki Activity

Support cluster/details for CCS minimize_roundtrips=false (#98457)

Browse source 
This commit tracks progress for each shard search by cluster alias
using a new SearchProgressListener (CCSSingleCoordinatorSearchProgressListener).
Both sync and async CCS searches use this new progress listener when
minimize_roundtrips=false.

Two of the SearchProgressListener method had to be extended to allow tracking
per-cluster took values (TransportSearchAction.SearchTimeProvider) and
whether searches timed out (by passing in QuerySearchResult to the onQueryResult
listener method).

This commit brings parity between minimize_roundtrips=true and false to have
the same _cluster/details sections in CCS search responses.

Note that there are still a few differences between minimize_roundtrips=true and false.
1. The per-cluster took value for minimize_roundtrips=true is accurate, but the
   for 'false' it is only measured at the granualarity of each partial reduce,
   so the per cluster took time is overestimated in basically all cases.
2. For minimize_roundtrips=true, a skip_unavailable=false cluster that disconnects
   during the search or has all searches on all shards fail, will cause the entire
   search to fail. This is (still) not true for minimize_roundtrips=false. The search
   is only failed if the skip_unavailable=false cluster cannot be connected to at the
   start of the search. (This will likely be changed in a follow up ticket that implements
   fail-fast logic for in-progress searches that should fail due to a skip_unavailable=true
   cluster failing.)
3. The shard accounting for minimize_roundtrips=false is always accurate (total shard counts
   are known at the start of the search). For minimize_roundtrips=true, the shard accounting
   is only accurate per cluster unless all clusters have successful (or partially successful)
   searches. For clusters that have failures we do not have shard count info.
This commit is contained in:
Michael Peterson 2023-08-31 12:56:20 -04:00 • committed by GitHub
parent e31eed37c5
commit 649821e992
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
25 changed files with 2040 additions and 504 deletions
Download patch file Download diff file Expand all files Collapse all files

2
qa/multi-cluster-search/src/test/java/org/elasticsearch/search/CCSDuelIT.java
View file

@ -1075,7 +1075,6 @@ public class CCSDuelIT extends ESRestTestCase {
assertEquals(clustersMRT.getSuccessful(), clustersMRTFalse.getSuccessful());
assertEquals(clustersMRT.getSkipped(), clustersMRTFalse.getSkipped());
boolean removeSkipped = searchRequest.source().collapse() != null;
Map<String, Object> minimizeRoundtripsResponseMap = responseToMap(minimizeRoundtripsSearchResponse);
if (clustersMRT.hasClusterObjects() && clustersMRTFalse.hasClusterObjects()) {
Map<String, Object> fanOutResponseMap = responseToMap(fanOutSearchResponse);
@ -1148,7 +1147,6 @@ public class CCSDuelIT extends ESRestTestCase {
assertEquals(clustersMRT.getSuccessful(), clustersMRTFalse.getSuccessful());
assertEquals(clustersMRT.getSkipped(), clustersMRTFalse.getSkipped());
boolean removeSkipped = searchRequest.source().collapse() != null;
Map<String, Object> minimizeRoundtripsResponseMap = responseToMap(minimizeRoundtripsSearchResponse);
if (clustersMRT.hasClusterObjects() && clustersMRTFalse.hasClusterObjects()) {
Map<String, Object> fanOutResponseMap = responseToMap(fanOutSearchResponse);