mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-25 07:37:19 -04:00
230acb8ac5
Fixes two bugs in _resolve/cluster. First, the code that detects older clusters versions and does a fallback to the _resolve/index endpoint was using an outdated string match for error detection. That has been adjusted. Second, upon security exceptions, the _resolve/cluster endpoint was marking the clusters as connected: true, under the assumption that all security exceptions related to cross cluster calls and remote index access were coming from the remote cluster, but that is not always the case. Some cross-cluster security violations can be detected on the local querying cluster after issuing the remoteClient.execute call but before the transport layer actually sends the request remotely. So we now mark the connected status as false for all ElasticsearchSecurityException cases. End user docs have been updated with this information.
286 lines
9.6 KiB
Text
286 lines
9.6 KiB
Text
[[indices-resolve-cluster-api]]
|
|
=== Resolve cluster API
|
|
++++
|
|
<titleabbrev>Resolve cluster</titleabbrev>
|
|
++++
|
|
|
|
.New API reference
|
|
[sidebar]
|
|
--
|
|
For the most up-to-date API details, refer to {api-es}/group/endpoint-indices[Index APIs].
|
|
--
|
|
|
|
Resolves the specified index expressions to return information about
|
|
each cluster, including the local cluster, if included.
|
|
|
|
This endpoint is useful before doing a <<modules-cross-cluster-search,{ccs}>> in
|
|
order to determine which remote clusters should be included in a search.
|
|
|
|
You use the same index expression with this endpoint as you would for cross-cluster
|
|
search. Index and <<exclude-problematic-clusters,cluster exclusions>> are also supported
|
|
with this endpoint.
|
|
|
|
For each cluster in the index expression, information is returned about:
|
|
|
|
1. whether the querying ("local") cluster is currently connected to each remote cluster
|
|
in the index expression scope
|
|
2. whether each remote cluster is configured with `skip_unavailable` as `true` or `false`
|
|
3. whether there are any indices, aliases or data streams on that cluster that match
|
|
the index expression
|
|
4. whether the search is likely to have errors returned when you do the {ccs} (including any
|
|
authorization errors if your user does not have permission to query a remote cluster or
|
|
the indices on that cluster)
|
|
5. (in some cases) cluster version information, including the Elasticsearch server version
|
|
|
|
[TIP]
|
|
====
|
|
Whenever a security exception is returned for a remote cluster, that remote
|
|
will always be marked as connected=false in the response, since your user does not have
|
|
permissions to access that cluster (or perhaps the remote index) you are querying.
|
|
Once the proper security permissions are obtained, then you can rely on the `connected` field
|
|
in the response to determine whether the remote cluster is available and ready for querying.
|
|
====
|
|
|
|
|
|
////
|
|
[source,console]
|
|
--------------------------------
|
|
PUT _cluster/settings
|
|
{
|
|
"persistent": {
|
|
"cluster": {
|
|
"remote": {
|
|
"cluster_one": {
|
|
"seeds": [
|
|
"35.238.149.1:9300"
|
|
],
|
|
"skip_unavailable": true
|
|
},
|
|
"cluster_two": {
|
|
"seeds": [
|
|
"35.238.149.2:9300"
|
|
],
|
|
"skip_unavailable": false
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
--------------------------------
|
|
// TEST[setup:host]
|
|
// TEST[s/35.238.149.\d+:930\d+/\${transport_host}/]
|
|
////
|
|
|
|
[source,console]
|
|
----
|
|
GET /_resolve/cluster/my-index-*,cluster*:my-index-*
|
|
----
|
|
// TEST[continued]
|
|
|
|
This will return information about the local cluster and all remotely configured
|
|
clusters that start with the alias `cluster*`. Each cluster will return information
|
|
about whether it has any indices, aliases or data streams that match `my-index-*`.
|
|
|
|
[[resolve-cluster-api-request]]
|
|
==== {api-request-title}
|
|
|
|
`GET /_resolve/cluster/<index_expression>`
|
|
|
|
[[resolve-cluster-api-prereqs]]
|
|
==== {api-prereq-title}
|
|
|
|
* If the {es} {security-features} are enabled, you must have the
|
|
`view_index_metadata`, `read`, or `manage` <<privileges-list-indices,index
|
|
privilege>> for the target data stream, index, or alias.
|
|
|
|
[[resolve-cluster-api-path-params]]
|
|
==== {api-path-parms-title}
|
|
|
|
`<index_expression>`::
|
|
+
|
|
--
|
|
(Required, string) Comma-separated name(s) or index pattern(s) of the
|
|
indices, aliases, and data streams to resolve, using <<api-multi-index>>.
|
|
Resources on <<remote-clusters,remote clusters>> can be specified using the
|
|
`<cluster>:<name>` syntax.
|
|
--
|
|
|
|
[[resolve-cluster-api-query-params]]
|
|
==== {api-query-parms-title}
|
|
|
|
include::{es-ref-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
|
|
+
|
|
Defaults to `open`.
|
|
|
|
include::{es-ref-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
|
|
+
|
|
Defaults to `false`.
|
|
|
|
include::{es-ref-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
|
|
+
|
|
Defaults to `true`.
|
|
|
|
`ignore_throttled`::
|
|
(Optional, Boolean) If `true`, concrete, expanded or aliased indices are
|
|
ignored when frozen. Defaults to `false`.
|
|
+
|
|
deprecated:[7.16.0]
|
|
|
|
|
|
[discrete]
|
|
[[usecases-for-resolve-cluster]]
|
|
=== Advantages of using this endpoint before a {ccs}
|
|
|
|
You may want to exclude a cluster or index from a search when:
|
|
|
|
1. A remote cluster is not currently connected and is configured with `skip_unavailable`=`false`.
|
|
Executing a {ccs} under those conditions will cause
|
|
<<cross-cluster-search-failures,the entire search to fail>>.
|
|
|
|
2. A cluster has no matching indices, aliases or data streams for the index expression
|
|
(or your user does not have permissions to search them). For example, suppose your
|
|
index expression is `logs*,remote1:logs*` and the `remote1` cluster has no indices, aliases or data
|
|
streams that match `logs*`. In that case, that cluster will return no results from that cluster if
|
|
you include it in a {ccs}.
|
|
|
|
3. The index expression (combined with any query parameters you specify) will likely cause an exception
|
|
to be thrown when you do the search. In these cases, the "error" field in the `_resolve/cluster` response
|
|
will be present. (This is also where security/permission errors will be shown.)
|
|
|
|
4. A remote cluster is an older version that does not support the feature you want to
|
|
use in your search.
|
|
|
|
|
|
[[resolve-cluster-api-example]]
|
|
==== {api-examples-title}
|
|
|
|
[source,console]
|
|
----
|
|
GET /_resolve/cluster/my-index*,clust*:my-index*
|
|
----
|
|
// TEST[continued]
|
|
// TEST[setup:my_index]
|
|
|
|
The API returns the following response:
|
|
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"(local)": { <1>
|
|
"connected": true,
|
|
"skip_unavailable": false,
|
|
"matching_indices": true,
|
|
"version": {
|
|
"number": "8.13.0",
|
|
"build_flavor": "default",
|
|
"minimum_wire_compatibility_version": "7.17.0",
|
|
"minimum_index_compatibility_version": "7.0.0"
|
|
}
|
|
},
|
|
"cluster_one": {
|
|
"connected": true, <2>
|
|
"skip_unavailable": true, <3>
|
|
"matching_indices": true, <4>
|
|
"version": {
|
|
"number": "8.13.0", <5>
|
|
"build_flavor": "default",
|
|
"minimum_wire_compatibility_version": "7.17.0",
|
|
"minimum_index_compatibility_version": "7.0.0"
|
|
}
|
|
},
|
|
"cluster_two": {
|
|
"connected": true,
|
|
"skip_unavailable": false,
|
|
"matching_indices": true,
|
|
"version": {
|
|
"number": "8.13.0",
|
|
"build_flavor": "default",
|
|
"minimum_wire_compatibility_version": "7.17.0",
|
|
"minimum_index_compatibility_version": "7.0.0"
|
|
}
|
|
}
|
|
}
|
|
----
|
|
// TESTRESPONSE[s/"number": "8.13.0"/"number": "$body.$_path"/]
|
|
// TESTRESPONSE[s/"minimum_wire_compatibility_version": "7.17.0"/"minimum_wire_compatibility_version": "$body.$_path"/]
|
|
// TESTRESPONSE[s/"minimum_index_compatibility_version": "7.0.0"/"minimum_index_compatibility_version": "$body.$_path"/]
|
|
|
|
<1> Each cluster has its own response section. The cluster you sent the request to is labelled as "(local)".
|
|
<2> The querying cluster attempts to make a request to each remote cluster. If successful, `connected`=`true`.
|
|
<3> The `skip_unavailable` setting for each remote cluster, as configured on the local cluster.
|
|
<4> Indicates whether any index, alias or data stream matches the index expression specified for that cluster.
|
|
<5> The Elasticsearch server version.
|
|
|
|
|
|
[discrete]
|
|
[[resolve-cluster-api-error-example]]
|
|
==== Identifying potential problems with your {ccs}
|
|
|
|
The following request shows several examples of how modifying your query can
|
|
prevent search failures.
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
GET /_resolve/cluster/not-present,clust*:my-index*,oldcluster:*?ignore_unavailable=false
|
|
--------------------------------------------------
|
|
// TEST[continued]
|
|
// TEST[s/,oldcluster:*//]
|
|
|
|
[source,console-result]
|
|
--------------------------------------------------
|
|
{
|
|
"(local)": {
|
|
"connected": true,
|
|
"skip_unavailable": false,
|
|
"error": "no such index [not_present]" <1>
|
|
},
|
|
"cluster_one": {
|
|
"connected": true,
|
|
"skip_unavailable": true,
|
|
"matching_indices": false, <2>
|
|
"version": {
|
|
"number": "8.13.0",
|
|
"build_flavor": "default",
|
|
"minimum_wire_compatibility_version": "7.17.0",
|
|
"minimum_index_compatibility_version": "7.0.0"
|
|
}
|
|
},
|
|
"cluster_two": {
|
|
"connected": false, <3>
|
|
"skip_unavailable": false,
|
|
"matching_indices": true,
|
|
"version": {
|
|
"number": "8.13.0",
|
|
"build_flavor": "default",
|
|
"minimum_wire_compatibility_version": "7.17.0",
|
|
"minimum_index_compatibility_version": "7.0.0"
|
|
}
|
|
},
|
|
"oldcluster": { <4>
|
|
"connected": true,
|
|
"skip_unavailable": false,
|
|
"matching_indices": true
|
|
}
|
|
}
|
|
--------------------------------------------------
|
|
// TEST[skip: too many illustrative error variations to reproduce]
|
|
|
|
<1> The local cluster has no index called `not_present`. Searching against it
|
|
using the specified `ignore_unavailable=false` param will return a "no such
|
|
index" error. Other types of errors can show up here as well, such as security
|
|
permission errors when the user does not have authorization to search the
|
|
specified index.
|
|
<2> The `cluster_one` remote cluster has no indices that match the pattern
|
|
`my-index*`. There may be no indices that match the pattern or the index
|
|
could be closed. (You can check this by using the
|
|
<<indices-resolve-index-api,resolve index>> API.)
|
|
<3> The `cluster_two` remote cluster is not connected (the attempt to connect
|
|
failed). Since this cluster is marked as `skip_unavailable=false`, you should
|
|
probably exclude this cluster from the search by adding `-cluster_two:*` to the
|
|
search index expression.
|
|
<4> The `oldcluster` remote cluster shows that it has matching indices, but no
|
|
version information is included. This indicates that the cluster version predates
|
|
the introduction of the `_resolve/cluster` API in 8.13.0., so you may want to
|
|
exclude it from your {ccs}. (Note: the endpoint was able to tell there were
|
|
matching indices because it fell back to using the <<indices-resolve-index-api,
|
|
resolve index>> API.)
|