github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-06-28 17:34:17 -04:00
Code Issues Projects Releases Packages Wiki Activity

Docs for hot-reloadable remote cluster credentials (#105483)

Browse source 
Docs PR to accompany
https://github.com/elastic/elasticsearch/pull/103215.

Resolves: ES-7625
This commit is contained in:
Nikolaj Volgushev 2024-02-15 12:02:13 +01:00 • committed by GitHub
parent ac2e54d56e
commit e241a91a4e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 37 additions and 33 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/reference/modules/cluster/remote-clusters-api-key.asciidoc
View file

@ -31,7 +31,7 @@ In this model, cross-cluster operations use <<remote_cluster.port,a dedicated
server port>> (remote cluster interface) for communication between clusters. A server port>> (remote cluster interface) for communication between clusters. A
remote cluster must enable this port for local clusters to connect. Configure remote cluster must enable this port for local clusters to connect. Configure
Transport Layer Security (TLS) for this port to maximize security (as explained Transport Layer Security (TLS) for this port to maximize security (as explained
in <<remote-clusters-security-api-key>>). in <<remote-clusters-security-api-key>>).
The local cluster must trust the remote cluster on the remote cluster interface. The local cluster must trust the remote cluster on the remote cluster interface.
This means that the local cluster trusts the remote cluster's certificate This means that the local cluster trusts the remote cluster's certificate
@ -65,15 +65,15 @@ information, refer to https://www.elastic.co/subscriptions.
===== On the remote cluster ===== On the remote cluster
// tag::remote-cluster-steps[] // tag::remote-cluster-steps[]
. Enable the remote cluster server on every node of the remote cluster. In . Enable the remote cluster server on every node of the remote cluster. In
`elasticsearch.yml`: `elasticsearch.yml`:
.. Set <<remote-cluster-network-settings,`remote_cluster_server.enabled`>> to .. Set <<remote-cluster-network-settings,`remote_cluster_server.enabled`>> to
`true`. `true`.
.. Configure the bind and publish address for remote cluster server traffic, for .. Configure the bind and publish address for remote cluster server traffic, for
example using <<remote-cluster-network-settings,`remote_cluster.host`>>. Without example using <<remote-cluster-network-settings,`remote_cluster.host`>>. Without
configuring the address, remote cluster traffic may be bound to the local configuring the address, remote cluster traffic may be bound to the local
interface, and remote clusters running on other machines can't connect. interface, and remote clusters running on other machines can't connect.
.. Optionally, configure the remote server port using .. Optionally, configure the remote server port using
<<remote_cluster.port,`remote_cluster.port`>> (defaults to `9443`). <<remote_cluster.port,`remote_cluster.port`>> (defaults to `9443`).
. Next, generate a certificate authority (CA) and a server certificate/key pair. . Next, generate a certificate authority (CA) and a server certificate/key pair.
On one of the nodes of the remote cluster, from the directory where {es} has On one of the nodes of the remote cluster, from the directory where {es} has
@ -86,8 +86,8 @@ been installed:
./bin/elasticsearch-certutil ca --pem --out=cross-cluster-ca.zip --pass CA_PASSWORD ./bin/elasticsearch-certutil ca --pem --out=cross-cluster-ca.zip --pass CA_PASSWORD
---- ----
+ +
Replace `CA_PASSWORD` with the password you want to use for the CA. You can Replace `CA_PASSWORD` with the password you want to use for the CA. You can
remove the `--pass` option and its argument if you are not deploying to a remove the `--pass` option and its argument if you are not deploying to a
production environment. production environment.
.. Unzip the generated `cross-cluster-ca.zip` file. This compressed file .. Unzip the generated `cross-cluster-ca.zip` file. This compressed file
@ -100,7 +100,7 @@ contains the following content:
|_ ca.key |_ ca.key
---- ----
.. Generate a certificate and private key pair for the nodes in the remote .. Generate a certificate and private key pair for the nodes in the remote
cluster: cluster:
+ +
[source,sh] [source,sh]
@ -183,13 +183,16 @@ Replace `ALIAS` with the same name that you will use to create the remote cluste
later. When prompted, enter the encoded cross-cluster API key created on the later. When prompted, enter the encoded cross-cluster API key created on the
remote cluster earlier. remote cluster earlier.
. Restart the local cluster to load the keystore change. . Restart the local cluster to load changes to the keystore and settings.
**Note:** If you are configuring only the cross-cluster API key, you can call the <<cluster-nodes-reload-secure-settings>> API, instead of restarting the cluster.
Configuring the `remote_cluster_client` settings in `elasticsearch.yml` still requires a restart.
[[remote-clusters-connect-api-key]] [[remote-clusters-connect-api-key]]
==== Connect to a remote cluster ==== Connect to a remote cluster
:trust-mechanism: api-key :trust-mechanism: api-key
include::remote-clusters-connect.asciidoc[] include::remote-clusters-connect.asciidoc[]
:!trust-mechanism: :!trust-mechanism:
include::{es-repo-dir}/security/authentication/remote-clusters-privileges-api-key.asciidoc[leveloffset=+1] include::{es-repo-dir}/security/authentication/remote-clusters-privileges-api-key.asciidoc[leveloffset=+1]

20
docs/reference/modules/cluster/remote-clusters-migration.asciidoc
View file

@ -51,7 +51,7 @@ include::remote-clusters-api-key.asciidoc[tag=remote-cluster-steps]
[[remote-clusters-migration-stop]] [[remote-clusters-migration-stop]]
==== Stop cross-cluster operations ==== Stop cross-cluster operations
On the local cluster, stop any persistent tasks that refer to the remote On the local cluster, stop any persistent tasks that refer to the remote
cluster: cluster:
* Use the <<stop-transform>> API to stop any transforms. * Use the <<stop-transform>> API to stop any transforms.
@ -74,13 +74,13 @@ roles used for cross-cluster operations. You should be able to copy these
privileges from the original roles on the remote cluster, where they are defined privileges from the original roles on the remote cluster, where they are defined
under the certification based security model. under the certification based security model.
** The roles on the local cluster can't exceed the `access` privilege granted by ** The roles on the local cluster can't exceed the `access` privilege granted by
the cross-cluster API key. Any extra local privileges will be suppressed by the the cross-cluster API key. Any extra local privileges will be suppressed by the
cross-cluster API key's privileges. cross-cluster API key's privileges.
** No update is needed if the {ccr} or {ccs} tasks have been configured with a ** No update is needed if the {ccr} or {ccs} tasks have been configured with a
`superuser` role. The `superuser` role is automatically updated to allow access `superuser` role. The `superuser` role is automatically updated to allow access
to all remote indices. to all remote indices.
** Tasks that are run as regular users with named roles are immediately updated ** Tasks that are run as regular users with named roles are immediately updated
with the new privileges. A task will load a new definition the next time it with the new privileges. A task will load a new definition the next time it
runs. runs.
** You need to restart tasks that are run using an API key (done in a later ** You need to restart tasks that are run using an API key (done in a later
step). step).
@ -123,7 +123,7 @@ created on the remote cluster earlier.
. If you've dynamically configured the remote cluster (via the cluster settings . If you've dynamically configured the remote cluster (via the cluster settings
API): API):
.. Restart the local cluster to load changes to the keystore. .. Restart the local cluster to load changes to the keystore and settings.
.. Re-add the remote cluster. Use the same remote cluster alias, and change the .. Re-add the remote cluster. Use the same remote cluster alias, and change the
transport port into the remote cluster port. For example: transport port into the remote cluster port. For example:
@ -188,7 +188,7 @@ remote cluster:
---- ----
// TEST[skip:TODO] // TEST[skip:TODO]
<1> The remote cluster is connected. <1> The remote cluster is connected.
<2> If present, indicates the remote cluster has connected using API key <2> If present, indicates the remote cluster has connected using API key
authentication. authentication.
[[remote-clusters-migration-resume]] [[remote-clusters-migration-resume]]
@ -204,7 +204,7 @@ task will update the task with the updated API key.
* Use the <<start-transform>> API to start any transforms. * Use the <<start-transform>> API to start any transforms.
* Use the <<ml-open-job>> API to open any anomaly detection jobs. * Use the <<ml-open-job>> API to open any anomaly detection jobs.
* Use the <<ccr-post-resume-follow>> API to resume any auto-follow {ccr}. * Use the <<ccr-post-resume-follow>> API to resume any auto-follow {ccr}.
* Use the <<ccr-resume-auto-follow-pattern>> API to resume any manual {ccr} or * Use the <<ccr-resume-auto-follow-pattern>> API to resume any manual {ccr} or
existing indices that were created from the auto-follow pattern. existing indices that were created from the auto-follow pattern.
[[remote-clusters-migration-disable-cert]] [[remote-clusters-migration-disable-cert]]
@ -232,8 +232,8 @@ or distributed, is no longer trusted.
Another solution is to apply IP filters to the transport interface, blocking Another solution is to apply IP filters to the transport interface, blocking
traffic from outside the cluster. traffic from outside the cluster.
. Optionally, delete any roles on the remote cluster that were only used for . Optionally, delete any roles on the remote cluster that were only used for
cross-cluster operations. These roles are no longer used under the API key based cross-cluster operations. These roles are no longer used under the API key based
security model. security model.
[[remote-clusters-migration-rollback]] [[remote-clusters-migration-rollback]]
@ -252,7 +252,7 @@ the migration.
. On each node, remove the `remote_cluster_client.ssl.*` settings from . On each node, remove the `remote_cluster_client.ssl.*` settings from
`elasticsearch.yml`. `elasticsearch.yml`.
. Restart the local cluster to apply changes to the keystore and . Restart the local cluster to apply changes to the keystore and
`elasticsearch.yml`. `elasticsearch.yml`.
. On the local cluster, apply the original remote cluster settings. If the . On the local cluster, apply the original remote cluster settings. If the
@ -263,4 +263,4 @@ remote cluster connection has been configured statically (using the
local cluster has connected to the remote cluster. The response should have local cluster has connected to the remote cluster. The response should have
`"connected": true` and not have `"cluster_credentials": "::es_redacted::"`. `"connected": true` and not have `"cluster_credentials": "::es_redacted::"`.
. Restart any persistent tasks that you've stopped earlier. . Restart any persistent tasks that you've stopped earlier.

5
docs/reference/modules/cluster/remote-clusters-settings.asciidoc
View file

@ -65,7 +65,8 @@ mode are described separately.
is used as the fallback setting. is used as the fallback setting.
`cluster.remote.<cluster_alias>.credentials` (<<secure-settings,Secure>>):: `cluster.remote.<cluster_alias>.credentials` (<<secure-settings,Secure>>, <<reloadable-secure-settings,Reloadable>>)::
[[remote-cluster-credentials-setting]]
beta:[] beta:[]
Per cluster setting for configuring <<remote-clusters-api-key,remote clusters with the API Key based model>>. Per cluster setting for configuring <<remote-clusters-api-key,remote clusters with the API Key based model>>.
@ -75,6 +76,8 @@ beta:[]
The presence (or not) of this setting determines which model a remote cluster uses. The presence (or not) of this setting determines which model a remote cluster uses.
If present, the remote cluster uses the API key based model. If present, the remote cluster uses the API key based model.
Otherwise, it uses the certificate based model. Otherwise, it uses the certificate based model.
If the setting is added, removed, or updated in the <<secure-settings,{es} keystore>> and reloaded via the
<<cluster-nodes-reload-secure-settings>> API, the cluster will automatically rebuild its connection to the remote.
[[remote-cluster-sniff-settings]] [[remote-cluster-sniff-settings]]
==== Sniff mode remote cluster settings ==== Sniff mode remote cluster settings

23
docs/reference/modules/cluster/remote-clusters-troubleshooting.asciidoc
View file

@ -31,13 +31,13 @@ incoming cross-cluster requests by default, while it is ready to send outgoing
cross-cluster requests. Ensure you've enabled the remote cluster server on every cross-cluster requests. Ensure you've enabled the remote cluster server on every
node of the remote cluster. In `elasticsearch.yml`: node of the remote cluster. In `elasticsearch.yml`:
* Set <<remote-cluster-network-settings,`remote_cluster_server.enabled`>> to * Set <<remote-cluster-network-settings,`remote_cluster_server.enabled`>> to
`true`. `true`.
* Configure the bind and publish address for remote cluster server traffic, for * Configure the bind and publish address for remote cluster server traffic, for
example using <<remote-cluster-network-settings,`remote_cluster.host`>>. Without example using <<remote-cluster-network-settings,`remote_cluster.host`>>. Without
configuring the address, remote cluster traffic may be bound to the local configuring the address, remote cluster traffic may be bound to the local
interface, and remote clusters running on other machines can't connect. interface, and remote clusters running on other machines can't connect.
* Optionally, configure the remote server port using * Optionally, configure the remote server port using
<<remote_cluster.port,`remote_cluster.port`>> (defaults to `9443`). <<remote_cluster.port,`remote_cluster.port`>> (defaults to `9443`).
[[remote-clusters-troubleshooting-common-issues]] [[remote-clusters-troubleshooting-common-issues]]
@ -73,7 +73,7 @@ org.elasticsearch.transport.ConnectTransportException: [][192.168.0.42:9443] *co
====== Resolution ====== Resolution
* Check the host and port for the remote cluster are correct. * Check the host and port for the remote cluster are correct.
* Ensure the <<remote-clusters-troubleshooting-enable-server,remote cluster * Ensure the <<remote-clusters-troubleshooting-enable-server,remote cluster
server is enabled>> on the remote cluster. server is enabled>> on the remote cluster.
* Ensure no firewall is blocking the communication. * Ensure no firewall is blocking the communication.
@ -108,11 +108,11 @@ cause of the failure. For example:
* Is the remote cluster certificate not signed by a trusted CA? This is the most * Is the remote cluster certificate not signed by a trusted CA? This is the most
likely cause. likely cause.
* Is hostname verification failing? * Is hostname verification failing?
* Is the certificate expired? * Is the certificate expired?
Once you know the cause, you should be able to fix it by adjusting the remote Once you know the cause, you should be able to fix it by adjusting the remote
cluster related SSL settings on either the local cluster or the remote cluster. cluster related SSL settings on either the local cluster or the remote cluster.
Often, the issue is on the local cluster. For example, fix it by configuring necessary Often, the issue is on the local cluster. For example, fix it by configuring necessary
trusted CAs (`xpack.security.remote_cluster_client.ssl.certificate_authorities`). trusted CAs (`xpack.security.remote_cluster_client.ssl.certificate_authorities`).
@ -275,7 +275,7 @@ This does not show up in the logs of the remote cluster.
====== Resolution ====== Resolution
Add the cross-cluster API key to {es} keystore on every node of the local Add the cross-cluster API key to {es} keystore on every node of the local
cluster. Restart the local cluster to reload the keystore. cluster. Use the <<cluster-nodes-reload-secure-settings>> API to reload the keystore.
[[remote-clusters-troubleshooting-wrong-api-key-type]] [[remote-clusters-troubleshooting-wrong-api-key-type]]
===== Using the wrong API key type ===== Using the wrong API key type
@ -302,8 +302,7 @@ This does not show up in the logs of the remote cluster.
Ask the remote cluster administrator to create and distribute a Ask the remote cluster administrator to create and distribute a
<<security-api-create-cross-cluster-api-key,cross-cluster API key>>. Replace the <<security-api-create-cross-cluster-api-key,cross-cluster API key>>. Replace the
existing API key in the {es} keystore with this cross-cluster API key on every existing API key in the {es} keystore with this cross-cluster API key on every
node of the local cluster. Restart the local cluster for keystore changes to node of the local cluster. Use the <<cluster-nodes-reload-secure-settings>> API to reload the keystore.
take effect.
[[remote-clusters-troubleshooting-non-valid-api-key]] [[remote-clusters-troubleshooting-non-valid-api-key]]
===== Invalid API key ===== Invalid API key
@ -334,15 +333,14 @@ The remote cluster logs `Authentication using apikey failed`:
Ask the remote cluster administrator to create and distribute a Ask the remote cluster administrator to create and distribute a
<<security-api-create-cross-cluster-api-key,cross-cluster API key>>. Replace the <<security-api-create-cross-cluster-api-key,cross-cluster API key>>. Replace the
existing API key in the {es} keystore with this cross-cluster API key on every existing API key in the {es} keystore with this cross-cluster API key on every
node of the local cluster. Restart the local cluster for keystore changes to node of the local cluster. Use the <<cluster-nodes-reload-secure-settings>> API to reload the keystore.
take effect.
[[remote-clusters-troubleshooting-insufficient-privileges]] [[remote-clusters-troubleshooting-insufficient-privileges]]
===== API key or local user has insufficient privileges ===== API key or local user has insufficient privileges
The effective permission for a local user running requests on a remote cluster The effective permission for a local user running requests on a remote cluster
is determined by the intersection of the cross-cluster API key's privileges and is determined by the intersection of the cross-cluster API key's privileges and
the local user's `remote_indices` privileges. the local user's `remote_indices` privileges.
====== Symptom ====== Symptom
@ -366,8 +364,7 @@ This does not show up in any logs.
create and distribute a create and distribute a
<<security-api-create-cross-cluster-api-key,cross-cluster API key>>. Replace the <<security-api-create-cross-cluster-api-key,cross-cluster API key>>. Replace the
existing API key in the {es} keystore with this cross-cluster API key on every existing API key in the {es} keystore with this cross-cluster API key on every
node of the local cluster. Restart the local cluster for keystore changes to node of the local cluster. Use the <<cluster-nodes-reload-secure-settings>> API to reload the keystore.
take effect.
[[remote-clusters-troubleshooting-no-remote_indices-privileges]] [[remote-clusters-troubleshooting-no-remote_indices-privileges]]
===== Local user has no `remote_indices` privileges ===== Local user has no `remote_indices` privileges

1
docs/reference/setup/secure-settings.asciidoc
View file

@ -64,3 +64,4 @@ There are reloadable secure settings for:
* <<ref-jwt-settings, JWT realm>> * <<ref-jwt-settings, JWT realm>>
* <<ref-ad-settings, Active Directory realm>> * <<ref-ad-settings, Active Directory realm>>
* <<ref-ldap-settings, LDAP realm>> * <<ref-ldap-settings, LDAP realm>>
* <<remote-cluster-credentials-setting, Remote cluster credentials for the API key based security model>>