// tag::cloud[] In order to restore the indices and data streams that are missing data: **Use {kib}** //tag::kibana-api-ex[] . Log in to the {ess-console}[{ecloud} console]. + . On the **Elasticsearch Service** panel, click the name of your deployment. + NOTE: If the name of your deployment is disabled your {kib} instances might be unhealthy, in which case please contact https://support.elastic.co[Elastic Support]. If your deployment doesn't include {kib}, all you need to do is {cloud}/ec-access-kibana.html[enable it first]. + . Open your deployment's side navigation menu (placed under the Elastic logo in the upper left corner) and go to **Dev Tools > Console**. + [role="screenshot"] image::images/kibana-console.png[{kib} Console,align="center"] . To view the affected indices using the <>. + [source,console] ---- GET _cat/indices?v&health=red&h=index,status,health ---- + The response will look like this: + [source,console-result] ---- index status health .ds-my-data-stream-2022.06.17-000001 open red kibana_sample_data_flights open red ---- // TEST[skip:illustration purposes only] + The `red` health of the indices above indicates that these indices are missing primary shards, meaning they are missing data. + . In order to restore the data we need to find a snapshot that contains these two indices. To find such a snapshot use the <>. + [source,console] ---- GET _snapshot/my_repository/*?verbose=false ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- { "snapshots" : [ { "snapshot" : "snapshot-20200617", <1> "uuid" : "dZyPs1HyTwS-cnKdH08EPg", "repository" : "my_repository", <2> "indices" : [ <3> ".apm-agent-configuration", ".apm-custom-link", ".ds-ilm-history-5-2022.06.17-000001", ".ds-my-data-stream-2022.06.17-000001", ".geoip_databases", ".kibana-event-log-8.2.2-000001", ".kibana_8.2.2_001", ".kibana_task_manager_8.2.2_001", "kibana_sample_data_ecommerce", "kibana_sample_data_flights", "kibana_sample_data_logs" ], "data_streams" : [ ], "state" : "SUCCESS" <4> } ], "total" : 1, "remaining" : 0 } ---- // TEST[skip:illustration purposes only] + <1> The name of the snapshot. + <2> The repository of the snapshot. + <3> The indices backed up in the snapshot. + <4> If the snapshot was successful. . The snapshot `snapshot-20200617` contains the two indices we want to restore. You might have multiple snapshots from which you could restore the target indices. Choose the latest snapshot. + . Now that we found a snapshot, we will proceed with the data stream preparation for restoring the lost data. We will check the index metadata to see if any index is part of a data stream: + [source,console] ---- GET kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001?features=settings&flat_settings ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- { ".ds-my-data-stream-2022.06.17-000001" : { <1> "aliases" : { }, "mappings" : { }, "settings" : { <2> "index.creation_date" : "1658406121699", "index.hidden" : "true", "index.lifecycle.name" : "my-lifecycle-policy", "index.number_of_replicas" : "1", "index.number_of_shards" : "1", "index.provided_name" : ".ds-my-data-stream-2022.06.17-000001", "index.routing.allocation.include._tier_preference" : "data_hot", "index.uuid" : "HmlFXp6VSu2XbQ-O3hVrwQ", "index.version.created" : "8020299" }, "data_stream" : "my-data-stream" <3> }, "kibana_sample_data_flights" : { <4> "aliases" : { }, "mappings" : { }, "settings" : { "index.creation_date" : "1655121541454", "index.number_of_replicas" : "0", "index.number_of_shards" : "1", "index.provided_name" : "kibana_sample_data_flights", "index.routing.allocation.include._tier_preference" : "data_content", "index.uuid" : "jMOlwKPPSzSraeeBWyuoDA", "index.version.created" : "8020299" } } } ---- // TEST[skip:illustration purposes only] + <1> The name of an index. + <2> The settings of this index that contains the metadata we are looking for. + <3> This indicates that this index is part of a data stream and displays the data stream name. + <4> The name of the other index we requested. + The response above shows that `kibana_sample_data_flights` is not part of a data stream because it doesn't have a field called `data_stream` in the settings. + On the contrary, `.ds-my-data-stream-2022.06.17-000001` is part of the data stream called `my-data-stream`. When you find an index like this, which belongs to a data stream, you need to check if data are still being indexed. You can see that by checking the `settings`, if you can find this property: `"index.lifecycle.indexing_complete" : "true"`, it means that indexing is completed in this index and you can continue to the next step. + If `index.lifecycle.indexing_complete` is not there or is configured to `false` you need to rollover the data stream so you can restore the missing data without blocking the ingestion of new data. The following command will achieve that. + [source,console] ---- POST my-data-stream/_rollover ---- // TEST[skip:illustration purposes only] . Now that the data stream preparation is done, we will close the target indices by using the <>. + [source,console] ---- POST kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001/_close ---- // TEST[skip:illustration purposes only] + You can confirm that they are closed with the <>. + [source,console] ---- GET _cat/indices?v&health=red&h=index,status,health ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- index status health .ds-my-data-stream-2022.06.17-000001 close red kibana_sample_data_flights close red ---- + . The indices are closed, now we can restore them from snapshots without causing any complications using the <>: + [source,console] ---- POST _snapshot/my_repository/snapshot-20200617/_restore { "indices": "kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001", <1> "include_aliases": true <2> } ---- // TEST[skip:illustration purposes only] + <1> The indices to restore. + <2> We also want to restore the aliases. + NOTE: If any <> need to be restored we'll need to specify them using the `feature_states` field and the indices that belong to the feature states we restore must not be specified under `indices`. The <> returns both the `indices` and `feature_states` that need to be restored for the restore from snapshot diagnosis. e.g.: + [source,console] ---- POST _snapshot/my_repository/snapshot-20200617/_restore { "feature_states": [ "geoip" ], "indices": "kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001", "include_aliases": true } ---- // TEST[skip:illustration purposes only] . Finally we can verify that the indices health is now `green` via the <>. + [source,console] ---- GET _cat/indices?v&index=.ds-my-data-stream-2022.06.17-000001,kibana_sample_data_flightsh=index,status,health ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- index status health .ds-my-data-stream-2022.06.17-000001 open green kibana_sample_data_flights open green ---- // TEST[skip:illustration purposes only] + As we can see above the indices are `green` and open. The issue is resolved. For more guidance on creating and restoring snapshots see <>. //end::kibana-api-ex[] // end::cloud[] // tag::self-managed[] In order to restore the indices that are missing shards: . View the affected indices using the <>. + [source,console] ---- GET _cat/indices?v&health=red&h=index,status,health ---- + The response will look like this: + [source,console-result] ---- index status health .ds-my-data-stream-2022.06.17-000001 open red kibana_sample_data_flights open red ---- // TEST[skip:illustration purposes only] + The `red` health of the indices above indicates that these indices are missing primary shards, meaning they are missing data. + . In order to restore the data we need to find a snapshot that contains these two indices. To find such a snapshot use the <>. + [source,console] ---- GET _snapshot/my_repository/*?verbose=false ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- { "snapshots" : [ { "snapshot" : "snapshot-20200617", <1> "uuid" : "dZyPs1HyTwS-cnKdH08EPg", "repository" : "my_repository", <2> "indices" : [ <3> ".apm-agent-configuration", ".apm-custom-link", ".ds-ilm-history-5-2022.06.17-000001", ".ds-my-data-stream-2022.06.17-000001", ".geoip_databases", ".kibana-event-log-8.2.2-000001", ".kibana_8.2.2_001", ".kibana_task_manager_8.2.2_001", "kibana_sample_data_ecommerce", "kibana_sample_data_flights", "kibana_sample_data_logs" ], "data_streams" : [ ], "state" : "SUCCESS" <4> } ], "total" : 1, "remaining" : 0 } ---- // TEST[skip:illustration purposes only] + <1> The name of the snapshot. + <2> The repository of the snapshot. + <3> The indices backed up in the snapshot. + <4> If the snapshot was successful. . The snapshot `snapshot-20200617` contains the two indices we want to restore. You might have multiple snapshots from which you could restore the target indices. Choose the latest snapshot. + . Now that we found a snapshot, we will proceed with the data stream preparation for restoring the lost data. We will check the index metadata to see if any index is part of a data stream: + [source,console] ---- GET kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001?features=settings&flat_settings ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- { ".ds-my-data-stream-2022.06.17-000001" : { <1> "aliases" : { }, "mappings" : { }, "settings" : { <2> "index.creation_date" : "1658406121699", "index.hidden" : "true", "index.lifecycle.name" : "my-lifecycle-policy", "index.number_of_replicas" : "1", "index.number_of_shards" : "1", "index.provided_name" : ".ds-my-data-stream-2022.06.17-000001", "index.routing.allocation.include._tier_preference" : "data_hot", "index.uuid" : "HmlFXp6VSu2XbQ-O3hVrwQ", "index.version.created" : "8020299" }, "data_stream" : "my-data-stream" <3> }, "kibana_sample_data_flights" : { <4> "aliases" : { }, "mappings" : { }, "settings" : { "index.creation_date" : "1655121541454", "index.number_of_replicas" : "0", "index.number_of_shards" : "1", "index.provided_name" : "kibana_sample_data_flights", "index.routing.allocation.include._tier_preference" : "data_content", "index.uuid" : "jMOlwKPPSzSraeeBWyuoDA", "index.version.created" : "8020299" } } } ---- // TEST[skip:illustration purposes only] + <1> The name of an index. + <2> The settings of this index that contains the metadata we are looking for. + <3> This indicates that this index is part of a data stream and displays the data stream name. + <4> The name of the other index we requested. + The response above shows that `kibana_sample_data_flights` is not part of a data stream because it doesn't have a field called `data_stream` in the settings. + On the contrary, `.ds-my-data-stream-2022.06.17-000001` is part of the data stream called `my-data-stream`. When you find an index like this, which belongs to a data stream, you need to check if data are still being indexed. You can see that by checking the `settings`, if you can find this property: `"index.lifecycle.indexing_complete" : "true"`, it means that indexing is completed in this index and you can continue to the next step. + If `index.lifecycle.indexing_complete` is not there or is configured to `false` you need to rollover the data stream so you can restore the missing data without blocking the ingestion of new data. The following command will achieve that. + [source,console] ---- POST my-data-stream/_rollover ---- // TEST[skip:illustration purposes only] . Now that the data stream preparation is done, we will close the target indices by using the <>. + [source,console] ---- POST kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001/_close ---- // TEST[skip:illustration purposes only] + You can confirm that they are closed with the <>. + [source,console] ---- GET _cat/indices?v&health=red&h=index,status,health ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- index status health .ds-my-data-stream-2022.06.17-000001 close red kibana_sample_data_flights close red ---- + . The indices are closed, now we can restore them from snapshots without causing any complications using the <>: + [source,console] ---- POST _snapshot/my_repository/snapshot-20200617/_restore { "indices": "kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001", <1> "include_aliases": true <2> } ---- // TEST[skip:illustration purposes only] + <1> The indices to restore. + <2> We also want to restore the aliases. + NOTE: If any <> need to be restored we'll need to specify them using the `feature_states` field and the indices that belong to the feature states we restore must not be specified under `indices`. The <> returns both the `indices` and `feature_states` that need to be restored for the restore from snapshot diagnosis. e.g.: + [source,console] ---- POST _snapshot/my_repository/snapshot-20200617/_restore { "feature_states": [ "geoip" ], "indices": "kibana_sample_data_flights,.ds-my-data-stream-2022.06.17-000001", "include_aliases": true } ---- // TEST[skip:illustration purposes only] . Finally we can verify that the indices health is now `green` via the <>. + [source,console] ---- GET _cat/indices?v&index=.ds-my-data-stream-2022.06.17-000001,kibana_sample_data_flightsh=index,status,health ---- // TEST[skip:illustration purposes only] + The response will look like this: + [source,console-result] ---- index status health .ds-my-data-stream-2022.06.17-000001 open green kibana_sample_data_flights open green ---- // TEST[skip:illustration purposes only] + As we can see above the indices are `green` and open. The issue is resolved. For more guidance on creating and restoring snapshots see <>. // end::self-managed[]