[Upgrade Assistant] First iteration of batch reindex docs (#59887) (#60731)

* First iteration of batch reindex docs

Tested with docs generator repo

* Add top level bullet points and remove cruft

* Address PR feedback

Also move the experimental marker to similar position (before
description) on existing endpoint docs for UA.

docs/api/upgrade-assistant.asciidoc

@@ -10,11 +10,17 @@ The following upgrade assistant APIs are available:
 
 * <<start-resume-reindex, Start or resume reindex API>> to start a new reindex or resume a paused reindex
 
+* <<batch-start-resume-reindex, Batch start or resume reindex API>> to start or resume multiple reindex tasks
+
+* <<batch-reindex-queue, Batch reindex queue API>> to check the current reindex batch queue
+
 * <<check-reindex-status, Check reindex status API>> to check the status of the reindex operation
 
 * <<cancel-reindex, Cancel reindex API>> to cancel reindexes that are waiting for the Elasticsearch reindex task to complete
 
 include::upgrade-assistant/status.asciidoc[]
 include::upgrade-assistant/reindexing.asciidoc[]
+include::upgrade-assistant/batch_reindexing.asciidoc[]
+include::upgrade-assistant/batch_queue.asciidoc[]
 include::upgrade-assistant/check_reindex_status.asciidoc[]
 include::upgrade-assistant/cancel_reindex.asciidoc[]

docs/api/upgrade-assistant/batch_queue.asciidoc

@@ -0,0 +1,68 @@
+[[batch-reindex-queue]]
+=== Batch reindex queue API
+++++
+<titleabbrev>Batch reindex queue</titleabbrev>
+++++
+
+experimental["The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are experimental."]
+
+Check the current reindex batch queue.
+
+[[batch-reindex-queue-request]]
+==== Request
+
+`GET /api/upgrade_assistant/reindex/batch/queue`
+
+[[batch-reindex-queue-request-codes]]
+==== Response code
+
+`200`::
+  Indicates a successful call.
+
+[[batch-reindex-queue-example]]
+==== Example
+
+The API returns the following:
+
+[source,js]
+--------------------------------------------------
+{
+  "queue": [ <1>
+    {
+      "indexName": "index1",
+      "newIndexName": "reindexed-v8-index2",
+      "status": 3,
+      "lastCompletedStep": 0,
+      "locked": null,
+      "reindexTaskId": null,
+      "reindexTaskPercComplete": null,
+      "errorMessage": null,
+      "runningReindexCount": null,
+      "reindexOptions": {
+        "queueSettings": {
+          "queuedAt": 1583406985489
+        }
+      }
+    },
+    {
+      "indexName": "index2",
+      "newIndexName": "reindexed-v8-index2",
+      "status": 3,
+      "lastCompletedStep": 0,
+      "locked": null,
+      "reindexTaskId": null,
+      "reindexTaskPercComplete": null,
+      "errorMessage": null,
+      "runningReindexCount": null,
+      "reindexOptions": {
+        "queueSettings": {
+          "queuedAt": 1583406987334
+        }
+      }
+    }
+  ]
+}
+--------------------------------------------------
+
+<1> Items in this array indicate reindex tasks at a given point in time and the order in which they will be executed.
+

docs/api/upgrade-assistant/batch_reindexing.asciidoc

@@ -0,0 +1,81 @@
+[[batch-start-resume-reindex]]
+=== Batch start or resume reindex API
+++++
+<titleabbrev>Batch start or resume reindex</titleabbrev>
+++++
+
+experimental["The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are experimental."]
+
+Start or resume multiple reindexing tasks in one request. Additionally, reindexing tasks started or resumed
+via the batch endpoint will be placed on a queue and executed one-by-one, which ensures that minimal cluster resources
+are consumed over time.
+
+[[batch-start-resume-reindex-request]]
+==== Request
+
+`POST /api/upgrade_assistant/reindex/batch`
+
+[[batch-start-resume-reindex-request-body]]
+==== Request body
+
+`indexNames`::
+  (Required, array) The list of index names to be reindexed.
+
+[[batch-start-resume-reindex-codes]]
+==== Response code
+
+`200`::
+  Indicates a successful call.
+
+[[batch-start-resume-example]]
+==== Example
+
+[source,js]
+--------------------------------------------------
+POST /api/upgrade_assistant/reindex/batch
+{
+  "indexNames": [ <1>
+    "index1",
+    "index2"
+  ]
+}
+--------------------------------------------------
+<1> The order in which the indices are provided here determines the order in which the reindex tasks will be executed.
+
+Similar to the <<start-resume-reindex, start or resume endpoint>>, the API returns the following:
+
+[source,js]
+--------------------------------------------------
+{
+  "enqueued": [ <1>
+    {
+      "indexName": "index1",
+      "newIndexName": "reindexed-v8-index1",
+      "status": 3,
+      "lastCompletedStep": 0,
+      "locked": null,
+      "reindexTaskId": null,
+      "reindexTaskPercComplete": null,
+      "errorMessage": null,
+      "runningReindexCount": null,
+      "reindexOptions": { <2>
+        "queueSettings": {
+          "queuedAt": 1583406985489 <3>
+        }
+      }
+    }
+  ],
+  "errors": [ <4>
+    {
+      "indexName": "index2",
+      "message": "Something went wrong!"
+    }
+  ]
+}
+--------------------------------------------------
+
+<1> A list of reindex operations created, the order in the array indicates the order in which tasks will be executed.
+<2> Presence of this key indicates that the reindex job will occur in the batch.
+<3> A Unix timestamp of when the reindex task was placed in the queue.
+<4> A list of errors that may have occurred preventing the reindex task from being created.
+

docs/api/upgrade-assistant/cancel_reindex.asciidoc

@@ -4,10 +4,10 @@
 <titleabbrev>Cancel reindex</titleabbrev>
 ++++
 
-Cancel reindexes that are waiting for the Elasticsearch reindex task to complete. For example, `lastCompletedStep` set to `40`.
-
 experimental["The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are experimental."]
 
+Cancel reindexes that are waiting for the Elasticsearch reindex task to complete. For example, `lastCompletedStep` set to `40`.
+
 [[cancel-reindex-request]]
 ==== Request
 

docs/api/upgrade-assistant/check_reindex_status.asciidoc

@@ -4,10 +4,10 @@
 <titleabbrev>Check reindex status</titleabbrev>
 ++++
 
-Check the status of the reindex operation.
-
 experimental["The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are experimental."]
 
+Check the status of the reindex operation.
+
 [[check-reindex-status-request]]
 ==== Request
 

docs/api/upgrade-assistant/reindexing.asciidoc

@@ -4,10 +4,10 @@
 <titleabbrev>Start or resume reindex</titleabbrev>
 ++++
 
-Start a new reindex or resume a paused reindex.
-
 experimental["The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are experimental."]
 
+Start a new reindex or resume a paused reindex.
+
 [[start-resume-reindex-request]]
 ==== Request
 

docs/api/upgrade-assistant/status.asciidoc

@@ -4,10 +4,10 @@
 <titleabbrev>Upgrade readiness status</titleabbrev>
 ++++
 
-Check the status of your cluster.
-
 experimental["The underlying Upgrade Assistant concepts are stable, but the APIs for managing Upgrade Assistant are experimental."]
 
+Check the status of your cluster.
+
 [[upgrade-assistant-api-status-request]]
 ==== Request