Migrates the existing TaskManager plugin from Legacy to Kibana Platform.
We retain the Legacy API to prevent a breaking change, but under the hood, the legacy plugin is now using the Kibana Platform plugin.
Another reason we retain the Legacy plugin to support several features that the Platform team has yet to migrate to Kibana Platform (mapping, SO schema and migrations).
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Adds a `runNow` api to Task Manager, allowing us to force the refresh of a recurring task.
This PR includes a couple of sustainability changes as well as the feature itself.
1. **Declarative query composition.** At the moment the queries in the TaskStore are huge JSON objects that are hard to maintain and understand. This PR introduces a pattern where the different parts of the query are composed out of type-checked functions, making it easier to maintain and to construct dynamically as needs change. _This was included in this PR as the **markAvailableTasksAsClaimed** query needs different query clauses depending on whether there are specific Tasks we wish to claim first.
2. **Refactoring of the Task Poller** As the `runNow` api is introduced we find Task Manager's lifecycle in a weird state where it has both a _pull_ model, where timeouts & callbacks interact without having to responsd to any external requests, and a _push_ model where requests are made to the new `runNow` api. Balancing these two proved error prone, hard to maintain and had the potential of _lossy_ behaviour where requests are dropped accidentally. To address this TaskPoller has been refactored using Rxjs observables, remodelling the existing _pull_ mechanism as a _push_ mechanism so Task Manager can _respond_ to both _polling_ calls and _runNow_ in a similar fashion.
And ofcourse the main feature of this PR:
3. **runNow api** An api on TaskManager that takes a _task ID_ and attempts to run the task. The call returns a promise which resolves with a result which notifies the caller when the task has either completed successfully, or result in an error.
This PR include three key changes:
1. Run tasks as soon as they have been marked as running, rather than wait for the whole batch to me marked
2. Use a custom refresh setting of refresh: false where possible, in place of wait_for, in order to speed up Task Manager's internal workflow
3. Instrumentation of Task Manager exposing Activity / Inactivity metrics in Performance test runs
Task Manager now uses updateByQuery to identify tasks it needs to pick up and claim ownership of them in a single call to Elasticsearch.
This reduces the version conflicts that often plague Task Manager when running multiple Kibana against a single ES cluster.
This has already shown small perf gains locally, and lays the ground work to a second phase (bulk processing the markTaskAsRunning step) which should allow to paralelise the task running feature further.
Introduces a disabled (enabled manually when needed) Integration Test for stress testing Task Manager.
This is used to generate baseline stats for Task manager performance which can then be rerun to compare performance oriented changes and measure their performance.
