[ML] Adds progress reporting for transforms (#41278)

* [ML] Adds progress reporting for transforms

* fixing after master merge

* Addressing PR comments

* removing unused imports

* Adjusting afterKey handling and percentage to be 100*

* Making sure it is a linked hashmap for serialization

* removing unused import

* addressing PR comments

* removing unused import

* simplifying code, only storing total docs and decrementing

* adjusting for rewrite

* removing initial progress gathering from executor
This commit is contained in:
Benjamin Trent 2019-04-25 07:54:06 -05:00 committed by GitHub
parent f26addc03c
commit 9bf8b5a442
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
28 changed files with 1285 additions and 265 deletions

View file

@ -37,4 +37,6 @@ include-tagged::{doc-tests-file}[{api}-response]
<1> The response contains a list of `DataFrameTransformStateAndStats` objects
<2> The running state of the transform task e.g `started`
<3> The running state of the transform indexer e.g `started`, `indexing`, etc.
<4> The transform progress statistics recording the number of documents indexed etc
<4> The overall transform statistics recording the number of documents indexed etc.
<5> The progress of the current run in the transform. Supplies the number of docs left until the next checkpoint
and the total number of docs expected.