Commit graph

1 commit

Author SHA1 Message Date
Benjamin Trent
374f995e4e
[7.x] [ML] add new bucket_correlation aggregation with initial count_correlation function (#72133) (#72896)
* [ML] add new bucket_correlation aggregation with initial count_correlation function (#72133)

This commit adds a new pipeline aggregation that allows correlation within the aggregation frame work in bucketed values.

The initial function is a `count_correlation` function. The purpose of which is to correlate the count in a consistent number of buckets with a pre calculated indicator. The indicator and the aggregated buckets should related to the same metrics with in documents.

Example for correlating terms within a `service.version.keyword` with latency percentiles. The percentiles and provided correlation indicator both refer to the same source data where the indicator was previously calculated.:
```
GET apm-7.12.0-transaction-generated/_search
{
  "size": 0,
  "aggs": {
    "field_terms": {
      "terms": {
        "field": "service.version.keyword",
        "size": 20
      },
      "aggs": {
        "latency_range": {
          "range": {
            "field": "transaction.duration.us",
            "ranges": [<snip>],
            "keyed": true
          }
        },
        "correlation": {
          "bucket_correlation": {
            "buckets_path": "latency_range>_count",
            "count_correlation": {
              "indicator": {
                 "expectations": [<snip>],
                 "doc_count": 20000
               }
            }
          }
        }
      }
    }
  }
}
```
2021-05-10 14:34:21 -04:00