[Alerting]: get type-checking, tests, and ui working for index threshold (#59064) (#59142)

This is a follow-on to https://github.com/elastic/kibana/pull/57030 ,
"[alerting] initial index threshold alertType and supporting APIs",
to get it working with the existing alerting UI.  The parameter shape
was different between the two, so the alertType was changed to fix
the existing UI shapes expected.
This commit is contained in:
Patrick Mueller 2020-03-03 11:21:21 -05:00 committed by GitHub
parent dbcf47f573
commit e099793ae5
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
18 changed files with 344 additions and 238 deletions

View file

@ -58,43 +58,47 @@ Finally, create the alert:
```
kbn-alert create .index-threshold 'es-hb-sim threshold' 1s \
'{
index: es-hb-sim
timeField: @timestamp
aggType: average
aggField: summary.up
groupField: monitor.name.keyword
window: 5s
comparator: lessThan
threshold: [ 0.6 ]
index: es-hb-sim
timeField: @timestamp
aggType: avg
aggField: summary.up
groupBy: top
termSize: 100
termField: monitor.name.keyword
timeWindowSize: 5
timeWindowUnit: s
thresholdComparator: <
threshold: [ 0.6 ]
}' \
"[
{
group: threshold met
id: '$ACTION_ID'
group: threshold met
id: '$ACTION_ID'
params: {
level: warn
message: '{{context.message}}'
level: warn
message: '{{{context.message}}}'
}
}
]"
```
This alert will run a query over the `es-hb-sim` index, using the `@timestamp`
field as the date field, using an `average` aggregation over the `summary.up`
field. The results are then aggregated by `monitor.name.keyword`. If we ran
field as the date field, aggregating over groups of the field value
`monitor.name.keyword` (the top 100 groups), then aggregating those values
using an `average` aggregation over the `summary.up` field. If we ran
another instance of `es-hb-sim`, using `host-B` instead of `host-A`, then the
alert will end up potentially scheduling actions for both, independently.
Within the alerting plugin, this grouping is also referred to as "instanceIds"
(`host-A` and `host-B` being distinct instanceIds, which can have actions
scheduled against them independently).
The `window` is set to `5s` which is 5 seconds. That means, every time the
The time window is set to 5 seconds. That means, every time the
alert runs it's queries (every second, in the example above), it will run it's
ES query over the last 5 seconds. Thus, the queries, over time, will overlap.
Sometimes that's what you want. Other times, maybe you just want to do
sampling, running an alert every hour, with a 5 minute window. Up to the you!
Using the `comparator` `lessThan` and `threshold` `[0.6]`, the alert will
Using the `thresholdComparator` `<` and `threshold` `[0.6]`, the alert will
calculate the average of all the `summary.up` fields for each unique
`monitor.name.keyword`, and then if the value is less than 0.6, it will
schedule the specified action (server log) to run. The `message` param
@ -110,11 +114,10 @@ working:
```
server log [17:32:10.060] [warning][actions][actions][plugins] \
Server log: alert es-hb-sim threshold instance host-A value 0 \
exceeded threshold average(summary.up) lessThan 0.6 over 5s \
Server log: alert es-hb-sim threshold group host-A value 0 \
exceeded threshold avg(summary.up) < 0.6 over 5s \
on 2020-02-20T22:32:07.000Z
```
[kbn-action]: https://github.com/pmuellr/kbn-action
[es-hb-sim]: https://github.com/pmuellr/es-hb-sim
[now-iso]: https://github.com/pmuellr/now-iso
@ -144,15 +147,18 @@ This example uses [now-iso][] to generate iso date strings.
```console
curl -k "https://elastic:changeme@localhost:5601/api/alerting_builtins/index_threshold/_time_series_query" \
-H "kbn-xsrf: foo" -H "content-type: application/json" -d "{
\"index\": \"es-hb-sim\",
\"timeField\": \"@timestamp\",
\"aggType\": \"average\",
\"aggField\": \"summary.up\",
\"groupField\": \"monitor.name.keyword\",
\"interval\": \"1s\",
\"dateStart\": \"`now-iso -10s`\",
\"dateEnd\": \"`now-iso`\",
\"window\": \"5s\"
\"index\": \"es-hb-sim\",
\"timeField\": \"@timestamp\",
\"aggType\": \"avg\",
\"aggField\": \"summary.up\",
\"groupBy\": \"top\",
\"termSize\": 100,
\"termField\": \"monitor.name.keyword\",
\"interval\": \"1s\",
\"dateStart\": \"`now-iso -10s`\",
\"dateEnd\": \"`now-iso`\",
\"timeWindowSize\": 5,
\"timeWindowUnit\": \"s\"
}"
```
@ -184,13 +190,16 @@ To get the current value of the calculated metric, you can leave off the date:
```
curl -k "https://elastic:changeme@localhost:5601/api/alerting_builtins/index_threshold/_time_series_query" \
-H "kbn-xsrf: foo" -H "content-type: application/json" -d '{
"index": "es-hb-sim",
"timeField": "@timestamp",
"aggType": "average",
"aggField": "summary.up",
"groupField": "monitor.name.keyword",
"interval": "1s",
"window": "5s"
"index": "es-hb-sim",
"timeField": "@timestamp",
"aggType": "avg",
"aggField": "summary.up",
"groupBy": "top",
"termField": "monitor.name.keyword",
"termSize": 100,
"interval": "1s",
"timeWindowSize": 5,
"timeWindowUnit": "s"
}'
```
@ -254,7 +263,7 @@ be ~24 time series points in the output.
For preview purposes:
- The `groupLimit` parameter should be used to help cut
- The `termSize` parameter should be used to help cut
down on the amount of work ES does, and keep the generated graphs a little
simpler. Probably something like `10`.
@ -263,9 +272,9 @@ simpler. Probably something like `10`.
could result in a lot of time-series points being generated, which is both
costly in ES, and may result in noisy graphs.
- The `window` parameter should be the same as what the alert is using,
- The `timeWindow*` parameters should be the same as what the alert is using,
especially for the `count` and `sum` aggregation types. Those aggregations
don't scale the same way the others do, when the window changes. Even for
the other aggregations, changing the window could result in dramatically
different values being generated - `averages` will be more "average-y", `min`
different values being generated - `avg` will be more "average-y", `min`
and `max` will be a little stickier.

View file

@ -21,8 +21,12 @@ describe('ActionContext', () => {
index: '[index]',
timeField: '[timeField]',
aggType: 'count',
window: '5m',
comparator: 'greaterThan',
groupBy: 'top',
termField: 'x',
termSize: 100,
timeWindowSize: 5,
timeWindowUnit: 'm',
thresholdComparator: '>',
threshold: [4],
});
const context = addMessages(base, params);
@ -30,7 +34,7 @@ describe('ActionContext', () => {
`"alert [name] group [group] exceeded threshold"`
);
expect(context.message).toMatchInlineSnapshot(
`"alert [name] group [group] value 42 exceeded threshold count greaterThan 4 over 5m on 2020-01-01T00:00:00.000Z"`
`"alert [name] group [group] value 42 exceeded threshold count > 4 over 5m on 2020-01-01T00:00:00.000Z"`
);
});
@ -46,10 +50,14 @@ describe('ActionContext', () => {
const params = ParamsSchema.validate({
index: '[index]',
timeField: '[timeField]',
aggType: 'average',
aggType: 'avg',
groupBy: 'top',
termField: 'x',
termSize: 100,
aggField: '[aggField]',
window: '5m',
comparator: 'greaterThan',
timeWindowSize: 5,
timeWindowUnit: 'm',
thresholdComparator: '>',
threshold: [4.2],
});
const context = addMessages(base, params);
@ -57,7 +65,7 @@ describe('ActionContext', () => {
`"alert [name] group [group] exceeded threshold"`
);
expect(context.message).toMatchInlineSnapshot(
`"alert [name] group [group] value 42 exceeded threshold average([aggField]) greaterThan 4.2 over 5m on 2020-01-01T00:00:00.000Z"`
`"alert [name] group [group] value 42 exceeded threshold avg([aggField]) > 4.2 over 5m on 2020-01-01T00:00:00.000Z"`
);
});
@ -74,8 +82,12 @@ describe('ActionContext', () => {
index: '[index]',
timeField: '[timeField]',
aggType: 'count',
window: '5m',
comparator: 'between',
groupBy: 'top',
termField: 'x',
termSize: 100,
timeWindowSize: 5,
timeWindowUnit: 'm',
thresholdComparator: 'between',
threshold: [4, 5],
});
const context = addMessages(base, params);

View file

@ -47,8 +47,9 @@ export function addMessages(c: BaseActionContext, p: Params): ActionContext {
);
const agg = p.aggField ? `${p.aggType}(${p.aggField})` : `${p.aggType}`;
const humanFn = `${agg} ${p.comparator} ${p.threshold.join(',')}`;
const humanFn = `${agg} ${p.thresholdComparator} ${p.threshold.join(',')}`;
const window = `${p.timeWindowSize}${p.timeWindowUnit}`;
const message = i18n.translate(
'xpack.alertingBuiltins.indexThreshold.alertTypeContextMessageDescription',
{
@ -59,7 +60,7 @@ export function addMessages(c: BaseActionContext, p: Params): ActionContext {
group: c.group,
value: c.value,
function: humanFn,
window: p.window,
window,
date: c.date,
},
}

View file

@ -6,6 +6,7 @@
import { loggingServiceMock } from '../../../../../../src/core/server/mocks';
import { getAlertType } from './alert_type';
import { Params } from './alert_type_params';
describe('alertType', () => {
const service = {
@ -24,12 +25,14 @@ describe('alertType', () => {
});
it('validator succeeds with valid params', async () => {
const params = {
const params: Partial<Writable<Params>> = {
index: 'index-name',
timeField: 'time-field',
aggType: 'count',
window: '5m',
comparator: 'greaterThan',
groupBy: 'all',
timeWindowSize: 5,
timeWindowUnit: 'm',
thresholdComparator: '<',
threshold: [0],
};
@ -40,12 +43,14 @@ describe('alertType', () => {
const paramsSchema = alertType.validate?.params;
if (!paramsSchema) throw new Error('params validator not set');
const params = {
const params: Partial<Writable<Params>> = {
index: 'index-name',
timeField: 'time-field',
aggType: 'foo',
window: '5m',
comparator: 'greaterThan',
groupBy: 'all',
timeWindowSize: 5,
timeWindowUnit: 'm',
thresholdComparator: '>',
threshold: [0],
};

View file

@ -8,6 +8,7 @@ import { i18n } from '@kbn/i18n';
import { AlertType, AlertExecutorOptions } from '../../types';
import { Params, ParamsSchema } from './alert_type_params';
import { BaseActionContext, addMessages } from './action_context';
import { TimeSeriesQuery } from './lib/time_series_query';
export const ID = '.index-threshold';
@ -46,24 +47,26 @@ export function getAlertType(service: Service): AlertType {
const { alertId, name, services } = options;
const params: Params = options.params as Params;
const compareFn = ComparatorFns.get(params.comparator);
const compareFn = ComparatorFns.get(params.thresholdComparator);
if (compareFn == null) {
throw new Error(getInvalidComparatorMessage(params.comparator));
throw new Error(getInvalidComparatorMessage(params.thresholdComparator));
}
const callCluster = services.callCluster;
const date = new Date().toISOString();
// the undefined values below are for config-schema optional types
const queryParams = {
const queryParams: TimeSeriesQuery = {
index: params.index,
timeField: params.timeField,
aggType: params.aggType,
aggField: params.aggField,
groupField: params.groupField,
groupLimit: params.groupLimit,
groupBy: params.groupBy,
termField: params.termField,
termSize: params.termSize,
dateStart: date,
dateEnd: date,
window: params.window,
timeWindowSize: params.timeWindowSize,
timeWindowUnit: params.timeWindowUnit,
interval: undefined,
};
const result = await service.indexThreshold.timeSeriesQuery({
@ -100,7 +103,7 @@ export function getAlertType(service: Service): AlertType {
export function getInvalidComparatorMessage(comparator: string) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidComparatorErrorMessage', {
defaultMessage: 'invalid comparator specified: {comparator}',
defaultMessage: 'invalid thresholdComparator specified: {comparator}',
values: {
comparator,
},
@ -111,10 +114,10 @@ type ComparatorFn = (value: number, threshold: number[]) => boolean;
function getComparatorFns(): Map<string, ComparatorFn> {
const fns: Record<string, ComparatorFn> = {
lessThan: (value: number, threshold: number[]) => value < threshold[0],
lessThanOrEqual: (value: number, threshold: number[]) => value <= threshold[0],
greaterThanOrEqual: (value: number, threshold: number[]) => value >= threshold[0],
greaterThan: (value: number, threshold: number[]) => value > threshold[0],
'<': (value: number, threshold: number[]) => value < threshold[0],
'<=': (value: number, threshold: number[]) => value <= threshold[0],
'>=': (value: number, threshold: number[]) => value >= threshold[0],
'>': (value: number, threshold: number[]) => value > threshold[0],
between: (value: number, threshold: number[]) => value >= threshold[0] && value <= threshold[1],
notBetween: (value: number, threshold: number[]) =>
value < threshold[0] || value > threshold[1],

View file

@ -4,15 +4,17 @@
* you may not use this file except in compliance with the Elastic License.
*/
import { ParamsSchema } from './alert_type_params';
import { ParamsSchema, Params } from './alert_type_params';
import { runTests } from './lib/core_query_types.test';
const DefaultParams = {
const DefaultParams: Writable<Partial<Params>> = {
index: 'index-name',
timeField: 'time-field',
aggType: 'count',
window: '5m',
comparator: 'greaterThan',
groupBy: 'all',
timeWindowSize: 5,
timeWindowUnit: 'm',
thresholdComparator: '>',
threshold: [0],
};
@ -29,28 +31,29 @@ describe('alertType Params validate()', () => {
});
it('passes for maximal valid input', async () => {
params.aggType = 'average';
params.aggType = 'avg';
params.aggField = 'agg-field';
params.groupField = 'group-field';
params.groupLimit = 100;
params.groupBy = 'top';
params.termField = 'group-field';
params.termSize = 100;
expect(validate()).toBeTruthy();
});
it('fails for invalid comparator', async () => {
params.comparator = '[invalid-comparator]';
params.thresholdComparator = '[invalid-comparator]';
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[comparator]: invalid comparator specified: [invalid-comparator]"`
`"[thresholdComparator]: invalid thresholdComparator specified: [invalid-comparator]"`
);
});
it('fails for invalid threshold length', async () => {
params.comparator = 'lessThan';
params.threshold = [0, 1];
params.thresholdComparator = '<';
params.threshold = [0, 1, 2];
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[threshold]: must have one element for the \\"lessThan\\" comparator"`
`"[threshold]: array size is [3], but cannot be greater than [2]"`
);
params.comparator = 'between';
params.thresholdComparator = 'between';
params.threshold = [0];
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[threshold]: must have two elements for the \\"between\\" comparator"`

View file

@ -17,7 +17,7 @@ export const ParamsSchema = schema.object(
{
...CoreQueryParamsSchemaProperties,
// the comparison function to use to determine if the threshold as been met
comparator: schema.string({ validate: validateComparator }),
thresholdComparator: schema.string({ validate: validateComparator }),
// the values to use as the threshold; `between` and `notBetween` require
// two values, the others require one.
threshold: schema.arrayOf(schema.number(), { minSize: 1, maxSize: 2 }),
@ -35,26 +35,16 @@ function validateParams(anyParams: any): string | undefined {
const coreQueryValidated = validateCoreQueryBody(anyParams);
if (coreQueryValidated) return coreQueryValidated;
const { comparator, threshold }: Params = anyParams;
const { thresholdComparator, threshold }: Params = anyParams;
if (betweenComparators.has(comparator)) {
if (threshold.length === 1) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidThreshold2ErrorMessage', {
defaultMessage: '[threshold]: must have two elements for the "{comparator}" comparator',
values: {
comparator,
},
});
}
} else {
if (threshold.length === 2) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidThreshold1ErrorMessage', {
defaultMessage: '[threshold]: must have one element for the "{comparator}" comparator',
values: {
comparator,
},
});
}
if (betweenComparators.has(thresholdComparator) && threshold.length === 1) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidThreshold2ErrorMessage', {
defaultMessage:
'[threshold]: must have two elements for the "{thresholdComparator}" comparator',
values: {
thresholdComparator,
},
});
}
}

View file

@ -7,14 +7,16 @@
// tests of common properties on time_series_query and alert_type_params
import { ObjectType } from '@kbn/config-schema';
import { CoreQueryParams } from './core_query_types';
import { MAX_GROUPS } from '../index';
const DefaultParams: Record<string, any> = {
const DefaultParams: Writable<Partial<CoreQueryParams>> = {
index: 'index-name',
timeField: 'time-field',
aggType: 'count',
window: '5m',
groupBy: 'all',
timeWindowSize: 5,
timeWindowUnit: 'm',
};
export function runTests(schema: ObjectType, defaultTypeParams: Record<string, any>): void {
@ -30,28 +32,48 @@ export function runTests(schema: ObjectType, defaultTypeParams: Record<string, a
});
it('succeeds with maximal properties', async () => {
params.aggType = 'average';
params.aggType = 'avg';
params.aggField = 'agg-field';
params.groupField = 'group-field';
params.groupLimit = 200;
params.groupBy = 'top';
params.termField = 'group-field';
params.termSize = 200;
expect(validate()).toBeTruthy();
params.index = ['index-name-1', 'index-name-2'];
params.aggType = 'avg';
params.aggField = 'agg-field';
params.groupBy = 'top';
params.termField = 'group-field';
params.termSize = 200;
expect(validate()).toBeTruthy();
});
it('fails for invalid index', async () => {
delete params.index;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[index]: expected value of type [string] but got [undefined]"`
`"[index]: expected at least one defined value but got [undefined]"`
);
params.index = 42;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[index]: expected value of type [string] but got [number]"`
);
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(`
"[index]: types that failed validation:
- [index.0]: expected value of type [string] but got [number]
- [index.1]: expected value of type [array] but got [number]"
`);
params.index = '';
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[index]: value is [] but it must have a minimum length of [1]."`
);
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(`
"[index]: types that failed validation:
- [index.0]: value is [] but it must have a minimum length of [1].
- [index.1]: could not parse array value from []"
`);
params.index = ['', 'a'];
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(`
"[index]: types that failed validation:
- [index.0]: expected value of type [string] but got [Array]
- [index.1.0]: value is [] but it must have a minimum length of [1]."
`);
});
it('fails for invalid timeField', async () => {
@ -95,58 +117,67 @@ export function runTests(schema: ObjectType, defaultTypeParams: Record<string, a
);
});
it('fails for invalid groupField', async () => {
params.groupField = 42;
it('fails for invalid termField', async () => {
params.groupBy = 'top';
params.termField = 42;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[groupField]: expected value of type [string] but got [number]"`
`"[termField]: expected value of type [string] but got [number]"`
);
params.groupField = '';
params.termField = '';
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[groupField]: value is [] but it must have a minimum length of [1]."`
`"[termField]: value is [] but it must have a minimum length of [1]."`
);
});
it('fails for invalid groupLimit', async () => {
params.groupLimit = 'foo';
it('fails for invalid termSize', async () => {
params.groupBy = 'top';
params.termField = 'fee';
params.termSize = 'foo';
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[groupLimit]: expected value of type [number] but got [string]"`
`"[termSize]: expected value of type [number] but got [string]"`
);
params.groupLimit = 0;
params.termSize = MAX_GROUPS + 1;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[groupLimit]: must be greater than 0"`
`"[termSize]: must be less than or equal to 1000"`
);
params.groupLimit = MAX_GROUPS + 1;
params.termSize = 0;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[groupLimit]: must be less than or equal to 1000"`
`"[termSize]: Value is [0] but it must be equal to or greater than [1]."`
);
});
it('fails for invalid window', async () => {
params.window = 42;
it('fails for invalid timeWindowSize', async () => {
params.timeWindowSize = 'foo';
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[window]: expected value of type [string] but got [number]"`
`"[timeWindowSize]: expected value of type [number] but got [string]"`
);
params.window = 'x';
params.timeWindowSize = 0;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[window]: invalid duration: \\"x\\""`
`"[timeWindowSize]: Value is [0] but it must be equal to or greater than [1]."`
);
});
it('fails for invalid timeWindowUnit', async () => {
params.timeWindowUnit = 42;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[timeWindowUnit]: expected value of type [string] but got [number]"`
);
params.timeWindowUnit = 'x';
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[timeWindowUnit]: invalid timeWindowUnit: \\"x\\""`
);
});
it('fails for invalid aggType/aggField', async () => {
params.aggType = 'count';
params.aggField = 'agg-field-1';
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[aggField]: must not have a value when [aggType] is \\"count\\""`
);
params.aggType = 'average';
params.aggType = 'avg';
delete params.aggField;
expect(onValidate()).toThrowErrorMatchingInlineSnapshot(
`"[aggField]: must have a value when [aggType] is \\"average\\""`
`"[aggField]: must have a value when [aggType] is \\"avg\\""`
);
});
});

View file

@ -10,23 +10,29 @@ import { i18n } from '@kbn/i18n';
import { schema, TypeOf } from '@kbn/config-schema';
import { MAX_GROUPS } from '../index';
import { parseDuration } from '../../../../../alerting/server';
export const CoreQueryParamsSchemaProperties = {
// name of the index to search
index: schema.string({ minLength: 1 }),
// name of the indices to search
index: schema.oneOf([
schema.string({ minLength: 1 }),
schema.arrayOf(schema.string({ minLength: 1 }), { minSize: 1 }),
]),
// field in index used for date/time
timeField: schema.string({ minLength: 1 }),
// aggregation type
aggType: schema.string({ validate: validateAggType }),
// aggregation field
aggField: schema.maybe(schema.string({ minLength: 1 })),
// group field
groupField: schema.maybe(schema.string({ minLength: 1 })),
// how to group
groupBy: schema.string({ validate: validateGroupBy }),
// field to group on (for groupBy: top)
termField: schema.maybe(schema.string({ minLength: 1 })),
// limit on number of groups returned
groupLimit: schema.maybe(schema.number()),
termSize: schema.maybe(schema.number({ min: 1 })),
// size of time window for date range aggregations
window: schema.string({ validate: validateDuration }),
timeWindowSize: schema.number({ min: 1 }),
// units of time window for date range aggregations
timeWindowUnit: schema.string({ validate: validateTimeWindowUnits }),
};
const CoreQueryParamsSchema = schema.object(CoreQueryParamsSchemaProperties);
@ -37,17 +43,7 @@ export type CoreQueryParams = TypeOf<typeof CoreQueryParamsSchema>;
// above.
// Using direct type not allowed, circular reference, so body is typed to any.
export function validateCoreQueryBody(anyParams: any): string | undefined {
const { aggType, aggField, groupLimit }: CoreQueryParams = anyParams;
if (aggType === 'count' && aggField) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.aggTypeNotEmptyErrorMessage', {
defaultMessage: '[aggField]: must not have a value when [aggType] is "{aggType}"',
values: {
aggType,
},
});
}
const { aggType, aggField, groupBy, termField, termSize }: CoreQueryParams = anyParams;
if (aggType !== 'count' && !aggField) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.aggTypeRequiredErrorMessage', {
defaultMessage: '[aggField]: must have a value when [aggType] is "{aggType}"',
@ -57,21 +53,23 @@ export function validateCoreQueryBody(anyParams: any): string | undefined {
});
}
// schema.number doesn't seem to check the max value ...
if (groupLimit != null) {
if (groupLimit <= 0) {
return i18n.translate(
'xpack.alertingBuiltins.indexThreshold.invalidGroupMinimumErrorMessage',
{
defaultMessage: '[groupLimit]: must be greater than 0',
}
);
// check grouping
if (groupBy === 'top') {
if (termField == null) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.termFieldRequiredErrorMessage', {
defaultMessage: '[termField]: termField required when [groupBy] is top',
});
}
if (groupLimit > MAX_GROUPS) {
if (termSize == null) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.termSizeRequiredErrorMessage', {
defaultMessage: '[termSize]: termSize required when [groupBy] is top',
});
}
if (termSize > MAX_GROUPS) {
return i18n.translate(
'xpack.alertingBuiltins.indexThreshold.invalidGroupMaximumErrorMessage',
'xpack.alertingBuiltins.indexThreshold.invalidTermSizeMaximumErrorMessage',
{
defaultMessage: '[groupLimit]: must be less than or equal to {maxGroups}',
defaultMessage: '[termSize]: must be less than or equal to {maxGroups}',
values: {
maxGroups: MAX_GROUPS,
},
@ -81,10 +79,12 @@ export function validateCoreQueryBody(anyParams: any): string | undefined {
}
}
const AggTypes = new Set(['count', 'average', 'min', 'max', 'sum']);
const AggTypes = new Set(['count', 'avg', 'min', 'max', 'sum']);
function validateAggType(aggType: string): string | undefined {
if (AggTypes.has(aggType)) return;
if (AggTypes.has(aggType)) {
return;
}
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidAggTypeErrorMessage', {
defaultMessage: 'invalid aggType: "{aggType}"',
@ -94,15 +94,33 @@ function validateAggType(aggType: string): string | undefined {
});
}
export function validateDuration(duration: string): string | undefined {
try {
parseDuration(duration);
} catch (err) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidDurationErrorMessage', {
defaultMessage: 'invalid duration: "{duration}"',
values: {
duration,
},
});
export function validateGroupBy(groupBy: string): string | undefined {
if (groupBy === 'all' || groupBy === 'top') {
return;
}
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidGroupByErrorMessage', {
defaultMessage: 'invalid groupBy: "{groupBy}"',
values: {
groupBy,
},
});
}
const TimeWindowUnits = new Set(['s', 'm', 'h', 'd']);
export function validateTimeWindowUnits(timeWindowUnit: string): string | undefined {
if (TimeWindowUnits.has(timeWindowUnit)) {
return;
}
return i18n.translate(
'xpack.alertingBuiltins.indexThreshold.invalidTimeWindowUnitsErrorMessage',
{
defaultMessage: 'invalid timeWindowUnit: "{timeWindowUnit}"',
values: {
timeWindowUnit,
},
}
);
}

View file

@ -9,28 +9,30 @@
import { loggingServiceMock } from '../../../../../../../src/core/server/mocks';
import { coreMock } from '../../../../../../../src/core/server/mocks';
import { AlertingBuiltinsPlugin } from '../../../plugin';
import { TimeSeriesQueryParameters, TimeSeriesResult } from './time_series_query';
import { TimeSeriesQueryParameters, TimeSeriesResult, TimeSeriesQuery } from './time_series_query';
type TimeSeriesQuery = (params: TimeSeriesQueryParameters) => Promise<TimeSeriesResult>;
type TimeSeriesQueryFn = (query: TimeSeriesQueryParameters) => Promise<TimeSeriesResult>;
const DefaultQueryParams = {
const DefaultQueryParams: TimeSeriesQuery = {
index: 'index-name',
timeField: 'time-field',
aggType: 'count',
aggField: undefined,
window: '5m',
timeWindowSize: 5,
timeWindowUnit: 'm',
dateStart: undefined,
dateEnd: undefined,
interval: undefined,
groupField: undefined,
groupLimit: undefined,
groupBy: 'all',
termField: undefined,
termSize: undefined,
};
describe('timeSeriesQuery', () => {
let params: TimeSeriesQueryParameters;
const mockCallCluster = jest.fn();
let timeSeriesQuery: TimeSeriesQuery;
let timeSeriesQueryFn: TimeSeriesQueryFn;
beforeEach(async () => {
// rather than use the function from an import, retrieve it from the plugin
@ -38,26 +40,26 @@ describe('timeSeriesQuery', () => {
const plugin = new AlertingBuiltinsPlugin(context);
const coreStart = coreMock.createStart();
const service = await plugin.start(coreStart);
timeSeriesQuery = service.indexThreshold.timeSeriesQuery;
timeSeriesQueryFn = service.indexThreshold.timeSeriesQuery;
mockCallCluster.mockReset();
params = {
logger: loggingServiceMock.create().get(),
callCluster: mockCallCluster,
query: { ...DefaultQueryParams },
query: DefaultQueryParams,
};
});
it('fails as expected when the callCluster call fails', async () => {
mockCallCluster.mockRejectedValue(new Error('woopsie'));
expect(timeSeriesQuery(params)).rejects.toThrowErrorMatchingInlineSnapshot(
expect(timeSeriesQueryFn(params)).rejects.toThrowErrorMatchingInlineSnapshot(
`"error running search"`
);
});
it('fails as expected when the query params are invalid', async () => {
params.query = { ...params.query, dateStart: 'x' };
expect(timeSeriesQuery(params)).rejects.toThrowErrorMatchingInlineSnapshot(
expect(timeSeriesQueryFn(params)).rejects.toThrowErrorMatchingInlineSnapshot(
`"invalid date format for dateStart: \\"x\\""`
);
});

View file

@ -21,8 +21,17 @@ export async function timeSeriesQuery(
params: TimeSeriesQueryParameters
): Promise<TimeSeriesResult> {
const { logger, callCluster, query: queryParams } = params;
const { index, window, interval, timeField, dateStart, dateEnd } = queryParams;
const {
index,
timeWindowSize,
timeWindowUnit,
interval,
timeField,
dateStart,
dateEnd,
} = queryParams;
const window = `${timeWindowSize}${timeWindowUnit}`;
const dateRangeInfo = getDateRangeInfo({ dateStart, dateEnd, window, interval });
// core query
@ -51,10 +60,10 @@ export async function timeSeriesQuery(
};
// add the aggregations
const { aggType, aggField, groupField, groupLimit } = queryParams;
const { aggType, aggField, termField, termSize } = queryParams;
const isCountAgg = aggType === 'count';
const isGroupAgg = !!groupField;
const isGroupAgg = !!termField;
let aggParent = esQuery.body;
@ -63,8 +72,8 @@ export async function timeSeriesQuery(
aggParent.aggs = {
groupAgg: {
terms: {
field: groupField,
size: groupLimit || DEFAULT_GROUPS,
field: termField,
size: termSize || DEFAULT_GROUPS,
},
},
};
@ -83,11 +92,10 @@ export async function timeSeriesQuery(
aggParent = aggParent.aggs.dateAgg;
// finally, the metric aggregation, if requested
const actualAggType = aggType === 'average' ? 'avg' : aggType;
if (!isCountAgg) {
aggParent.aggs = {
metricAgg: {
[actualAggType]: {
[aggType]: {
field: aggField,
},
},

View file

@ -4,14 +4,16 @@
* you may not use this file except in compliance with the Elastic License.
*/
import { TimeSeriesQuerySchema } from './time_series_types';
import { TimeSeriesQuerySchema, TimeSeriesQuery } from './time_series_types';
import { runTests } from './core_query_types.test';
const DefaultParams = {
const DefaultParams: Writable<Partial<TimeSeriesQuery>> = {
index: 'index-name',
timeField: 'time-field',
aggType: 'count',
window: '5m',
groupBy: 'all',
timeWindowSize: 5,
timeWindowUnit: 'm',
};
describe('TimeSeriesParams validate()', () => {
@ -27,10 +29,11 @@ describe('TimeSeriesParams validate()', () => {
});
it('passes for maximal valid input', async () => {
params.aggType = 'average';
params.aggType = 'avg';
params.aggField = 'agg-field';
params.groupField = 'group-field';
params.groupLimit = 100;
params.groupBy = 'top';
params.termField = 'group-field';
params.termSize = 100;
params.dateStart = new Date().toISOString();
params.dateEnd = new Date().toISOString();
params.interval = '1s';

View file

@ -12,11 +12,7 @@ import { schema, TypeOf } from '@kbn/config-schema';
import { parseDuration } from '../../../../../alerting/server';
import { MAX_INTERVALS } from '../index';
import {
CoreQueryParamsSchemaProperties,
validateCoreQueryBody,
validateDuration,
} from './core_query_types';
import { CoreQueryParamsSchemaProperties, validateCoreQueryBody } from './core_query_types';
import {
getTooManyIntervalsErrorMessage,
getDateStartAfterDateEndErrorMessage,
@ -104,3 +100,16 @@ function validateDate(dateString: string): string | undefined {
});
}
}
export function validateDuration(duration: string): string | undefined {
try {
parseDuration(duration);
} catch (err) {
return i18n.translate('xpack.alertingBuiltins.indexThreshold.invalidDurationErrorMessage', {
defaultMessage: 'invalid duration: "{duration}"',
values: {
duration,
},
});
}
}

View file

@ -45,8 +45,6 @@ const DEFAULT_VALUES = {
THRESHOLD_COMPARATOR: COMPARATORS.GREATER_THAN,
TIME_WINDOW_SIZE: 5,
TIME_WINDOW_UNIT: 'm',
TRIGGER_INTERVAL_SIZE: 1,
TRIGGER_INTERVAL_UNIT: 'm',
THRESHOLD: [1000, 5000],
GROUP_BY: 'all',
};
@ -140,7 +138,6 @@ export const IndexThresholdAlertTypeExpression: React.FunctionComponent<IndexThr
thresholdComparator: DEFAULT_VALUES.THRESHOLD_COMPARATOR,
timeWindowSize: DEFAULT_VALUES.TIME_WINDOW_SIZE,
timeWindowUnit: DEFAULT_VALUES.TIME_WINDOW_UNIT,
triggerIntervalUnit: DEFAULT_VALUES.TRIGGER_INTERVAL_UNIT,
groupBy: DEFAULT_VALUES.GROUP_BY,
threshold: DEFAULT_VALUES.THRESHOLD,
});

View file

@ -11,7 +11,7 @@ import { builtInGroupByTypes, builtInAggregationTypes } from '../../../../common
export function getAlertType(): AlertTypeModel {
return {
id: 'threshold',
id: '.index-threshold',
name: 'Index Threshold',
iconClass: 'alert',
alertParamsExpression: IndexThresholdAlertTypeExpression,

View file

@ -61,6 +61,7 @@ export const GroupByExpression = ({
const groupByTypes = customGroupByTypes ?? builtInGroupByTypes;
const [groupByPopoverOpen, setGroupByPopoverOpen] = useState(false);
const MIN_TERM_SIZE = 1;
const MAX_TERM_SIZE = 1000;
const firstFieldOption = {
text: i18n.translate(
'xpack.triggersActionsUI.common.expressionItems.groupByType.timeFieldOptionLabel',
@ -159,6 +160,7 @@ export const GroupByExpression = ({
onChangeSelectedTermSize(termSizeVal);
}}
min={MIN_TERM_SIZE}
max={MAX_TERM_SIZE}
/>
</EuiFormRow>
</EuiFlexItem>

View file

@ -25,8 +25,8 @@ const INTERVAL_MINUTES = 1;
const INTERVAL_DURATION = `${INTERVAL_MINUTES}m`;
const INTERVAL_MILLIS = INTERVAL_MINUTES * 60 * 1000;
const WINDOW_MINUTES = 5;
const WINDOW_DURATION = `${WINDOW_MINUTES}m`;
const WINDOW_DURATION_SIZE = 5;
const WINDOW_DURATION_UNITS = 'm';
// interesting dates pertaining to docs and intervals
const START_DATE_PLUS_YEAR = `2021-${START_DATE_MM_DD_HH_MM_SS_MS}`;
@ -154,7 +154,7 @@ export default function queryDataEndpointTests({ getService }: FtrProviderContex
it('should return correct count for all intervals, grouped', async () => {
const query = getQueryBody({
groupField: 'group',
termField: 'group',
dateStart: START_DATE_MINUS_2INTERVALS,
dateEnd: START_DATE_MINUS_0INTERVALS,
});
@ -185,9 +185,11 @@ export default function queryDataEndpointTests({ getService }: FtrProviderContex
it('should return correct average for all intervals, grouped', async () => {
const query = getQueryBody({
aggType: 'average',
aggType: 'avg',
aggField: 'testedValue',
groupField: 'group',
groupBy: 'top',
termField: 'group',
termSize: 100,
dateStart: START_DATE_MINUS_2INTERVALS,
dateEnd: START_DATE_MINUS_0INTERVALS,
});
@ -266,11 +268,13 @@ function getQueryBody(body: Partial<TimeSeriesQuery> = {}): TimeSeriesQuery {
timeField: 'date',
aggType: 'count',
aggField: undefined,
groupField: undefined,
groupLimit: undefined,
groupBy: 'all',
termField: undefined,
termSize: undefined,
dateStart: START_DATE_MINUS_0INTERVALS,
dateEnd: undefined,
window: WINDOW_DURATION,
timeWindowSize: WINDOW_DURATION_SIZE,
timeWindowUnit: WINDOW_DURATION_UNITS,
interval: INTERVAL_DURATION,
};
return Object.assign({}, defaults, body);

View file

@ -45,16 +45,21 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
it('should create an alert', async () => {
const alertName = generateUniqueKey();
await pageObjects.triggersActionsUI.clickCreateAlertButton();
const nameInput = await testSubjects.find('alertNameInput');
await nameInput.click();
await nameInput.clearValue();
await nameInput.type(alertName);
await testSubjects.click('threshold-SelectOption');
await testSubjects.click('.index-threshold-SelectOption');
await testSubjects.click('selectIndexExpression');
const comboBox = await find.byCssSelector('#indexSelectSearchBox');
await comboBox.click();
await comboBox.type('k');
const filterSelectItem = await find.byCssSelector(`.euiFilterSelectItem`);
await filterSelectItem.click();
await testSubjects.click('thresholdAlertTimeFieldSelect');
const fieldOptions = await find.allByCssSelector('#thresholdTimeField option');
await fieldOptions[1].click();
await testSubjects.click('.slack-ActionTypeSelectOption');
await testSubjects.click('createActionConnectorButton');
const connectorNameInput = await testSubjects.find('nameInput');
@ -62,28 +67,32 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
await connectorNameInput.clearValue();
const connectorName = generateUniqueKey();
await connectorNameInput.type(connectorName);
const slackWebhookUrlInput = await testSubjects.find('slackWebhookUrlInput');
await slackWebhookUrlInput.click();
await slackWebhookUrlInput.clearValue();
await slackWebhookUrlInput.type('https://test');
await find.clickByCssSelector('[data-test-subj="saveActionButtonModal"]:not(disabled)');
const loggingMessageInput = await testSubjects.find('slackMessageTextArea');
await loggingMessageInput.click();
await loggingMessageInput.clearValue();
await loggingMessageInput.type('test message');
await testSubjects.click('slackAddVariableButton');
const variableMenuButton = await testSubjects.find('variableMenuButton-0');
await variableMenuButton.click();
await testSubjects.click('selectIndexExpression');
await find.clickByCssSelector('[data-test-subj="cancelSaveAlertButton"]');
// TODO: implement saving to the server, when threshold API will be ready
// TODO: uncomment variables test when server API will be ready
// await testSubjects.click('slackAddVariableButton');
// const variableMenuButton = await testSubjects.find('variableMenuButton-0');
// await variableMenuButton.click();
await find.clickByCssSelector('[data-test-subj="saveAlertButton"]');
const toastTitle = await pageObjects.common.closeToast();
expect(toastTitle).to.eql(`Saved '${alertName}'`);
await pageObjects.triggersActionsUI.searchAlerts(alertName);
const searchResultsAfterEdit = await pageObjects.triggersActionsUI.getAlertsList();
expect(searchResultsAfterEdit).to.eql([
{
name: alertName,
tagsText: '',
alertType: 'Index Threshold',
interval: '1m',
},
]);
});
it('should search for alert', async () => {