github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-23 17:28:26 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Security Solution][Detections] Adds rule execution log table (#126215)

Browse source 
## Summary

Resolves #119598, #119599, #101014

Test plan ([internal doc](https://docs.google.com/document/d/1-prIUGYaPHiwGA79CgSdw1926lxIPKGWWkYOUD2BM1U/edit#heading=h.womzsfdt6zt8))

Adds `Rule Execution Log` table to Rule Details page:

<p align="center">
  <img width="700" src="https://user-images.githubusercontent.com/2946766/158540840-e9cddb9b-f33d-4b95-86ad-cb3e0a00cf39.gif" />
</p>


### Implementation notes

The useful metrics within `event-log` for a given rule execution are spread between a few different platform (`execute-start`, `execute`) and security (`execution-metrics`, `status-change`) events. In effort to provide consolidated metrics per rule execution (and avoiding a lot of empty cells and mis-matched statuses like in the image below)

<p align="center">
  <img width="700" src="https://user-images.githubusercontent.com/2946766/151933881-2e58f4d7-4cda-4528-9d44-37cb7bd5de9c.png" />
</p>



these rule execution events are aggregated by their `executionId`, and then fields are merged from each different event. This PR was re-worked to take advantage of the new event-log aggregation support added in https://github.com/elastic/kibana/pull/126948, and is no longer implemented as an in-memory aggregation server side.

* Due to restrictions around supplying search filters that may match multiple sub-agg buckets and missing data ([see discussion here](https://github.com/elastic/kibana/pull/127339/files#r825240516)), it was decided that we'd disable the search bar for the time being. We have both a near-term (writing single rollup event) and long-term (ES|QL) solution that will allow us to re-enable this functionality.

* Note, since a `terms` agg is used to fetch all execution events, an upper bound must be set. See [this discussion](https://github.com/elastic/kibana/pull/127339/files#r823035420) for more details, but setting this max to `1000` events for the time being, and returning total cardinality of execution events back within `total` to allow the UI to inform the user that they should narrow their search further to better isolate and find possible issues. This should be a be a reasonable constraint for most all rules as a rule executing every 5 minutes, 1000 executions would cover over 3 days of execution time.

<p align="center">
  <img width="700" src="https://user-images.githubusercontent.com/2946766/159045563-966896b4-3cd1-475d-9f0e-c2d300683546.png" />
</p>


The `Filter for alerts` action will be available on all `Succeeded`/`Partial Failure` executions even if there weren't alerts generated until https://github.com/elastic/kibana/pull/126210 is merged and we can start returning the alert count, at which point we can programmatically enabled/disable this action based on alert count.



<p align="center">
  <img width="300" src="https://user-images.githubusercontent.com/2946766/159051762-e2f97ba4-4ce1-4f67-8ae1-395e4b191cab.png" />
</p>
This commit is contained in:
Garrett Spong 2022-03-28 16:42:46 -06:00 • committed by GitHub
parent ba6be79adb
commit 9bc4c0c22c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
46 changed files with 4595 additions and 183 deletions
Download patch file Download diff file Expand all files Collapse all files

12
packages/kbn-securitysolution-rules/src/configuration_constants.ts Normal file
View file

@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
/**
* Max number of execution events to aggregate in memory for the Rule Execution Log
*/
export const MAX_EXECUTION_EVENTS_DISPLAYED = 1000 as const;

1
packages/kbn-securitysolution-rules/src/index.ts
View file

@ -6,6 +6,7 @@
* Side Public License, v 1.
*/
export * from './configuration_constants';
export * from './rule_type_constants';
export * from './rule_type_mappings';
export * from './utils';

3
x-pack/plugins/security_solution/common/constants.ts
View file

@ -440,3 +440,6 @@ export const RULES_TABLE_PAGE_SIZE_OPTIONS = [5, 10, 20, 50, RULES_TABLE_MAX_PAG
*/
export const RULES_MANAGEMENT_FEATURE_TOUR_STORAGE_KEY =
'securitySolution.rulesManagementPage.newFeaturesTour.v8.1';
export const RULE_DETAILS_EXECUTION_LOG_TABLE_SHOW_METRIC_COLUMNS_STORAGE_KEY =
'securitySolution.ruleDetails.ruleExecutionLog.showMetrics.v8.2';

28
x-pack/plugins/security_solution/common/detection_engine/schemas/common/rule_monitoring.ts
View file

@ -106,3 +106,31 @@ export const ruleExecutionEvent = t.type({
});
export type RuleExecutionEvent = t.TypeOf<typeof ruleExecutionEvent>;
// -------------------------------------------------------------------------------------------------
// Aggregate Rule execution events
export const aggregateRuleExecutionEvent = t.type({
execution_uuid: t.string,
timestamp: IsoDateString,
duration_ms: t.number,
status: t.string,
message: t.string,
num_active_alerts: t.number,
num_new_alerts: t.number,
num_recovered_alerts: t.number,
num_triggered_actions: t.number,
num_succeeded_actions: t.number,
num_errored_actions: t.number,
total_search_duration_ms: t.number,
es_search_duration_ms: t.number,
schedule_delay_ms: t.number,
timed_out: t.boolean,
indexing_duration_ms: t.number,
search_duration_ms: t.number,
gap_duration_ms: t.number,
security_status: t.string,
security_message: t.string,
});
export type AggregateRuleExecutionEvent = t.TypeOf<typeof aggregateRuleExecutionEvent>;

15
x-pack/plugins/security_solution/common/detection_engine/schemas/request/get_rule_execution_events_request.ts → x-pack/plugins/security_solution/common/detection_engine/schemas/request/get_rule_execution_events_schema.ts
View file

@ -7,12 +7,27 @@
import * as t from 'io-ts';
import { sortFieldOrUndefined, sortOrderOrUndefined } from '../common';
export const GetRuleExecutionEventsRequestParams = t.exact(
t.type({
ruleId: t.string,
})
);
export const GetRuleExecutionEventsQueryParams = t.exact(
t.type({
start: t.string,
end: t.string,
query_text: t.union([t.string, t.undefined]),
status_filters: t.union([t.string, t.undefined]),
per_page: t.union([t.string, t.undefined]),
page: t.union([t.string, t.undefined]),
sort_field: sortFieldOrUndefined,
sort_order: sortOrderOrUndefined,
})
);
export type GetRuleExecutionEventsRequestParams = t.TypeOf<
typeof GetRuleExecutionEventsRequestParams
>;

13
x-pack/plugins/security_solution/common/detection_engine/schemas/response/get_rule_execution_events_response.ts
View file

@ -6,7 +6,7 @@
*/
import * as t from 'io-ts';
import { ruleExecutionEvent } from '../common';
import { aggregateRuleExecutionEvent, ruleExecutionEvent } from '../common';
export const GetRuleExecutionEventsResponse = t.exact(
t.type({
@ -15,3 +15,14 @@ export const GetRuleExecutionEventsResponse = t.exact(
);
export type GetRuleExecutionEventsResponse = t.TypeOf<typeof GetRuleExecutionEventsResponse>;
export const GetAggregateRuleExecutionEventsResponse = t.exact(
t.type({
events: t.array(aggregateRuleExecutionEvent),
total: t.number,
})
);
export type GetAggregateRuleExecutionEventsResponse = t.TypeOf<
typeof GetAggregateRuleExecutionEventsResponse
>;

36
x-pack/plugins/security_solution/public/detections/containers/detection_engine/rules/__mocks__/api.ts
View file

@ -5,9 +5,8 @@
* 2.0.
*/
import { RuleExecutionStatus } from '../../../../../../common/detection_engine/schemas/common';
import {
GetRuleExecutionEventsResponse,
GetAggregateRuleExecutionEventsResponse,
RulesSchema,
} from '../../../../../../common/detection_engine/schemas/response';
@ -62,19 +61,44 @@ export const fetchRules = async (_: FetchRulesProps): Promise<FetchRulesResponse
export const fetchRuleExecutionEvents = async ({
ruleId,
start,
end,
filters,
signal,
}: {
ruleId: string;
start: string;
end: string;
filters?: string;
signal?: AbortSignal;
}): Promise<GetRuleExecutionEventsResponse> => {
}): Promise<GetAggregateRuleExecutionEventsResponse> => {
return Promise.resolve({
events: [
{
date: '2021-12-29T10:42:59.996Z',
status: RuleExecutionStatus.succeeded,
message: 'Rule executed successfully',
duration_ms: 3866,
es_search_duration_ms: 1236,
execution_uuid: '88d15095-7937-462c-8f21-9763e1387cad',
gap_duration_ms: 0,
indexing_duration_ms: 95,
message:
"rule executed: siem.queryRule:fb1fc150-a292-11ec-a2cf-c1b28b0392b0: 'Lots of Execution Events'",
num_active_alerts: 0,
num_errored_actions: 0,
num_new_alerts: 0,
num_recovered_alerts: 0,
num_succeeded_actions: 1,
num_triggered_actions: 1,
schedule_delay_ms: -127535,
search_duration_ms: 1255,
security_message: 'succeeded',
security_status: 'succeeded',
status: 'success',
timed_out: false,
timestamp: '2022-03-13T06:04:05.838Z',
total_search_duration_ms: 0,
},
],
total: 1,
});
};

28
x-pack/plugins/security_solution/public/detections/containers/detection_engine/rules/api.test.ts
View file

@ -595,19 +595,43 @@ describe('Detections Rules API', () => {
});
test('calls API with correct parameters', async () => {
await fetchRuleExecutionEvents({ ruleId: '42', signal: abortCtrl.signal });
await fetchRuleExecutionEvents({
ruleId: '42',
start: '2001-01-01T17:00:00.000Z',
end: '2001-01-02T17:00:00.000Z',
queryText: '',
statusFilters: '',
signal: abortCtrl.signal,
});
expect(fetchMock).toHaveBeenCalledWith(
'/internal/detection_engine/rules/42/execution/events',
{
method: 'GET',
query: {
end: '2001-01-02T17:00:00.000Z',
page: undefined,
per_page: undefined,
query_text: '',
sort_field: undefined,
sort_order: undefined,
start: '2001-01-01T17:00:00.000Z',
status_filters: '',
},
signal: abortCtrl.signal,
}
);
});
test('returns API response as is', async () => {
const response = await fetchRuleExecutionEvents({ ruleId: '42', signal: abortCtrl.signal });
const response = await fetchRuleExecutionEvents({
ruleId: '42',
start: 'now-30',
end: 'now',
queryText: '',
statusFilters: '',
signal: abortCtrl.signal,
});
expect(response).toEqual(responseMock);
});
});

45
x-pack/plugins/security_solution/public/detections/containers/detection_engine/rules/api.ts
View file

@ -6,6 +6,7 @@
*/
import { camelCase } from 'lodash';
import dateMath from '@elastic/datemath';
import { HttpStart } from 'src/core/public';
import {
@ -24,7 +25,7 @@ import {
} from '../../../../../common/detection_engine/schemas/request';
import {
RulesSchema,
GetRuleExecutionEventsResponse,
GetAggregateRuleExecutionEventsResponse,
} from '../../../../../common/detection_engine/schemas/response';
import {
@ -315,21 +316,57 @@ export const exportRules = async ({
/**
* Fetch rule execution events (e.g. status changes) from Event Log.
*
* @param ruleId string Saved Object ID of the rule (`rule.id`, not static `rule.rule_id`)
* @param ruleId Saved Object ID of the rule (`rule.id`, not static `rule.rule_id`)
* @param start Start daterange either in UTC ISO8601 or as datemath string (e.g. `2021-12-29T02:44:41.653Z` or `now-30`)
* @param end End daterange either in UTC ISO8601 or as datemath string (e.g. `2021-12-29T02:44:41.653Z` or `now/w`)
* @param queryText search string in querystring format (e.g. `event.duration > 1000 OR kibana.alert.rule.execution.metrics.execution_gap_duration_s > 100`)
* @param statusFilters comma separated string of `statusFilters` (e.g. `succeeded,failed,partial failure`)
* @param page current page to fetch
* @param perPage number of results to fetch per page
* @param sortField field to sort by
* @param sortOrder what order to sort by (e.g. `asc` or `desc`)
* @param signal AbortSignal Optional signal for cancelling the request
*
* @throws An error if response is not OK
*/
export const fetchRuleExecutionEvents = async ({
ruleId,
start,
end,
queryText,
statusFilters,
page,
perPage,
sortField,
sortOrder,
signal,
}: {
ruleId: string;
start: string;
end: string;
queryText?: string;
statusFilters?: string;
page?: number;
perPage?: number;
sortField?: string;
sortOrder?: string;
signal?: AbortSignal;
}): Promise<GetRuleExecutionEventsResponse> => {
}): Promise<GetAggregateRuleExecutionEventsResponse> => {
const url = detectionEngineRuleExecutionEventsUrl(ruleId);
return KibanaServices.get().http.fetch<GetRuleExecutionEventsResponse>(url, {
const startDate = dateMath.parse(start);
const endDate = dateMath.parse(end, { roundUp: true });
return KibanaServices.get().http.fetch<GetAggregateRuleExecutionEventsResponse>(url, {
method: 'GET',
query: {
start: startDate?.utc().toISOString(),
end: endDate?.utc().toISOString(),
query_text: queryText?.trim(),
status_filters: statusFilters?.trim(),
page,
per_page: perPage,
sort_field: sortField,
sort_order: sortOrder,
},
signal,
});
};

7
x-pack/plugins/security_solution/public/detections/containers/detection_engine/rules/translations.ts
View file

@ -14,6 +14,13 @@ export const RULE_AND_TIMELINE_FETCH_FAILURE = i18n.translate(
}
);
export const RULE_EXECUTION_FETCH_FAILURE = i18n.translate(
'xpack.securitySolution.containers.detectionEngine.ruleExecutionLogFailureDescription',
{
defaultMessage: 'Failed to fetch Rule Execution Events',
}
);
export const RULE_ADD_FAILURE = i18n.translate(
'xpack.securitySolution.containers.detectionEngine.addRuleFailDescription',
{

52
x-pack/plugins/security_solution/public/detections/containers/detection_engine/rules/use_rule_execution_events.test.tsx
View file

@ -12,7 +12,6 @@ import React from 'react';
import { QueryClient, QueryClientProvider } from 'react-query';
import { renderHook, cleanup } from '@testing-library/react-hooks';
import { RuleExecutionStatus } from '../../../../../common/detection_engine/schemas/common';
import { useRuleExecutionEvents } from './use_rule_execution_events';
import * as api from './api';
@ -45,9 +44,19 @@ describe('useRuleExecutionEvents', () => {
};
const render = () =>
renderHook(() => useRuleExecutionEvents(SOME_RULE_ID), {
wrapper: createReactQueryWrapper(),
});
renderHook(
() =>
useRuleExecutionEvents({
ruleId: SOME_RULE_ID,
start: 'now-30',
end: 'now',
queryText: '',
statusFilters: '',
}),
{
wrapper: createReactQueryWrapper(),
}
);
it('calls the API via fetchRuleExecutionEvents', async () => {
const fetchRuleExecutionEvents = jest.spyOn(api, 'fetchRuleExecutionEvents');
@ -77,13 +86,34 @@ describe('useRuleExecutionEvents', () => {
expect(result.current.isLoading).toEqual(false);
expect(result.current.isSuccess).toEqual(true);
expect(result.current.isError).toEqual(false);
expect(result.current.data).toEqual([
{
date: '2021-12-29T10:42:59.996Z',
status: RuleExecutionStatus.succeeded,
message: 'Rule executed successfully',
},
]);
expect(result.current.data).toEqual({
events: [
{
duration_ms: 3866,
es_search_duration_ms: 1236,
execution_uuid: '88d15095-7937-462c-8f21-9763e1387cad',
gap_duration_ms: 0,
indexing_duration_ms: 95,
message:
"rule executed: siem.queryRule:fb1fc150-a292-11ec-a2cf-c1b28b0392b0: 'Lots of Execution Events'",
num_active_alerts: 0,
num_errored_actions: 0,
num_new_alerts: 0,
num_recovered_alerts: 0,
num_succeeded_actions: 1,
num_triggered_actions: 1,
schedule_delay_ms: -127535,
search_duration_ms: 1255,
security_message: 'succeeded',
security_status: 'succeeded',
status: 'success',
timed_out: false,
timestamp: '2022-03-13T06:04:05.838Z',
total_search_duration_ms: 0,
},
],
total: 1,
});
});
it('handles exceptions from the API', async () => {

54
x-pack/plugins/security_solution/public/detections/containers/detection_engine/rules/use_rule_execution_events.tsx
View file

@ -6,18 +6,62 @@
*/
import { useQuery } from 'react-query';
import { GetAggregateRuleExecutionEventsResponse } from '../../../../../common/detection_engine/schemas/response';
import { useAppToasts } from '../../../../common/hooks/use_app_toasts';
import { fetchRuleExecutionEvents } from './api';
import * as i18n from './translations';
export const useRuleExecutionEvents = (ruleId: string) => {
interface UseRuleExecutionEventsArgs {
ruleId: string;
start: string;
end: string;
queryText?: string;
statusFilters?: string;
page?: number;
perPage?: number;
sortField?: string;
sortOrder?: string;
}
export const useRuleExecutionEvents = ({
ruleId,
start,
end,
queryText,
statusFilters,
page,
perPage,
sortField,
sortOrder,
}: UseRuleExecutionEventsArgs) => {
const { addError } = useAppToasts();
return useQuery(
['ruleExecutionEvents', ruleId],
return useQuery<GetAggregateRuleExecutionEventsResponse>(
[
'ruleExecutionEvents',
ruleId,
start,
end,
queryText,
statusFilters,
page,
perPage,
sortField,
sortOrder,
],
async ({ signal }) => {
const response = await fetchRuleExecutionEvents({ ruleId, signal });
return response.events;
return fetchRuleExecutionEvents({
ruleId,
start,
end,
queryText,
statusFilters,
page,
perPage,
sortField,
sortOrder,
signal,
});
},
{
onError: (e) => {

5
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/all/popover_tooltip.tsx
View file

@ -30,7 +30,10 @@ const PopoverTooltipComponent = ({ columnName, children }: PopoverTooltipProps)
button={
<EuiButtonIcon
aria-label={i18n.POPOVER_TOOLTIP_ARIA_LABEL(columnName)}
onClick={() => setIsPopoverOpen(!isPopoverOpen)}
onClick={(event: React.MouseEvent<HTMLButtonElement>) => {
setIsPopoverOpen(!isPopoverOpen);
event.stopPropagation();
}}
size="xs"
color="primary"
iconType="questionInCircle"

2
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/all/use_columns.tsx
View file

@ -346,7 +346,7 @@ export const useMonitoringColumns = ({ hasPermissions }: ColumnsProps): TableCol
href={`${docLinks.links.siem.troubleshootGaps}`}
target="_blank"
>
{'see documentation'}
{i18n.COLUMN_GAP_TOOLTIP_SEE_DOCUMENTATION}
</EuiLink>
),
}}

78
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/__snapshots__/execution_log_search_bar.test.tsx.snap generated Normal file
View file

@ -0,0 +1,78 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`ExecutionLogSearchBar snapshots renders correctly against snapshot 1`] = `
<EuiFlexGroup
gutterSize="s"
>
<EuiFlexItem
grow={true}
/>
<EuiFlexItem
grow={false}
>
<EuiFilterGroup>
<EuiPopover
anchorPosition="downCenter"
button={
<EuiFilterButton
data-test-subj="status-filter-popover-button"
grow={false}
hasActiveFilters={false}
iconType="arrowDown"
isSelected={false}
numActiveFilters={0}
numFilters={3}
onClick={[Function]}
>
Status
</EuiFilterButton>
}
closePopover={[Function]}
display="inlineBlock"
hasArrow={true}
isOpen={false}
ownFocus={true}
panelPaddingSize="none"
repositionOnScroll={true}
>
<EuiFilterSelectItem
key="0-succeeded"
onClick={[Function]}
showIcons={true}
title="succeeded"
>
<EuiHealth
color="success"
>
Succeeded
</EuiHealth>
</EuiFilterSelectItem>
<EuiFilterSelectItem
key="1-failed"
onClick={[Function]}
showIcons={true}
title="failed"
>
<EuiHealth
color="danger"
>
Failed
</EuiHealth>
</EuiFilterSelectItem>
<EuiFilterSelectItem
key="2-partial failure"
onClick={[Function]}
showIcons={true}
title="partial failure"
>
<EuiHealth
color="warning"
>
Partial failure
</EuiHealth>
</EuiFilterSelectItem>
</EuiPopover>
</EuiFilterGroup>
</EuiFlexItem>
</EuiFlexGroup>
`;

210
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/__snapshots__/execution_log_table.test.tsx.snap generated Normal file
View file

@ -0,0 +1,210 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`ExecutionLogTable snapshots renders correctly against snapshot 1`] = `
<Fragment>
<EuiFlexGroup
gutterSize="s"
>
<EuiFlexItem
grow={true}
>
<ExecutionLogSearchBar
onSearch={[Function]}
onStatusFilterChange={[Function]}
onlyShowFilters={true}
/>
</EuiFlexItem>
<EuiFlexItem
style={
Object {
"maxWidth": "582px",
}
}
>
<EuiSuperDatePicker
commonlyUsedRanges={
Array [
Object {
"end": "now/d",
"label": "Today",
"start": "now/d",
},
Object {
"end": "now/w",
"label": "This week",
"start": "now/w",
},
Object {
"end": "now/M",
"label": "This month",
"start": "now/M",
},
Object {
"end": "now/y",
"label": "This year",
"start": "now/y",
},
Object {
"end": "now-1d/d",
"label": "Yesterday",
"start": "now-1d/d",
},
Object {
"end": "now",
"label": "Week to date",
"start": "now/w",
},
Object {
"end": "now",
"label": "Month to date",
"start": "now/M",
},
Object {
"end": "now",
"label": "Year to date",
"start": "now/y",
},
]
}
dateFormat="MMM D, YYYY @ HH:mm:ss.SSS"
end="now"
isAutoRefreshOnly={false}
isDisabled={false}
isPaused={true}
onRefresh={[Function]}
onRefreshChange={[Function]}
onTimeChange={[Function]}
recentlyUsedRanges={Array []}
refreshInterval={1000}
showUpdateButton={true}
start="now-24h"
timeFormat="HH:mm"
width="full"
/>
</EuiFlexItem>
</EuiFlexGroup>
<EuiSpacer
size="s"
/>
<UtilityBar>
<UtilityBarSection>
<UtilityBarGroup>
<UtilityBarText
dataTestSubj="executionsShowing"
>
Showing 0 rule executions
</UtilityBarText>
</UtilityBarGroup>
</UtilityBarSection>
<UtilityBarSection>
<UtilityBarText
dataTestSubj="executionsShowing"
/>
<Styled(EuiSwitch)
checked={false}
compressed={true}
label="Show metrics columns"
onChange={[Function]}
/>
</UtilityBarSection>
</UtilityBar>
<EuiBasicTable
columns={
Array [
Object {
"field": "security_status",
"name": <Memo(TableHeaderTooltipCell)
title="Status"
tooltipContent="Overall status of execution."
/>,
"render": [Function],
"sortable": false,
"truncateText": false,
"width": "10%",
},
Object {
"field": "timestamp",
"name": <Memo(TableHeaderTooltipCell)
title="Timestamp"
tooltipContent="Datetime rule execution initiated."
/>,
"render": [Function],
"sortable": true,
"truncateText": false,
"width": "15%",
},
Object {
"field": "duration_ms",
"name": <Memo(TableHeaderTooltipCell)
title="Duration"
tooltipContent="The length of time it took for the rule to run (hh:mm:ss:SSS)."
/>,
"render": [Function],
"sortable": true,
"truncateText": false,
"width": "10%",
},
Object {
"field": "security_message",
"name": <Memo(TableHeaderTooltipCell)
title="Message"
tooltipContent="Relevant message from execution outcome."
/>,
"render": [Function],
"sortable": false,
"truncateText": false,
"width": "35%",
},
Object {
"actions": Array [
Object {
"data-test-subj": "action-filter-by-execution-id",
"description": "Filter alerts by rule execution ID.",
"field": "",
"icon": "filter",
"isPrimary": true,
"name": "Edit",
"onClick": [Function],
"type": "icon",
},
],
"field": "kibana.alert.rule.execution.uuid",
"name": "Actions",
"width": "5%",
},
]
}
items={Array []}
noItemsMessage={
<EuiI18n
default="No items found"
token="euiBasicTable.noItemsMessage"
/>
}
onChange={[Function]}
pagination={
Object {
"pageIndex": 0,
"pageSize": 5,
"pageSizeOptions": Array [
5,
10,
25,
50,
],
"totalItemCount": 0,
}
}
responsive={true}
sorting={
Object {
"sort": Object {
"direction": "desc",
"field": "timestamp",
},
}
}
tableLayout="fixed"
/>
</Fragment>
`;

9
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/__snapshots__/rule_duration_format.test.tsx.snap generated Normal file
View file

@ -0,0 +1,9 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`RuleDurationFormat snapshots renders correctly against snapshot 1`] = `
<span
data-test-subj="rule-duration-format-value"
>
00:00:00:000
</span>
`;

171
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/execution_log_columns.tsx Normal file
View file

@ -0,0 +1,171 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { EuiBasicTableColumn, EuiHealth, EuiLink, EuiText } from '@elastic/eui';
import { capitalize } from 'lodash';
import { FormattedMessage } from '@kbn/i18n-react';
import { DocLinksStart } from 'kibana/public';
import React from 'react';
import {
AggregateRuleExecutionEvent,
RuleExecutionStatus,
} from '../../../../../../../common/detection_engine/schemas/common';
import { getEmptyTagValue, getEmptyValue } from '../../../../../../common/components/empty_value';
import { FormattedDate } from '../../../../../../common/components/formatted_date';
import { getStatusColor } from '../../../../../components/rules/rule_execution_status/utils';
import { PopoverTooltip } from '../../all/popover_tooltip';
import { TableHeaderTooltipCell } from '../../all/table_header_tooltip_cell';
import * as i18n from './translations';
import { RuleDurationFormat } from './rule_duration_format';
export const EXECUTION_LOG_COLUMNS: Array<EuiBasicTableColumn<AggregateRuleExecutionEvent>> = [
{
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_STATUS}
tooltipContent={i18n.COLUMN_STATUS_TOOLTIP}
/>
),
field: 'security_status',
render: (value: RuleExecutionStatus, data) =>
value ? (
<EuiHealth color={getStatusColor(value)}>{capitalize(value)}</EuiHealth>
) : (
getEmptyTagValue()
),
sortable: false,
truncateText: false,
width: '10%',
},
{
field: 'timestamp',
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_TIMESTAMP}
tooltipContent={i18n.COLUMN_TIMESTAMP_TOOLTIP}
/>
),
render: (value: string) => <FormattedDate value={value} fieldName="date" />,
sortable: true,
truncateText: false,
width: '15%',
},
{
field: 'duration_ms',
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_DURATION}
tooltipContent={i18n.COLUMN_DURATION_TOOLTIP}
/>
),
render: (value: number) => (
<>{value ? <RuleDurationFormat duration={value} /> : getEmptyValue()}</>
),
sortable: true,
truncateText: false,
width: '10%',
},
{
field: 'security_message',
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_MESSAGE}
tooltipContent={i18n.COLUMN_MESSAGE_TOOLTIP}
/>
),
render: (value: string) => <>{value}</>,
sortable: false,
truncateText: false,
width: '35%',
},
];
export const GET_EXECUTION_LOG_METRICS_COLUMNS = (
docLinks: DocLinksStart
): Array<EuiBasicTableColumn<AggregateRuleExecutionEvent>> => [
{
field: 'gap_duration_ms',
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_GAP_DURATION}
customTooltip={
<div style={{ maxWidth: '20px' }}>
<PopoverTooltip columnName={i18n.COLUMN_GAP_DURATION}>
<EuiText size={'s'} style={{ width: 350 }}>
<p>
<FormattedMessage
defaultMessage="Duration of gap in Rule execution (hh:mm:ss:SSS). Adjust Rule look-back or {seeDocs} for mitigating gaps."
id="xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.gapDurationColumnTooltip"
values={{
seeDocs: (
<EuiLink href={`${docLinks.links.siem.troubleshootGaps}`} target="_blank">
{i18n.COLUMN_GAP_TOOLTIP_SEE_DOCUMENTATION}
</EuiLink>
),
}}
/>
</p>
</EuiText>
</PopoverTooltip>
</div>
}
/>
),
render: (value: number) => (
<>{value ? <RuleDurationFormat duration={value} isMillis={true} /> : getEmptyValue()}</>
),
sortable: true,
truncateText: false,
width: '10%',
},
{
field: 'indexing_duration_ms',
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_INDEX_DURATION}
tooltipContent={i18n.COLUMN_INDEX_DURATION_TOOLTIP}
/>
),
render: (value: number) => (
<>{value ? <RuleDurationFormat duration={value} /> : getEmptyValue()}</>
),
sortable: true,
truncateText: false,
width: '10%',
},
{
field: 'search_duration_ms',
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_SEARCH_DURATION}
tooltipContent={i18n.COLUMN_SEARCH_DURATION_TOOLTIP}
/>
),
render: (value: number) => (
<>{value ? <RuleDurationFormat duration={value} /> : getEmptyValue()}</>
),
sortable: true,
truncateText: false,
width: '10%',
},
{
field: 'schedule_delay_ms',
name: (
<TableHeaderTooltipCell
title={i18n.COLUMN_SCHEDULING_DELAY}
tooltipContent={i18n.COLUMN_SCHEDULING_DELAY_TOOLTIP}
/>
),
render: (value: number) => (
<>{value ? <RuleDurationFormat duration={value} /> : getEmptyValue()}</>
),
sortable: true,
truncateText: false,
width: '10%',
},
];

24
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/execution_log_search_bar.test.tsx Normal file
View file

@ -0,0 +1,24 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { shallow } from 'enzyme';
import React from 'react';
import { ExecutionLogSearchBar } from './execution_log_search_bar';
import { noop } from 'lodash/fp';
// TODO: Replace snapshot test with base test cases
describe('ExecutionLogSearchBar', () => {
describe('snapshots', () => {
test('renders correctly against snapshot', () => {
const wrapper = shallow(
<ExecutionLogSearchBar onlyShowFilters={true} onSearch={noop} onStatusFilterChange={noop} />
);
expect(wrapper).toMatchSnapshot();
});
});
});

147
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/execution_log_search_bar.tsx Normal file
View file

@ -0,0 +1,147 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import React, { useCallback, useEffect, useMemo, useState } from 'react';
import { capitalize, replace } from 'lodash';
import {
EuiHealth,
EuiFieldSearch,
EuiFlexGroup,
EuiFlexItem,
EuiPopover,
EuiFilterGroup,
EuiFilterButton,
EuiFilterSelectItem,
} from '@elastic/eui';
import { RuleExecutionStatus } from '../../../../../../../common/detection_engine/schemas/common';
import { getStatusColor } from '../../../../../components/rules/rule_execution_status/utils';
import * as i18n from './translations';
export const EXECUTION_LOG_SCHEMA_MAPPING = {
status: 'kibana.alert.rule.execution.status',
timestamp: '@timestamp',
duration: 'event.duration',
message: 'message',
gapDuration: 'kibana.alert.rule.execution.metrics.execution_gap_duration_s',
indexDuration: 'kibana.alert.rule.execution.metrics.total_indexing_duration_ms',
searchDuration: 'kibana.alert.rule.execution.metrics.total_search_duration_ms',
totalActions: 'kibana.alert.rule.execution.metrics.number_of_triggered_actions',
schedulingDelay: 'kibana.task.schedule_delay',
};
export const replaceQueryTextAliases = (queryText: string): string => {
return Object.entries(EXECUTION_LOG_SCHEMA_MAPPING).reduce<string>(
(updatedQuery, [key, value]) => {
return replace(updatedQuery, key, value);
},
queryText
);
};
const statuses = [
RuleExecutionStatus.succeeded,
RuleExecutionStatus.failed,
RuleExecutionStatus['partial failure'],
];
const statusFilters = statuses.map((status) => ({
label: <EuiHealth color={getStatusColor(status)}>{capitalize(status)}</EuiHealth>,
selected: false,
}));
interface ExecutionLogTableSearchProps {
onlyShowFilters: true;
onSearch: (queryText: string) => void;
onStatusFilterChange: (statusFilters: string[]) => void;
}
export const ExecutionLogSearchBar = React.memo<ExecutionLogTableSearchProps>(
({ onlyShowFilters, onSearch, onStatusFilterChange }) => {
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
const [selectedFilters, setSelectedFilters] = useState<RuleExecutionStatus[]>([]);
const onSearchCallback = useCallback(
(queryText: string) => {
onSearch(replaceQueryTextAliases(queryText));
},
[onSearch]
);
const onStatusFilterChangeCallback = useCallback(
(filter: RuleExecutionStatus) => {
setSelectedFilters(
selectedFilters.includes(filter)
? selectedFilters.filter((f) => f !== filter)
: [...selectedFilters, filter]
);
},
[selectedFilters]
);
const filtersComponent = useMemo(() => {
return statuses.map((filter, index) => (
<EuiFilterSelectItem
checked={selectedFilters.includes(filter) ? 'on' : undefined}
key={`${index}-${filter}`}
onClick={() => onStatusFilterChangeCallback(filter)}
title={filter}
>
<EuiHealth color={getStatusColor(filter)}>{capitalize(filter)}</EuiHealth>
</EuiFilterSelectItem>
));
}, [onStatusFilterChangeCallback, selectedFilters]);
useEffect(() => {
onStatusFilterChange(selectedFilters);
}, [onStatusFilterChange, selectedFilters]);
return (
<EuiFlexGroup gutterSize={'s'}>
<EuiFlexItem grow={true}>
{!onlyShowFilters && (
<EuiFieldSearch
data-test-subj="executionLogSearch"
aria-label={i18n.RULE_EXECUTION_LOG_SEARCH_PLACEHOLDER}
placeholder={i18n.RULE_EXECUTION_LOG_SEARCH_PLACEHOLDER}
onSearch={onSearchCallback}
isClearable={true}
fullWidth={true}
/>
)}
</EuiFlexItem>
<EuiFlexItem grow={false}>
<EuiFilterGroup>
<EuiPopover
button={
<EuiFilterButton
grow={false}
data-test-subj={'status-filter-popover-button'}
iconType="arrowDown"
onClick={() => setIsPopoverOpen(!isPopoverOpen)}
numFilters={statusFilters.length}
isSelected={isPopoverOpen}
hasActiveFilters={selectedFilters.length > 0}
numActiveFilters={selectedFilters.length}
>
{i18n.COLUMN_STATUS}
</EuiFilterButton>
}
isOpen={isPopoverOpen}
closePopover={() => setIsPopoverOpen(false)}
panelPaddingSize="none"
repositionOnScroll
>
{filtersComponent}
</EuiPopover>
</EuiFilterGroup>
</EuiFlexItem>
</EuiFlexGroup>
);
}
);
ExecutionLogSearchBar.displayName = 'ExecutionLogSearchBar';

103
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/execution_log_table.test.tsx Normal file
View file

@ -0,0 +1,103 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import React from 'react';
import { shallow } from 'enzyme';
import { noop } from 'lodash/fp';
import { useSourcererDataView } from '../../../../../../common/containers/sourcerer';
import { ExecutionLogTable } from './execution_log_table';
jest.mock('../../../../../containers/detection_engine/rules', () => {
const original = jest.requireActual('../../../../../containers/detection_engine/rules');
return {
...original,
useRuleExecutionEvents: jest.fn().mockReturnValue({
loading: true,
setQuery: () => undefined,
data: null,
response: '',
request: '',
refetch: null,
}),
};
});
jest.mock('../../../../../../common/containers/sourcerer');
jest.mock('../../../../../../common/hooks/use_app_toasts', () => {
const original = jest.requireActual('../../../../../../common/hooks/use_app_toasts');
return {
...original,
useAppToasts: () => ({
addSuccess: jest.fn(),
addError: jest.fn(),
}),
};
});
jest.mock('react-redux', () => {
const original = jest.requireActual('react-redux');
return {
...original,
useDispatch: () => jest.fn(),
useSelector: () => jest.fn(),
};
});
jest.mock('../../../../../../common/lib/kibana', () => {
const original = jest.requireActual('../../../../../../common/lib/kibana');
return {
...original,
useUiSetting$: jest.fn().mockReturnValue([]),
useKibana: () => ({
services: {
data: {
query: {
filterManager: jest.fn().mockReturnValue({}),
},
},
docLinks: {
links: {
siem: {
troubleshootGaps: 'link',
},
},
},
storage: {
get: jest.fn(),
set: jest.fn(),
},
timelines: {
getLastUpdated: jest.fn(),
getFieldBrowser: jest.fn(),
},
},
}),
};
});
const mockUseSourcererDataView = useSourcererDataView as jest.Mock;
mockUseSourcererDataView.mockReturnValue({
missingPatterns: {},
selectedPatterns: {},
scopeSelectedPatterns: {},
loading: false,
});
// TODO: Replace snapshot test with base test cases
describe('ExecutionLogTable', () => {
describe('snapshots', () => {
test('renders correctly against snapshot', () => {
const wrapper = shallow(<ExecutionLogTable ruleId={'0'} selectAlertsTab={noop} />);
expect(wrapper).toMatchSnapshot();
});
});
});

325
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/execution_log_table.tsx Normal file
View file

@ -0,0 +1,325 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { SortOrder } from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { DurationRange } from '@elastic/eui/src/components/date_picker/types';
import { get } from 'lodash';
import styled from 'styled-components';
import React, { useCallback, useMemo, useState } from 'react';
import {
EuiTextColor,
EuiFlexGroup,
EuiFlexItem,
EuiSuperDatePicker,
OnTimeChangeProps,
OnRefreshProps,
OnRefreshChangeProps,
EuiSpacer,
EuiSwitch,
EuiBasicTable,
} from '@elastic/eui';
import { buildFilter, FILTERS } from '@kbn/es-query';
import { MAX_EXECUTION_EVENTS_DISPLAYED } from '@kbn/securitysolution-rules';
import { RULE_DETAILS_EXECUTION_LOG_TABLE_SHOW_METRIC_COLUMNS_STORAGE_KEY } from '../../../../../../../common/constants';
import { AggregateRuleExecutionEvent } from '../../../../../../../common/detection_engine/schemas/common';
import {
UtilityBar,
UtilityBarGroup,
UtilityBarSection,
UtilityBarText,
} from '../../../../../../common/components/utility_bar';
import { useSourcererDataView } from '../../../../../../common/containers/sourcerer';
import { useAppToasts } from '../../../../../../common/hooks/use_app_toasts';
import { useKibana } from '../../../../../../common/lib/kibana';
import { SourcererScopeName } from '../../../../../../common/store/sourcerer/model';
import { useRuleExecutionEvents } from '../../../../../containers/detection_engine/rules';
import * as i18n from './translations';
import { EXECUTION_LOG_COLUMNS, GET_EXECUTION_LOG_METRICS_COLUMNS } from './execution_log_columns';
import { ExecutionLogSearchBar } from './execution_log_search_bar';
const EXECUTION_UUID_FIELD_NAME = 'kibana.alert.rule.execution.uuid';
const UtilitySwitch = styled(EuiSwitch)`
margin-left: 17px;
`;
interface ExecutionLogTableProps {
ruleId: string;
selectAlertsTab: () => void;
}
const ExecutionLogTableComponent: React.FC<ExecutionLogTableProps> = ({
ruleId,
selectAlertsTab,
}) => {
const {
docLinks,
data: {
query: { filterManager },
},
storage,
timelines,
} = useKibana().services;
// Datepicker state
const [recentlyUsedRanges, setRecentlyUsedRanges] = useState<DurationRange[]>([]);
const [refreshInterval, setRefreshInterval] = useState(1000);
const [isPaused, setIsPaused] = useState(true);
const [start, setStart] = useState('now-24h');
const [end, setEnd] = useState('now');
// Searchbar/Filter/Settings state
const [queryText, setQueryText] = useState('');
const [statusFilters, setStatusFilters] = useState('');
const [showMetricColumns, setShowMetricColumns] = useState<boolean>(
storage.get(RULE_DETAILS_EXECUTION_LOG_TABLE_SHOW_METRIC_COLUMNS_STORAGE_KEY) ?? false
);
// Pagination state
const [pageIndex, setPageIndex] = useState(0);
const [pageSize, setPageSize] = useState(5);
const [sortField, setSortField] = useState<keyof AggregateRuleExecutionEvent>('timestamp');
const [sortDirection, setSortDirection] = useState<SortOrder>('desc');
// Index for `add filter` action and toasts for errors
const { indexPattern } = useSourcererDataView(SourcererScopeName.detections);
const { addError } = useAppToasts();
// Table data state
const {
data: events,
dataUpdatedAt,
isFetching,
isLoading,
refetch,
} = useRuleExecutionEvents({
ruleId,
start,
end,
queryText,
statusFilters,
page: pageIndex,
perPage: pageSize,
sortField,
sortOrder: sortDirection,
});
const items = events?.events ?? [];
const maxEvents = events?.total ?? 0;
// Callbacks
const onTableChangeCallback = useCallback(({ page = {}, sort = {} }) => {
const { index, size } = page;
const { field, direction } = sort;
setPageIndex(index);
setPageSize(size);
setSortField(field);
setSortDirection(direction);
}, []);
const onTimeChangeCallback = useCallback(
(props: OnTimeChangeProps) => {
const recentlyUsedRange = recentlyUsedRanges.filter((range) => {
const isDuplicate = range.start === props.start && range.end === props.end;
return !isDuplicate;
});
recentlyUsedRange.unshift({ start: props.start, end: props.end });
setStart(props.start);
setEnd(props.end);
setRecentlyUsedRanges(
recentlyUsedRange.length > 10 ? recentlyUsedRange.slice(0, 9) : recentlyUsedRange
);
},
[recentlyUsedRanges]
);
const onRefreshChangeCallback = useCallback((props: OnRefreshChangeProps) => {
setIsPaused(props.isPaused);
setRefreshInterval(props.refreshInterval);
}, []);
const onRefreshCallback = useCallback(
(props: OnRefreshProps) => {
refetch();
},
[refetch]
);
const onSearchCallback = useCallback((updatedQueryText: string) => {
setQueryText(updatedQueryText);
}, []);
const onStatusFilterChangeCallback = useCallback((updatedStatusFilters: string[]) => {
setStatusFilters(updatedStatusFilters.sort().join(','));
}, []);
const onFilterByExecutionIdCallback = useCallback(
(executionId: string) => {
const field = indexPattern.fields.find((f) => f.name === EXECUTION_UUID_FIELD_NAME);
if (field != null) {
const filter = buildFilter(
indexPattern,
field,
FILTERS.PHRASE,
false,
false,
executionId,
null
);
filterManager.addFilters(filter);
selectAlertsTab();
} else {
addError(i18n.ACTIONS_FIELD_NOT_FOUND_ERROR, {
title: i18n.ACTIONS_FIELD_NOT_FOUND_ERROR_TITLE,
});
}
},
[addError, filterManager, indexPattern, selectAlertsTab]
);
const onShowMetricColumnsCallback = useCallback(
(showMetrics: boolean) => {
storage.set(RULE_DETAILS_EXECUTION_LOG_TABLE_SHOW_METRIC_COLUMNS_STORAGE_KEY, showMetrics);
setShowMetricColumns(showMetrics);
},
[storage]
);
// Memoized state
const pagination = useMemo(() => {
return {
pageIndex,
pageSize,
totalItemCount:
maxEvents > MAX_EXECUTION_EVENTS_DISPLAYED ? MAX_EXECUTION_EVENTS_DISPLAYED : maxEvents,
pageSizeOptions: [5, 10, 25, 50],
};
}, [maxEvents, pageIndex, pageSize]);
const sorting = useMemo(() => {
return {
sort: {
field: sortField,
direction: sortDirection,
},
};
}, [sortDirection, sortField]);
const actions = useMemo(
() => [
{
field: EXECUTION_UUID_FIELD_NAME,
name: i18n.COLUMN_ACTIONS,
width: '5%',
actions: [
{
name: 'Edit',
isPrimary: true,
field: '',
description: i18n.COLUMN_ACTIONS_TOOLTIP,
icon: 'filter',
type: 'icon',
onClick: (value: object) => {
const executionId = get(value, EXECUTION_UUID_FIELD_NAME);
if (executionId) {
onFilterByExecutionIdCallback(executionId);
}
},
'data-test-subj': 'action-filter-by-execution-id',
},
],
},
],
[onFilterByExecutionIdCallback]
);
const executionLogColumns = useMemo(
() =>
showMetricColumns
? [...EXECUTION_LOG_COLUMNS, ...GET_EXECUTION_LOG_METRICS_COLUMNS(docLinks), ...actions]
: [...EXECUTION_LOG_COLUMNS, ...actions],
[actions, docLinks, showMetricColumns]
);
return (
<>
<EuiFlexGroup gutterSize="s">
<EuiFlexItem grow={true}>
<ExecutionLogSearchBar
onSearch={onSearchCallback}
onStatusFilterChange={onStatusFilterChangeCallback}
onlyShowFilters={true}
/>
</EuiFlexItem>
<EuiFlexItem style={{ maxWidth: '582px' }}>
<EuiSuperDatePicker
start={start}
end={end}
onTimeChange={onTimeChangeCallback}
onRefresh={onRefreshCallback}
isPaused={isPaused}
isLoading={isFetching}
refreshInterval={refreshInterval}
onRefreshChange={onRefreshChangeCallback}
recentlyUsedRanges={recentlyUsedRanges}
width="full"
/>
</EuiFlexItem>
</EuiFlexGroup>
<EuiSpacer size="s" />
<UtilityBar>
<UtilityBarSection>
<UtilityBarGroup>
<UtilityBarText dataTestSubj="executionsShowing">
{i18n.SHOWING_EXECUTIONS(
maxEvents > MAX_EXECUTION_EVENTS_DISPLAYED
? MAX_EXECUTION_EVENTS_DISPLAYED
: maxEvents
)}
</UtilityBarText>
</UtilityBarGroup>
{maxEvents > MAX_EXECUTION_EVENTS_DISPLAYED && (
<UtilityBarGroup>
<UtilityBarText dataTestSubj="exceptionsShowing">
<EuiTextColor color="danger">
{i18n.RULE_EXECUTION_LOG_SEARCH_LIMIT_EXCEEDED(
maxEvents,
MAX_EXECUTION_EVENTS_DISPLAYED
)}
</EuiTextColor>
</UtilityBarText>
</UtilityBarGroup>
)}
</UtilityBarSection>
<UtilityBarSection>
<UtilityBarText dataTestSubj="executionsShowing">
{timelines.getLastUpdated({
showUpdating: isLoading || isFetching,
updatedAt: dataUpdatedAt,
})}
</UtilityBarText>
<UtilitySwitch
label={i18n.RULE_EXECUTION_LOG_SHOW_METRIC_COLUMNS_SWITCH}
checked={showMetricColumns}
compressed={true}
onChange={(e) => onShowMetricColumnsCallback(e.target.checked)}
/>
</UtilityBarSection>
</UtilityBar>
<EuiBasicTable
columns={executionLogColumns}
items={items}
loading={isFetching}
pagination={pagination}
sorting={sorting}
onChange={onTableChangeCallback}
/>
</>
);
};
export const ExecutionLogTable = React.memo(ExecutionLogTableComponent);
ExecutionLogTable.displayName = 'ExecutionLogTable';

21
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/rule_duration_format.test.tsx Normal file
View file

@ -0,0 +1,21 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { shallow } from 'enzyme';
import React from 'react';
import { RuleDurationFormat } from './rule_duration_format';
// TODO: Replace snapshot test with base test cases
describe('RuleDurationFormat', () => {
describe('snapshots', () => {
test('renders correctly against snapshot', () => {
const wrapper = shallow(<RuleDurationFormat duration={0} />);
expect(wrapper).toMatchSnapshot();
});
});
});

55
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/rule_duration_format.tsx Normal file
View file

@ -0,0 +1,55 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import numeral from '@elastic/numeral';
import moment from 'moment';
import React, { useMemo } from 'react';
interface Props {
duration: number;
isMillis?: boolean;
allowZero?: boolean;
}
export function getFormattedDuration(value: number) {
if (!value) {
return '00:00:00:000';
}
const duration = moment.duration(value);
const hours = Math.floor(duration.asHours()).toString().padStart(2, '0');
const minutes = Math.floor(duration.asMinutes()).toString().padStart(2, '0');
const seconds = duration.seconds().toString().padStart(2, '0');
const ms = duration.milliseconds().toString().padStart(3, '0');
return `${hours}:${minutes}:${seconds}:${ms}`;
}
export function getFormattedMilliseconds(value: number) {
const formatted = numeral(value).format('0,0');
return `${formatted} ms`;
}
/**
* Formats duration as (hh:mm:ss:SSS)
* @param props duration default as nanos, set isMillis:true to pass in ms
* @constructor
*/
const RuleDurationFormatComponent = (props: Props) => {
const { duration, isMillis = false, allowZero = true } = props;
const formattedDuration = useMemo(() => {
// Durations can be buggy and return negative
if (allowZero && duration >= 0) {
return getFormattedDuration(isMillis ? duration * 1000 : duration);
}
return 'N/A';
}, [allowZero, duration, isMillis]);
return <span data-test-subj="rule-duration-format-value">{formattedDuration}</span>;
};
export const RuleDurationFormat = React.memo(RuleDurationFormatComponent);
RuleDurationFormat.displayName = 'RuleDurationFormat';

182
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/execution_log_table/translations.ts Normal file
View file

@ -0,0 +1,182 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { i18n } from '@kbn/i18n';
export const SHOWING_EXECUTIONS = (totalItems: number) =>
i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.totalExecutionsLabel',
{
values: { totalItems },
defaultMessage:
'Showing {totalItems} {totalItems, plural, =1 {rule execution} other {rule executions}}',
}
);
export const RULE_EXECUTION_LOG_SEARCH_LIMIT_EXCEEDED = (totalItems: number, maxItems: number) =>
i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.searchLimitExceededLabel',
{
values: { totalItems, maxItems },
defaultMessage:
"More than {totalItems} rule executions match filters provided. Showing first {maxItems} by most recent '@timestamp'. Constrain filters further to view additional execution events.",
}
);
export const RULE_EXECUTION_LOG_SEARCH_PLACEHOLDER = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.searchPlaceholder',
{
defaultMessage: 'duration > 100 and gapDuration > 10',
}
);
export const RULE_EXECUTION_LOG_SHOW_METRIC_COLUMNS_SWITCH = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.showMetricColumnsSwitchTitle',
{
defaultMessage: 'Show metrics columns',
}
);
export const COLUMN_STATUS = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.statusColumn',
{
defaultMessage: 'Status',
}
);
export const COLUMN_STATUS_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.statusColumnTooltip',
{
defaultMessage: 'Overall status of execution.',
}
);
export const COLUMN_TIMESTAMP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.timestampColumn',
{
defaultMessage: 'Timestamp',
}
);
export const COLUMN_TIMESTAMP_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.timestampColumnTooltip',
{
defaultMessage: 'Datetime rule execution initiated.',
}
);
export const COLUMN_DURATION = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.durationColumn',
{
defaultMessage: 'Duration',
}
);
export const COLUMN_DURATION_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.durationColumnTooltip',
{
defaultMessage: 'The length of time it took for the rule to run (hh:mm:ss:SSS).',
}
);
export const COLUMN_MESSAGE = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.messageColumn',
{
defaultMessage: 'Message',
}
);
export const COLUMN_MESSAGE_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.messageColumnTooltip',
{
defaultMessage: 'Relevant message from execution outcome.',
}
);
export const COLUMN_GAP_DURATION = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.gapDurationColumn',
{
defaultMessage: 'Gap Duration',
}
);
export const COLUMN_GAP_TOOLTIP_SEE_DOCUMENTATION = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.gapTooltipSeeDocsDescription',
{
defaultMessage: 'see documentation',
}
);
export const COLUMN_INDEX_DURATION = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.indexDurationColumn',
{
defaultMessage: 'Index Duration',
}
);
export const COLUMN_INDEX_DURATION_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.indexDurationColumnTooltip',
{
defaultMessage: 'The length of time it took to index detected alerts (hh:mm:ss:SSS).',
}
);
export const COLUMN_SEARCH_DURATION = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.searchDurationColumn',
{
defaultMessage: 'Search Duration',
}
);
export const COLUMN_SEARCH_DURATION_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.searchDurationColumnTooltip',
{
defaultMessage: 'The length of time it took to search for alerts (hh:mm:ss:SSS).',
}
);
export const COLUMN_SCHEDULING_DELAY = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.schedulingDelayColumn',
{
defaultMessage: 'Scheduling Delay',
}
);
export const COLUMN_SCHEDULING_DELAY_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.schedulingDelayColumnTooltip',
{
defaultMessage: 'The length of time from rule scheduled till rule executed (hh:mm:ss:SSS).',
}
);
export const COLUMN_ACTIONS = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.actionsColumn',
{
defaultMessage: 'Actions',
}
);
export const COLUMN_ACTIONS_TOOLTIP = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.actionsColumnTooltip',
{
defaultMessage: 'Filter alerts by rule execution ID.',
}
);
export const ACTIONS_FIELD_NOT_FOUND_ERROR_TITLE = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.actionFieldNotFoundErrorTitle',
{
defaultMessage: 'Unable to filter alerts',
}
);
export const ACTIONS_FIELD_NOT_FOUND_ERROR = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLog.actionFieldNotFoundErrorDescription',
{
defaultMessage: "Cannot find field 'kibana.alert.rule.execution.uuid' in alerts index.",
}
);

80
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/failure_history.tsx
View file

@ -1,80 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import React from 'react';
import {
EuiBasicTable,
EuiPanel,
EuiLoadingContent,
EuiHealth,
EuiBasicTableColumn,
} from '@elastic/eui';
import { RuleExecutionEvent } from '../../../../../../common/detection_engine/schemas/common';
import { useRuleExecutionEvents } from '../../../../containers/detection_engine/rules';
import { HeaderSection } from '../../../../../common/components/header_section';
import * as i18n from './translations';
import { FormattedDate } from '../../../../../common/components/formatted_date';
const columns: Array<EuiBasicTableColumn<RuleExecutionEvent>> = [
{
name: i18n.COLUMN_STATUS_TYPE,
render: () => <EuiHealth color="danger">{i18n.TYPE_FAILED}</EuiHealth>,
truncateText: false,
width: '16%',
},
{
field: 'date',
name: i18n.COLUMN_FAILED_AT,
render: (value: string) => <FormattedDate value={value} fieldName="date" />,
sortable: false,
truncateText: false,
width: '24%',
},
{
field: 'message',
name: i18n.COLUMN_FAILED_MSG,
render: (value: string) => <>{value}</>,
sortable: false,
truncateText: false,
width: '60%',
},
];
interface FailureHistoryProps {
ruleId: string;
}
const FailureHistoryComponent: React.FC<FailureHistoryProps> = ({ ruleId }) => {
const events = useRuleExecutionEvents(ruleId);
const loading = events.isLoading;
const items = events.data ?? [];
if (loading) {
return (
<EuiPanel hasBorder>
<HeaderSection title={i18n.LAST_FIVE_ERRORS} />
<EuiLoadingContent />
</EuiPanel>
);
}
return (
<EuiPanel hasBorder>
<HeaderSection title={i18n.LAST_FIVE_ERRORS} />
<EuiBasicTable
columns={columns}
items={items}
loading={loading}
sorting={{ sort: { field: 'date', direction: 'desc' } }}
/>
</EuiPanel>
);
};
export const FailureHistory = React.memo(FailureHistoryComponent);
FailureHistory.displayName = 'FailureHistory';

22
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/index.tsx
View file

@ -104,10 +104,10 @@ import {
RuleStatusFailedCallOut,
ruleStatusI18n,
} from '../../../../components/rules/rule_execution_status';
import { FailureHistory } from './failure_history';
import * as detectionI18n from '../../translations';
import * as ruleI18n from '../translations';
import { ExecutionLogTable } from './execution_log_table/execution_log_table';
import * as i18n from './translations';
import { NeedAdminForUpdateRulesCallOut } from '../../../../components/callouts/need_admin_for_update_callout';
import { MissingPrivilegesCallOut } from '../../../../components/callouts/missing_privileges_callout';
@ -133,7 +133,7 @@ const StyledFullHeightContainer = styled.div`
enum RuleDetailTabs {
alerts = 'alerts',
failures = 'failures',
executionLogs = 'executionLogs',
exceptions = 'exceptions',
}
@ -151,10 +151,10 @@ const ruleDetailTabs = [
dataTestSubj: 'exceptionsTab',
},
{
id: RuleDetailTabs.failures,
name: i18n.FAILURE_HISTORY_TAB,
id: RuleDetailTabs.executionLogs,
name: i18n.RULE_EXECUTION_LOGS,
disabled: false,
dataTestSubj: 'failureHistoryTab',
dataTestSubj: 'executionLogsTab',
},
];
@ -431,7 +431,7 @@ const RuleDetailsPageComponent: React.FC<DetectionEngineComponentProps> = ({
<EuiTab
onClick={() => setRuleDetailTab(tab.id)}
isSelected={tab.id === ruleDetailTab}
disabled={tab.disabled}
disabled={tab.disabled || (tab.id === RuleDetailTabs.executionLogs && !isExistingRule)}
key={tab.id}
data-test-subj={tab.dataTestSubj}
>
@ -440,7 +440,7 @@ const RuleDetailsPageComponent: React.FC<DetectionEngineComponentProps> = ({
))}
</EuiTabs>
),
[ruleDetailTab, setRuleDetailTab, pageTabs]
[isExistingRule, ruleDetailTab, setRuleDetailTab, pageTabs]
);
const ruleIndices = useMemo(() => rule?.index ?? DEFAULT_INDEX_PATTERN, [rule?.index]);
@ -605,6 +605,10 @@ const RuleDetailsPageComponent: React.FC<DetectionEngineComponentProps> = ({
[containerElement, onSkipFocusBeforeEventsTable, onSkipFocusAfterEventsTable]
);
const selectAlertsTabCallback = useCallback(() => {
setRuleDetailTab(RuleDetailTabs.alerts);
}, []);
if (
redirectToDetections(
isSignalIndexExists,
@ -806,7 +810,9 @@ const RuleDetailsPageComponent: React.FC<DetectionEngineComponentProps> = ({
onRuleChange={refreshRule}
/>
)}
{ruleDetailTab === RuleDetailTabs.failures && <FailureHistory ruleId={ruleId} />}
{ruleDetailTab === RuleDetailTabs.executionLogs && (
<ExecutionLogTable ruleId={ruleId} selectAlertsTab={selectAlertsTabCallback} />
)}
</SecuritySolutionPageWrapper>
</StyledFullHeightContainer>

34
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/details/translations.ts
View file

@ -42,38 +42,10 @@ export const UNKNOWN = i18n.translate(
}
);
export const FAILURE_HISTORY_TAB = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.failureHistoryTab',
export const RULE_EXECUTION_LOGS = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.ruleExecutionLogsTab',
{
defaultMessage: 'Failure History',
}
);
export const LAST_FIVE_ERRORS = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.lastFiveErrorsTitle',
{
defaultMessage: 'Last five errors',
}
);
export const COLUMN_STATUS_TYPE = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.statusTypeColumn',
{
defaultMessage: 'Type',
}
);
export const COLUMN_FAILED_AT = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.statusFailedAtColumn',
{
defaultMessage: 'Failed at',
}
);
export const COLUMN_FAILED_MSG = i18n.translate(
'xpack.securitySolution.detectionEngine.ruleDetails.statusFailedMsgColumn',
{
defaultMessage: 'Failed message',
defaultMessage: 'Rule execution logs ',
}
);

7
x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules/translations.ts
View file

@ -584,6 +584,13 @@ export const COLUMN_GAP = i18n.translate(
}
);
export const COLUMN_GAP_TOOLTIP_SEE_DOCUMENTATION = i18n.translate(
'xpack.securitySolution.detectionEngine.rules.allRules.columns.gapTooltipSeeDocsDescription',
{
defaultMessage: 'see documentation',
}
);
export const RULES_TAB = i18n.translate(
'xpack.securitySolution.detectionEngine.rules.allRules.tabs.rules',
{

60
x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_responses.ts
View file

@ -25,6 +25,7 @@ import {
DETECTION_ENGINE_RULES_BULK_ACTION,
DETECTION_ENGINE_RULE_EXECUTION_EVENTS_URL,
} from '../../../../../common/constants';
import { GetAggregateRuleExecutionEventsResponse } from '../../../../../common/detection_engine/schemas/response';
import { RuleAlertType, HapiReadableStream } from '../../rules/types';
import { requestMock } from './request';
import { QuerySignalsSchemaDecoded } from '../../../../../common/detection_engine/schemas/request/query_signals_index_schema';
@ -242,6 +243,12 @@ export const getRuleExecutionEventsRequest = () =>
params: {
ruleId: '04128c15-0d1b-4716-a4c5-46997ac7f3bd',
},
query: {
start: 'now-30',
end: 'now',
query_text: '',
status_filters: '',
},
});
export const getImportRulesRequest = (hapiStream?: HapiReadableStream) =>
@ -552,6 +559,59 @@ export const getLastFailures = (): RuleExecutionEvent[] => [
},
];
export const getAggregateExecutionEvents = (): GetAggregateRuleExecutionEventsResponse => ({
events: [
{
execution_uuid: '34bab6e0-89b6-4d10-9cbb-cda76d362db6',
timestamp: '2022-03-11T22:04:05.931Z',
duration_ms: 1975,
status: 'success',
message:
"rule executed: siem.queryRule:f78f3550-a186-11ec-89a1-0bce95157aba: 'This Rule Makes Alerts, Actions, AND Moar!'",
num_active_alerts: 0,
num_new_alerts: 0,
num_recovered_alerts: 0,
num_triggered_actions: 0,
num_succeeded_actions: 0,
num_errored_actions: 0,
total_search_duration_ms: 0,
es_search_duration_ms: 538,
schedule_delay_ms: 2091,
timed_out: false,
indexing_duration_ms: 7,
search_duration_ms: 551,
gap_duration_ms: 0,
security_status: 'succeeded',
security_message: 'succeeded',
},
{
execution_uuid: '254d8400-9dc7-43c5-ad4b-227273d1a44b',
timestamp: '2022-03-11T22:02:41.923Z',
duration_ms: 11916,
status: 'success',
message:
"rule executed: siem.queryRule:f78f3550-a186-11ec-89a1-0bce95157aba: 'This Rule Makes Alerts, Actions, AND Moar!'",
num_active_alerts: 0,
num_new_alerts: 0,
num_recovered_alerts: 0,
num_triggered_actions: 1,
num_succeeded_actions: 1,
num_errored_actions: 0,
total_search_duration_ms: 0,
es_search_duration_ms: 1406,
schedule_delay_ms: 1583,
timed_out: false,
indexing_duration_ms: 0,
search_duration_ms: 0,
gap_duration_ms: 0,
security_status: 'partial failure',
security_message:
'Check privileges failed to execute ResponseError: index_not_found_exception: [index_not_found_exception] Reason: no such index [broken-index] name: "This Rule Makes Alerts, Actions, AND Moar!" id: "f78f3550-a186-11ec-89a1-0bce95157aba" rule id: "b64b4540-d035-4826-a1e7-f505bf4b9653" execution id: "254d8400-9dc7-43c5-ad4b-227273d1a44b" space ID: "default"',
},
],
total: 2,
});
export const getBasicEmptySearchResponse = (): estypes.SearchResponse<unknown> => ({
took: 1,
timed_out: false,

17
x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/get_rule_execution_events_route.test.ts
View file

@ -6,9 +6,14 @@
*/
import { serverMock, requestContextMock } from '../__mocks__';
import { getRuleExecutionEventsRequest, getLastFailures } from '../__mocks__/request_responses';
import {
getRuleExecutionEventsRequest,
getAggregateExecutionEvents,
} from '../__mocks__/request_responses';
import { getRuleExecutionEventsRoute } from './get_rule_execution_events_route';
// TODO: Add additional tests for param validation
describe('getRuleExecutionEventsRoute', () => {
let server: ReturnType<typeof serverMock.create>;
let { clients, context } = requestContextMock.createTools();
@ -22,21 +27,19 @@ describe('getRuleExecutionEventsRoute', () => {
describe('when it finds events in rule execution log', () => {
it('returns 200 response with the events', async () => {
const lastFailures = getLastFailures();
clients.ruleExecutionLog.getLastFailures.mockResolvedValue(lastFailures);
const executionEvents = getAggregateExecutionEvents();
clients.ruleExecutionLog.getAggregateExecutionEvents.mockResolvedValue(executionEvents);
const response = await server.inject(getRuleExecutionEventsRequest(), context);
expect(response.status).toEqual(200);
expect(response.body).toEqual({
events: lastFailures,
});
expect(response.body).toEqual(executionEvents);
});
});
describe('when rule execution log client throws an error', () => {
it('returns 500 response with it', async () => {
clients.ruleExecutionLog.getLastFailures.mockRejectedValue(new Error('Boom!'));
clients.ruleExecutionLog.getAggregateExecutionEvents.mockRejectedValue(new Error('Boom!'));
const response = await server.inject(getRuleExecutionEventsRequest(), context);

42
x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/get_rule_execution_events_route.ts
View file

@ -6,20 +6,20 @@
*/
import { transformError } from '@kbn/securitysolution-es-utils';
import { GetAggregateRuleExecutionEventsResponse } from '../../../../../common/detection_engine/schemas/response';
import { buildRouteValidation } from '../../../../utils/build_validation/route_validation';
import { buildSiemResponse } from '../utils';
import type { SecuritySolutionPluginRouter } from '../../../../types';
import { DETECTION_ENGINE_RULE_EXECUTION_EVENTS_URL } from '../../../../../common/constants';
import { GetRuleExecutionEventsRequestParams } from '../../../../../common/detection_engine/schemas/request/get_rule_execution_events_request';
import { GetRuleExecutionEventsResponse } from '../../../../../common/detection_engine/schemas/response/get_rule_execution_events_response';
import {
GetRuleExecutionEventsQueryParams,
GetRuleExecutionEventsRequestParams,
} from '../../../../../common/detection_engine/schemas/request/get_rule_execution_events_schema';
/**
* Returns execution events of a given rule (e.g. status changes) from Event Log.
* Accepts rule's saved object ID (`rule.id`).
*
* NOTE: This endpoint is under construction. It will be extended and finalized.
* https://github.com/elastic/kibana/issues/119598
* Returns execution events of a given rule (aggregated by executionId) from Event Log.
* Accepts rule's saved object ID (`rule.id`), `start`, `end` and `filters` query params.
*/
export const getRuleExecutionEventsRoute = (router: SecuritySolutionPluginRouter) => {
router.get(
@ -27,6 +27,7 @@ export const getRuleExecutionEventsRoute = (router: SecuritySolutionPluginRouter
path: DETECTION_ENGINE_RULE_EXECUTION_EVENTS_URL,
validate: {
params: buildRouteValidation(GetRuleExecutionEventsRequestParams),
query: buildRouteValidation(GetRuleExecutionEventsQueryParams),
},
options: {
tags: ['access:securitySolution'],
@ -34,14 +35,35 @@ export const getRuleExecutionEventsRoute = (router: SecuritySolutionPluginRouter
},
async (context, request, response) => {
const { ruleId } = request.params;
const {
start,
end,
query_text: queryText = '',
status_filters: statusFilters = '',
page,
per_page: perPage,
sort_field: sortField = 'timestamp',
sort_order: sortOrder = 'desc',
} = request.query;
const siemResponse = buildSiemResponse(response);
try {
const executionLog = context.securitySolution.getRuleExecutionLog();
const executionEvents = await executionLog.getLastFailures(ruleId);
const { events, total } = await executionLog.getAggregateExecutionEvents({
ruleId,
start,
end,
queryText,
statusFilters: statusFilters.length ? statusFilters.split(',') : [],
page: page != null ? parseInt(page, 10) : 0,
perPage: perPage != null ? parseInt(perPage, 10) : 10,
sortField,
sortOrder,
});
const responseBody: GetRuleExecutionEventsResponse = {
events: executionEvents,
const responseBody: GetAggregateRuleExecutionEventsResponse = {
events,
total,
};
return response.ok({ body: responseBody });

1
x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/__mocks__/index.ts
View file

@ -13,6 +13,7 @@ import {
const ruleExecutionLogForRoutesMock = {
create: (): jest.Mocked<IRuleExecutionLogForRoutes> => ({
getAggregateExecutionEvents: jest.fn(),
getExecutionSummariesBulk: jest.fn(),
getExecutionSummary: jest.fn(),
clearExecutionSummary: jest.fn(),

44
x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/client_for_routes/client.ts
View file

@ -7,6 +7,7 @@
import { chunk, mapValues } from 'lodash';
import { Logger } from 'src/core/server';
import { GetAggregateRuleExecutionEventsResponse } from '../../../../../common/detection_engine/schemas/response';
import { initPromisePool } from '../../../../utils/promise_pool';
import { withSecuritySpan } from '../../../../utils/with_security_span';
@ -14,7 +15,7 @@ import { RuleExecutionStatus } from '../../../../../common/detection_engine/sche
import { IEventLogReader } from '../event_log/event_log_reader';
import { IRuleExecutionSavedObjectsClient } from '../execution_saved_object/saved_objects_client';
import { IRuleExecutionLogForRoutes } from './client_interface';
import { GetAggregateExecutionEventsArgs, IRuleExecutionLogForRoutes } from './client_interface';
import { ExtMeta } from '../utils/console_logging';
import { truncateList } from '../utils/normalization';
@ -28,6 +29,47 @@ export const createClientForRoutes = (
logger: Logger
): IRuleExecutionLogForRoutes => {
return {
getAggregateExecutionEvents({
ruleId,
start,
end,
queryText,
statusFilters,
page,
perPage,
sortField,
sortOrder,
}: GetAggregateExecutionEventsArgs): Promise<GetAggregateRuleExecutionEventsResponse> {
return withSecuritySpan(
'IRuleExecutionLogForRoutes.getAggregateExecutionEvents',
async () => {
try {
return await eventLog.getAggregateExecutionEvents({
ruleId,
start,
end,
queryText,
statusFilters,
page,
perPage,
sortField,
sortOrder,
});
} catch (e) {
const logMessage =
'Error getting last aggregation of execution failures from event log';
const logAttributes = `rule id: "${ruleId}"`;
const logReason = e instanceof Error ? e.message : String(e);
const logMeta: ExtMeta = {
rule: { id: ruleId },
};
logger.error<ExtMeta>(`${logMessage}; ${logAttributes}; ${logReason}`, logMeta);
throw e;
}
}
);
},
/**
* Get the current rule execution summary for each of the given rule IDs.
* This method splits work into chunks so not to overwhelm Elasticsearch

38
x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/client_for_routes/client_interface.ts
View file

@ -9,6 +9,19 @@ import {
RuleExecutionEvent,
RuleExecutionSummary,
} from '../../../../../common/detection_engine/schemas/common';
import { GetAggregateRuleExecutionEventsResponse } from '../../../../../common/detection_engine/schemas/response';
export interface GetAggregateExecutionEventsArgs {
ruleId: string;
start: string;
end: string;
queryText: string;
statusFilters: string[];
page: number;
perPage: number;
sortField: string;
sortOrder: string;
}
/**
* Used from route handlers to fetch and manage various information about the rule execution:
@ -16,6 +29,31 @@ import {
* - execution events such as recent failures and status changes
*/
export interface IRuleExecutionLogForRoutes {
/**
* Fetches list of execution events aggregated by executionId, combining data from both alerting
* and security-solution event-log documents
* @param ruleId Saved object id of the rule (`rule.id`).
* @param start start of daterange to filter to
* @param end end of daterange to filter to
* @param queryText string of field-based filters, e.g. kibana.alert.rule.execution.status:*
* @param statusFilters array of status filters, e.g. ['succeeded', 'going to run']
* @param page current page to fetch
* @param perPage number of results to fetch per page
* @param sortField field to sort by
* @param sortOrder what order to sort by (e.g. `asc` or `desc`)
*/
getAggregateExecutionEvents({
ruleId,
start,
end,
queryText,
statusFilters,
page,
perPage,
sortField,
sortOrder,
}: GetAggregateExecutionEventsArgs): Promise<GetAggregateRuleExecutionEventsResponse>;
/**
* Fetches a list of current execution summaries of multiple rules.
* @param ruleIds A list of saved object ids of multiple rules (`rule.id`).

82
x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/event_log_reader.ts
View file

@ -5,12 +5,15 @@
* 2.0.
*/
import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { MAX_EXECUTION_EVENTS_DISPLAYED } from '@kbn/securitysolution-rules';
import { IEventLogClient } from '../../../../../../event_log/server';
import {
RuleExecutionEvent,
RuleExecutionStatus,
} from '../../../../../common/detection_engine/schemas/common';
import { GetAggregateRuleExecutionEventsResponse } from '../../../../../common/detection_engine/schemas/response';
import { invariant } from '../../../../../common/utils/invariant';
import { withSecuritySpan } from '../../../../utils/with_security_span';
import {
@ -18,11 +21,32 @@ import {
RULE_EXECUTION_LOG_PROVIDER,
RuleExecutionLogAction,
} from './constants';
import {
formatExecutionEventResponse,
getExecutionEventAggregation,
} from './get_execution_event_aggregation';
import { ExecutionUuidAggResult } from './get_execution_event_aggregation/types';
export interface IEventLogReader {
getAggregateExecutionEvents(
args: GetAggregateExecutionEventsArgs
): Promise<GetAggregateRuleExecutionEventsResponse>;
getLastStatusChanges(args: GetLastStatusChangesArgs): Promise<RuleExecutionEvent[]>;
}
export interface GetAggregateExecutionEventsArgs {
ruleId: string;
start: string;
end: string;
queryText: string;
statusFilters: string[];
page: number;
perPage: number;
sortField: string;
sortOrder: string;
}
export interface GetLastStatusChangesArgs {
ruleId: string;
count: number;
@ -31,6 +55,64 @@ export interface GetLastStatusChangesArgs {
export const createEventLogReader = (eventLog: IEventLogClient): IEventLogReader => {
return {
async getAggregateExecutionEvents(
args: GetAggregateExecutionEventsArgs
): Promise<GetAggregateRuleExecutionEventsResponse> {
const { ruleId, start, end, statusFilters, page, perPage, sortField, sortOrder } = args;
const soType = RULE_SAVED_OBJECT_TYPE;
const soIds = [ruleId];
// Current workaround to support root level filters without missing fields in the aggregate event
// or including events from statuses that aren't selected
// TODO: See: https://github.com/elastic/kibana/pull/127339/files#r825240516
// First fetch execution uuid's by status filter if provided
let statusIds: string[] = [];
// If 0 or 3 statuses are selected we can search for all statuses and don't need this pre-filter by ID
if (statusFilters.length > 0 && statusFilters.length < 3) {
// TODO: Add cardinality agg and pass as maxEvents in response
const statusResults = await eventLog.aggregateEventsBySavedObjectIds(soType, soIds, {
start,
end,
filter: `kibana.alert.rule.execution.status:(${statusFilters.join(' OR ')})`,
aggs: {
filteredExecutionUUIDs: {
terms: {
field: 'kibana.alert.rule.execution.uuid',
size: MAX_EXECUTION_EVENTS_DISPLAYED,
},
},
},
});
const filteredExecutionUUIDs = statusResults.aggregations
?.filteredExecutionUUIDs as ExecutionUuidAggResult;
statusIds = filteredExecutionUUIDs?.buckets?.map((b) => b.key) ?? [];
// Early return if no results based on status filter
if (statusIds.length === 0) {
return {
total: 0,
events: [],
};
}
}
// Now query for aggregate events, and pass any ID's as filters as determined from the above status/queryText results
const idsFilter = statusIds.length
? `kibana.alert.rule.execution.uuid:(${statusIds.join(' OR ')})`
: '';
const results = await eventLog.aggregateEventsBySavedObjectIds(soType, soIds, {
start,
end,
filter: idsFilter,
aggs: getExecutionEventAggregation({
maxExecutions: MAX_EXECUTION_EVENTS_DISPLAYED,
page,
perPage,
sort: [{ [sortField]: { order: sortOrder } }] as estypes.Sort,
}),
});
return formatExecutionEventResponse(results);
},
async getLastStatusChanges(args) {
const soType = RULE_SAVED_OBJECT_TYPE;
const soIds = [args.ruleId];

1321
x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.test.ts Normal file
View file

File diff suppressed because it is too large Load diff

351
x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/index.ts Normal file
View file

@ -0,0 +1,351 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import { BadRequestError } from '@kbn/securitysolution-es-utils';
import { flatMap, get } from 'lodash';
import { MAX_EXECUTION_EVENTS_DISPLAYED } from '@kbn/securitysolution-rules';
import { AggregateEventsBySavedObjectResult } from '../../../../../../../event_log/server';
import { AggregateRuleExecutionEvent } from '../../../../../../common/detection_engine/schemas/common';
import { GetAggregateRuleExecutionEventsResponse } from '../../../../../../common/detection_engine/schemas/response';
import {
ExecutionEventAggregationOptions,
ExecutionUuidAggResult,
ExecutionUuidAggBucket,
} from './types';
// Base ECS fields
const ACTION_FIELD = 'event.action';
const DURATION_FIELD = 'event.duration';
const MESSAGE_FIELD = 'message';
const PROVIDER_FIELD = 'event.provider';
const OUTCOME_FIELD = 'event.outcome';
const START_FIELD = 'event.start';
const TIMESTAMP_FIELD = '@timestamp';
// Platform fields
const SCHEDULE_DELAY_FIELD = 'kibana.task.schedule_delay';
const ES_SEARCH_DURATION_FIELD = 'kibana.alert.rule.execution.metrics.es_search_duration_ms';
const TOTAL_ACTIONS_TRIGGERED_FIELD =
'kibana.alert.rule.execution.metrics.number_of_triggered_actions';
const EXECUTION_UUID_FIELD = 'kibana.alert.rule.execution.uuid';
// TODO: To be added in https://github.com/elastic/kibana/pull/126210
// const TOTAL_ALERTS_CREATED: 'kibana.alert.rule.execution.metrics.total_alerts_created',
// const TOTAL_ALERTS_DETECTED: 'kibana.alert.rule.execution.metrics.total_alerts_detected',
// Security fields
const GAP_DURATION_FIELD = 'kibana.alert.rule.execution.metrics.execution_gap_duration_s';
const INDEXING_DURATION_FIELD = 'kibana.alert.rule.execution.metrics.total_indexing_duration_ms';
const SEARCH_DURATION_FIELD = 'kibana.alert.rule.execution.metrics.total_search_duration_ms';
const STATUS_FIELD = 'kibana.alert.rule.execution.status';
const ONE_MILLISECOND_AS_NANOSECONDS = 1_000_000;
const SORT_FIELD_TO_AGG_MAPPING: Record<string, string> = {
timestamp: 'ruleExecution>executeStartTime',
duration_ms: 'ruleExecution>executionDuration',
indexing_duration_ms: 'securityMetrics>indexDuration',
search_duration_ms: 'securityMetrics>searchDuration',
gap_duration_ms: 'securityMetrics>gapDuration',
schedule_delay_ms: 'ruleExecution>scheduleDelay',
num_triggered_actions: 'ruleExecution>numTriggeredActions',
// TODO: To be added in https://github.com/elastic/kibana/pull/126210
// total_alerts_created: 'securityMetrics>totalAlertsDetected',
// total_alerts_detected: 'securityMetrics>totalAlertsCreated',
};
/**
* Returns `aggs` to be supplied to aggregateEventsBySavedObjectIds
* @param maxExecutions upper bounds of execution events to return (to narrow below max terms agg limit)
* @param page current page to retrieve, starting at 0
* @param perPage number of execution events to display per page
* @param sort field to sort on
*/
export const getExecutionEventAggregation = ({
maxExecutions,
page,
perPage,
sort,
}: ExecutionEventAggregationOptions): Record<string, estypes.AggregationsAggregationContainer> => {
// Last stop validation for any other consumers so there's a friendly message instead of failed ES Query
if (maxExecutions > MAX_EXECUTION_EVENTS_DISPLAYED) {
throw new BadRequestError(
`Invalid maxExecutions requested "${maxExecutions}" - must be less than ${MAX_EXECUTION_EVENTS_DISPLAYED}`
);
}
if (page < 0) {
throw new BadRequestError(`Invalid page field "${page}" - must be greater than 0`);
}
if (perPage <= 0) {
throw new BadRequestError(`Invalid perPage field "${perPage}" - must be greater than 0`);
}
const sortFields = flatMap(sort as estypes.SortCombinations[], (s) => Object.keys(s));
for (const field of sortFields) {
if (!Object.keys(SORT_FIELD_TO_AGG_MAPPING).includes(field)) {
throw new BadRequestError(
`Invalid sort field "${field}" - must be one of [${Object.keys(
SORT_FIELD_TO_AGG_MAPPING
).join(',')}]`
);
}
}
return {
// Total unique executions for given root filters
totalExecutions: {
cardinality: {
field: EXECUTION_UUID_FIELD,
},
},
executionUuid: {
// Bucket by execution UUID
terms: {
field: EXECUTION_UUID_FIELD,
size: maxExecutions,
order: formatSortForTermsSort(sort),
},
aggs: {
// Bucket sort for paging
executionUuidSorted: {
bucket_sort: {
sort: formatSortForBucketSort(sort),
from: page * perPage,
size: perPage,
gap_policy: 'insert_zeros',
},
},
// Filter by action execute doc to retrieve action outcomes (successful/failed)
actionExecution: {
filter: getProviderAndActionFilter('actions', 'execute'),
aggs: {
actionOutcomes: {
terms: {
field: OUTCOME_FIELD,
size: 2,
},
},
},
},
// Filter by alerting execute doc to retrieve platform metrics
ruleExecution: {
filter: getProviderAndActionFilter('alerting', 'execute'),
aggs: {
executeStartTime: {
min: {
field: START_FIELD,
},
},
scheduleDelay: {
max: {
field: SCHEDULE_DELAY_FIELD,
},
},
esSearchDuration: {
max: {
field: ES_SEARCH_DURATION_FIELD,
},
},
numTriggeredActions: {
max: {
field: TOTAL_ACTIONS_TRIGGERED_FIELD,
},
},
executionDuration: {
max: {
field: DURATION_FIELD,
},
},
outcomeAndMessage: {
top_hits: {
size: 1,
_source: {
includes: [OUTCOME_FIELD, MESSAGE_FIELD],
},
},
},
},
},
// Filter by securitySolution status-change doc to retrieve security metrics
securityMetrics: {
filter: getProviderAndActionFilter('securitySolution.ruleExecution', 'execution-metrics'),
aggs: {
gapDuration: {
min: {
field: GAP_DURATION_FIELD,
missing: 0, // Necessary for sorting since field isn't written if no gap
},
},
indexDuration: {
min: {
field: INDEXING_DURATION_FIELD,
},
},
searchDuration: {
min: {
field: SEARCH_DURATION_FIELD,
},
},
},
},
// Filter by securitySolution ruleExecution doc to retrieve status and message
securityStatus: {
filter: getProviderAndActionFilter('securitySolution.ruleExecution', 'status-change'),
aggs: {
status: {
top_hits: {
sort: {
[TIMESTAMP_FIELD]: {
order: 'desc',
},
},
size: 1,
_source: {
includes: STATUS_FIELD,
},
},
},
message: {
top_hits: {
size: 1,
sort: {
[TIMESTAMP_FIELD]: {
order: 'desc',
},
},
_source: {
includes: MESSAGE_FIELD,
},
},
},
},
},
// If there was a timeout, this filter will return non-zero doc count
timeoutMessage: {
filter: getProviderAndActionFilter('alerting', 'execute-timeout'),
},
},
},
};
};
/**
* Returns bool filter for matching a specific provider AND action combination
* @param provider provider to match
* @param action action to match
*/
export const getProviderAndActionFilter = (provider: string, action: string) => {
return {
bool: {
must: [
{
match: {
[ACTION_FIELD]: action,
},
},
{
match: {
[PROVIDER_FIELD]: provider,
},
},
],
},
};
};
/**
* Formats aggregate execution event from bucket response
* @param bucket
*/
export const formatAggExecutionEventFromBucket = (
bucket: ExecutionUuidAggBucket
): AggregateRuleExecutionEvent => {
const durationUs = bucket?.ruleExecution?.executionDuration?.value ?? 0;
const scheduleDelayUs = bucket?.ruleExecution?.scheduleDelay?.value ?? 0;
const timedOut = (bucket?.timeoutMessage?.doc_count ?? 0) > 0;
const actionOutcomes = bucket?.actionExecution?.actionOutcomes?.buckets ?? [];
const actionExecutionSuccess = actionOutcomes.find((b) => b?.key === 'success')?.doc_count ?? 0;
const actionExecutionError = actionOutcomes.find((b) => b?.key === 'failure')?.doc_count ?? 0;
return {
execution_uuid: bucket?.key ?? '',
timestamp: bucket?.ruleExecution?.executeStartTime.value_as_string ?? '',
duration_ms: durationUs / ONE_MILLISECOND_AS_NANOSECONDS,
status: bucket?.ruleExecution?.outcomeAndMessage?.hits?.hits[0]?._source?.event?.outcome,
message: bucket?.ruleExecution?.outcomeAndMessage?.hits?.hits[0]?._source?.message,
num_active_alerts: bucket?.alertCounts?.buckets?.activeAlerts?.doc_count ?? 0,
num_new_alerts: bucket?.alertCounts?.buckets?.newAlerts?.doc_count ?? 0,
num_recovered_alerts: bucket?.alertCounts?.buckets?.recoveredAlerts?.doc_count ?? 0,
num_triggered_actions: bucket?.ruleExecution?.numTriggeredActions?.value ?? 0,
num_succeeded_actions: actionExecutionSuccess,
num_errored_actions: actionExecutionError,
total_search_duration_ms: bucket?.ruleExecution?.totalSearchDuration?.value ?? 0,
es_search_duration_ms: bucket?.ruleExecution?.esSearchDuration?.value ?? 0,
schedule_delay_ms: scheduleDelayUs / ONE_MILLISECOND_AS_NANOSECONDS,
timed_out: timedOut,
// security fields
indexing_duration_ms: bucket?.securityMetrics?.indexDuration?.value ?? 0,
search_duration_ms: bucket?.securityMetrics?.searchDuration?.value ?? 0,
gap_duration_ms: bucket?.securityMetrics?.gapDuration?.value ?? 0,
security_status:
bucket?.securityStatus?.status?.hits?.hits[0]?._source?.kibana?.alert?.rule?.execution
?.status,
security_message: bucket?.securityStatus?.message?.hits?.hits[0]?._source?.message,
};
};
/**
* Formats getAggregateExecutionEvents response from Elasticsearch response
* @param results Elasticsearch response
*/
export const formatExecutionEventResponse = (
results: AggregateEventsBySavedObjectResult
): GetAggregateRuleExecutionEventsResponse => {
const { aggregations } = results;
if (!aggregations) {
return {
total: 0,
events: [],
};
}
const total = (aggregations.totalExecutions as estypes.AggregationsCardinalityAggregate).value;
const buckets = (aggregations.executionUuid as ExecutionUuidAggResult).buckets;
return {
total,
events: buckets.map((b: ExecutionUuidAggBucket) => formatAggExecutionEventFromBucket(b)),
};
};
/**
* Formats sort field into bucket_sort agg format
* @param sort
*/
export const formatSortForBucketSort = (sort: estypes.Sort) => {
return (sort as estypes.SortCombinations[]).map((s) =>
Object.keys(s).reduce(
(acc, curr) => ({ ...acc, [SORT_FIELD_TO_AGG_MAPPING[curr]]: get(s, curr) }),
{}
)
);
};
/**
* Formats sort field into terms agg format
* @param sort
*/
export const formatSortForTermsSort = (sort: estypes.Sort) => {
return (sort as estypes.SortCombinations[]).map((s) =>
Object.keys(s).reduce(
(acc, curr) => ({ ...acc, [SORT_FIELD_TO_AGG_MAPPING[curr]]: get(s, `${curr}.order`) }),
{}
)
);
};

61
x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/event_log/get_execution_event_aggregation/types.ts Normal file
View file

@ -0,0 +1,61 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
type AlertCounts = estypes.AggregationsMultiBucketAggregateBase & {
buckets: {
activeAlerts: estypes.AggregationsSingleBucketAggregateBase;
newAlerts: estypes.AggregationsSingleBucketAggregateBase;
recoveredAlerts: estypes.AggregationsSingleBucketAggregateBase;
};
};
type ActionExecution = estypes.AggregationsTermsAggregateBase<{
key: string;
doc_count: number;
}> & {
buckets: Array<{ key: string; doc_count: number }>;
};
export type ExecutionUuidAggBucket = estypes.AggregationsStringTermsBucketKeys & {
timeoutMessage: estypes.AggregationsMultiBucketBase;
ruleExecution: {
executeStartTime: estypes.AggregationsMinAggregate;
executionDuration: estypes.AggregationsMaxAggregate;
scheduleDelay: estypes.AggregationsMaxAggregate;
esSearchDuration: estypes.AggregationsMaxAggregate;
totalSearchDuration: estypes.AggregationsMaxAggregate;
numTriggeredActions: estypes.AggregationsMaxAggregate;
outcomeAndMessage: estypes.AggregationsTopHitsAggregate;
};
alertCounts: AlertCounts;
actionExecution: {
actionOutcomes: ActionExecution;
};
securityStatus: {
message: estypes.AggregationsTopHitsAggregate;
status: estypes.AggregationsTopHitsAggregate;
};
securityMetrics: {
searchDuration: estypes.AggregationsMinAggregate;
indexDuration: estypes.AggregationsMinAggregate;
gapDuration: estypes.AggregationsMinAggregate;
};
};
export type ExecutionUuidAggResult<TBucket = ExecutionUuidAggBucket> =
estypes.AggregationsAggregateBase & {
buckets: TBucket[];
};
export interface ExecutionEventAggregationOptions {
maxExecutions: number;
page: number;
perPage: number;
sort: estypes.Sort;
}

5
x-pack/plugins/translations/translations/fr-FR.json
View file

@ -20775,15 +20775,10 @@
"xpack.securitySolution.detectionEngine.ruleDetails.deletedRule": "Règle supprimée",
"xpack.securitySolution.detectionEngine.ruleDetails.exceptionsTab": "Exceptions",
"xpack.securitySolution.detectionEngine.ruleDetails.experimentalDescription": "Expérimental",
"xpack.securitySolution.detectionEngine.ruleDetails.failureHistoryTab": "Historique des échecs",
"xpack.securitySolution.detectionEngine.ruleDetails.lastFiveErrorsTitle": "Cinq dernières erreurs",
"xpack.securitySolution.detectionEngine.ruleDetails.pageTitle": "Détails de la règle",
"xpack.securitySolution.detectionEngine.ruleDetails.ruleCreationDescription": "Créé par : {by} le {date}",
"xpack.securitySolution.detectionEngine.ruleDetails.ruleUpdateDescription": "Mis à jour par : {by} le {date}",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedAtColumn": "Échoué à",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedDescription": "Échoué",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedMsgColumn": "Message échoué",
"xpack.securitySolution.detectionEngine.ruleDetails.statusTypeColumn": "Type",
"xpack.securitySolution.detectionEngine.ruleDetails.unknownDescription": "Inconnu",
"xpack.securitySolution.detectionEngine.rules.aboutRuleTitle": "À propos de la règle",
"xpack.securitySolution.detectionEngine.rules.addNewRuleTitle": "Créer une nouvelle règle",

5
x-pack/plugins/translations/translations/ja-JP.json
View file

@ -23754,15 +23754,10 @@
"xpack.securitySolution.detectionEngine.ruleDetails.deletedRule": "削除されたルール",
"xpack.securitySolution.detectionEngine.ruleDetails.exceptionsTab": "例外",
"xpack.securitySolution.detectionEngine.ruleDetails.experimentalDescription": "テクニカルプレビュー",
"xpack.securitySolution.detectionEngine.ruleDetails.failureHistoryTab": "エラー履歴",
"xpack.securitySolution.detectionEngine.ruleDetails.lastFiveErrorsTitle": "最後の5件のエラー",
"xpack.securitySolution.detectionEngine.ruleDetails.pageTitle": "ルール詳細",
"xpack.securitySolution.detectionEngine.ruleDetails.ruleCreationDescription": "作成者: {by} 日付: {date}",
"xpack.securitySolution.detectionEngine.ruleDetails.ruleUpdateDescription": "更新者：{by} 日付：{date}",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedAtColumn": "失敗",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedDescription": "失敗",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedMsgColumn": "失敗メッセージ",
"xpack.securitySolution.detectionEngine.ruleDetails.statusTypeColumn": "型",
"xpack.securitySolution.detectionEngine.ruleDetails.unknownDescription": "不明",
"xpack.securitySolution.detectionEngine.rules.aboutRuleTitle": "ルールについて",
"xpack.securitySolution.detectionEngine.rules.addNewRuleTitle": "新規ルールを作成",

5
x-pack/plugins/translations/translations/zh-CN.json
View file

@ -23781,15 +23781,10 @@
"xpack.securitySolution.detectionEngine.ruleDetails.deletedRule": "已删除规则",
"xpack.securitySolution.detectionEngine.ruleDetails.exceptionsTab": "例外",
"xpack.securitySolution.detectionEngine.ruleDetails.experimentalDescription": "技术预览",
"xpack.securitySolution.detectionEngine.ruleDetails.failureHistoryTab": "失败历史记录",
"xpack.securitySolution.detectionEngine.ruleDetails.lastFiveErrorsTitle": "上五个错误",
"xpack.securitySolution.detectionEngine.ruleDetails.pageTitle": "规则详情",
"xpack.securitySolution.detectionEngine.ruleDetails.ruleCreationDescription": "由 {by} 于 {date}创建",
"xpack.securitySolution.detectionEngine.ruleDetails.ruleUpdateDescription": "由 {by} 于 {date}更新",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedAtColumn": "失败于",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedDescription": "失败",
"xpack.securitySolution.detectionEngine.ruleDetails.statusFailedMsgColumn": "失败消息",
"xpack.securitySolution.detectionEngine.ruleDetails.statusTypeColumn": "类型",
"xpack.securitySolution.detectionEngine.ruleDetails.unknownDescription": "未知",
"xpack.securitySolution.detectionEngine.rules.aboutRuleTitle": "关于规则",
"xpack.securitySolution.detectionEngine.rules.addNewRuleTitle": "创建新规则",

227
x-pack/test/detection_engine_api_integration/security_and_spaces/tests/get_rule_execution_events.ts Normal file
View file

@ -0,0 +1,227 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import dateMath from '@elastic/datemath';
import expect from '@kbn/expect';
import moment from 'moment';
import { set } from '@elastic/safer-lodash-set';
import uuid from 'uuid';
import { detectionEngineRuleExecutionEventsUrl } from '../../../../plugins/security_solution/common/constants';
import { RuleExecutionStatus } from '../../../../plugins/security_solution/common/detection_engine/schemas/common';
import { FtrProviderContext } from '../../common/ftr_provider_context';
import {
createRule,
createSignalsIndex,
deleteAllAlerts,
deleteAllEventLogExecutionEvents,
deleteSignalsIndex,
getRuleForSignalTesting,
indexEventLogExecutionEvents,
waitForEventLogExecuteComplete,
waitForRuleSuccessOrStatus,
} from '../../utils';
import { failedGapExecution } from './template_data/execution_events';
// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext) => {
const supertest = getService('supertest');
const esArchiver = getService('esArchiver');
const es = getService('es');
const log = getService('log');
describe('Get Rule Execution Log Events', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/hosts');
await esArchiver.load('x-pack/test/functional/es_archives/security_solution/alias');
await createSignalsIndex(supertest, log);
});
after(async () => {
await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/hosts');
await esArchiver.unload('x-pack/test/functional/es_archives/security_solution/alias');
await deleteSignalsIndex(supertest, log);
});
beforeEach(async () => {
await deleteAllAlerts(supertest, log);
await deleteAllEventLogExecutionEvents(es, log);
});
afterEach(async () => {
await deleteAllAlerts(supertest, log);
await deleteAllEventLogExecutionEvents(es, log);
});
it('should return an error if rule does not exist', async () => {
const start = dateMath.parse('now-24h')?.utc().toISOString();
const end = dateMath.parse('now', { roundUp: true })?.utc().toISOString();
const response = await supertest
.get(detectionEngineRuleExecutionEventsUrl('1'))
.set('kbn-xsrf', 'true')
.query({ start, end });
expect(response.status).to.eql(404);
expect(response.text).to.eql(
'{"message":"Saved object [alert/1] not found","status_code":404}'
);
});
it('should return execution events for a rule that has executed successfully', async () => {
const rule = getRuleForSignalTesting(['auditbeat-*']);
const { id } = await createRule(supertest, log, rule);
await waitForRuleSuccessOrStatus(supertest, log, id);
await waitForEventLogExecuteComplete(es, log, id);
const start = dateMath.parse('now-24h')?.utc().toISOString();
const end = dateMath.parse('now', { roundUp: true })?.utc().toISOString();
const response = await supertest
.get(detectionEngineRuleExecutionEventsUrl(id))
.set('kbn-xsrf', 'true')
.query({ start, end });
expect(response.status).to.eql(200);
expect(response.body.total).to.eql(1);
expect(response.body.events[0].duration_ms).to.greaterThan(0);
expect(response.body.events[0].search_duration_ms).to.greaterThan(0);
expect(response.body.events[0].schedule_delay_ms).to.greaterThan(0);
expect(response.body.events[0].indexing_duration_ms).to.greaterThan(0);
expect(response.body.events[0].gap_duration_ms).to.eql(0);
expect(response.body.events[0].security_status).to.eql('succeeded');
expect(response.body.events[0].security_message).to.eql('succeeded');
});
it('should return execution events for a rule that has executed in a warning state', async () => {
const rule = getRuleForSignalTesting(['auditbeat-*', 'no-name-index']);
const { id } = await createRule(supertest, log, rule);
await waitForRuleSuccessOrStatus(supertest, log, id, RuleExecutionStatus['partial failure']);
await waitForEventLogExecuteComplete(es, log, id);
const start = dateMath.parse('now-24h')?.utc().toISOString();
const end = dateMath.parse('now', { roundUp: true })?.utc().toISOString();
const response = await supertest
.get(detectionEngineRuleExecutionEventsUrl(id))
.set('kbn-xsrf', 'true')
.query({ start, end });
expect(response.status).to.eql(200);
expect(response.body.total).to.eql(1);
expect(response.body.events[0].duration_ms).to.greaterThan(0);
expect(response.body.events[0].search_duration_ms).to.eql(0);
expect(response.body.events[0].schedule_delay_ms).to.greaterThan(0);
expect(response.body.events[0].indexing_duration_ms).to.eql(0);
expect(response.body.events[0].gap_duration_ms).to.eql(0);
expect(response.body.events[0].security_status).to.eql('partial failure');
expect(
response.body.events[0].security_message.startsWith(
'Check privileges failed to execute ResponseError: index_not_found_exception: [index_not_found_exception] Reason: no such index [no-name-index]'
)
).to.eql(true);
});
// TODO: Debug indexing
it.skip('should return execution events for a rule that has executed in a failure state with a gap', async () => {
const rule = getRuleForSignalTesting(['auditbeat-*'], uuid.v4(), false);
const { id } = await createRule(supertest, log, rule);
const start = dateMath.parse('now')?.utc().toISOString();
const end = dateMath.parse('now+24h', { roundUp: true })?.utc().toISOString();
// Create 5 timestamps a minute apart to use in the templated data
const dateTimes = [...Array(5).keys()].map((i) =>
moment(start)
.add(i + 1, 'm')
.toDate()
.toISOString()
);
const events = failedGapExecution.map((e, i) => {
set(e, '@timestamp', dateTimes[i]);
set(e, 'event.start', dateTimes[i]);
set(e, 'event.end', dateTimes[i]);
set(e, 'rule.id', id);
return e;
});
await indexEventLogExecutionEvents(es, log, events);
await waitForEventLogExecuteComplete(es, log, id);
const response = await supertest
.get(detectionEngineRuleExecutionEventsUrl(id))
.set('kbn-xsrf', 'true')
.query({ start, end });
// console.log(JSON.stringify(response));
expect(response.status).to.eql(200);
expect(response.body.total).to.eql(1);
expect(response.body.events[0].duration_ms).to.eql(4236);
expect(response.body.events[0].search_duration_ms).to.eql(0);
expect(response.body.events[0].schedule_delay_ms).to.greaterThan(0);
expect(response.body.events[0].indexing_duration_ms).to.eql(0);
expect(response.body.events[0].gap_duration_ms).to.greaterThan(0);
expect(response.body.events[0].security_status).to.eql('failed');
expect(
response.body.events[0].security_message.startsWith(
'Check privileges failed to execute ResponseError: index_not_found_exception: [index_not_found_exception] Reason: no such index [no-name-index]'
)
).to.eql(true);
});
// it('should return execution events when providing a status filter', async () => {
// const rule = getRuleForSignalTesting(['auditbeat-*', 'no-name-index']);
// const { id } = await createRule(supertest, log, rule);
// await waitForRuleSuccessOrStatus(supertest, log, id, RuleExecutionStatus.failed);
// await waitForSignalsToBePresent(supertest, log, 1, [id]);
//
// const start = dateMath.parse('now-24h')?.utc().toISOString();
// const end = dateMath.parse('now', { roundUp: true })?.utc().toISOString();
// const response = await supertest
// .get(detectionEngineRuleExecutionEventsUrl(id))
// .set('kbn-xsrf', 'true')
// .query({ start, end });
//
// expect(response.status).to.eql(200);
// expect(response.body.total).to.eql(1);
// expect(response.body.events[0].duration_ms).to.greaterThan(0);
// expect(response.body.events[0].search_duration_ms).to.eql(0);
// expect(response.body.events[0].schedule_delay_ms).to.greaterThan(0);
// expect(response.body.events[0].indexing_duration_ms).to.eql(0);
// expect(response.body.events[0].gap_duration_ms).to.eql(0);
// expect(response.body.events[0].security_status).to.eql('failed');
// expect(response.body.events[0].security_message).to.include(
// 'were not queried between this rule execution and the last execution, so signals may have been missed. '
// );
// });
// it('should return execution events when providing a status filter and sortField', async () => {
// const rule = getRuleForSignalTesting(['auditbeat-*', 'no-name-index']);
// const { id } = await createRule(supertest, log, rule);
// await waitForRuleSuccessOrStatus(supertest, log, id, RuleExecutionStatus.failed);
// await waitForSignalsToBePresent(supertest, log, 1, [id]);
//
// const start = dateMath.parse('now-24h')?.utc().toISOString();
// const end = dateMath.parse('now', { roundUp: true })?.utc().toISOString();
// const response = await supertest
// .get(detectionEngineRuleExecutionEventsUrl(id))
// .set('kbn-xsrf', 'true')
// .query({ start, end });
//
// expect(response.status).to.eql(200);
// expect(response.body.total).to.eql(1);
// expect(response.body.events[0].duration_ms).to.greaterThan(0);
// expect(response.body.events[0].search_duration_ms).to.eql(0);
// expect(response.body.events[0].schedule_delay_ms).to.greaterThan(0);
// expect(response.body.events[0].indexing_duration_ms).to.eql(0);
// expect(response.body.events[0].gap_duration_ms).to.eql(0);
// expect(response.body.events[0].security_status).to.eql('failed');
// expect(response.body.events[0].security_message).to.include(
// 'were not queried between this rule execution and the last execution, so signals may have been missed. '
// );
// });
});
};

1
x-pack/test/detection_engine_api_integration/security_and_spaces/tests/index.ts
View file

@ -32,6 +32,7 @@ export default ({ loadTestFile }: FtrProviderContext): void => {
loadTestFile(require.resolve('./find_rules'));
loadTestFile(require.resolve('./generating_signals'));
loadTestFile(require.resolve('./get_prepackaged_rules_status'));
loadTestFile(require.resolve('./get_rule_execution_events'));
loadTestFile(require.resolve('./import_rules'));
loadTestFile(require.resolve('./import_export_rules'));
loadTestFile(require.resolve('./read_rules'));

630
x-pack/test/detection_engine_api_integration/security_and_spaces/tests/template_data/execution_events.ts Normal file
View file

@ -0,0 +1,630 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export const successfulExecution = [
{
'@timestamp': '2022-03-17T22:59:31.360Z',
event: {
provider: 'alerting',
action: 'execute',
kind: 'alert',
category: ['siem'],
start: '2022-03-17T22:59:28.100Z',
outcome: 'success',
end: '2022-03-17T22:59:31.283Z',
duration: 3183000000,
},
kibana: {
alert: {
rule: {
execution: {
uuid: '7e62d859-aeaa-4e3d-8b9c-b3b737356383',
metrics: {
number_of_triggered_actions: 0,
number_of_searches: 2,
es_search_duration_ms: 1,
total_search_duration_ms: 15,
},
},
},
},
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
type_id: 'siem.queryRule',
},
],
task: {
scheduled: '2022-03-17T22:59:25.051Z',
schedule_delay: 3049000000,
},
alerting: {
status: 'ok',
},
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
license: 'basic',
category: 'siem.queryRule',
ruleset: 'siem',
name: 'Lots of Execution Events',
},
message:
"rule executed: siem.queryRule:fb1fc150-a292-11ec-a2cf-c1b28b0392b0: 'Lots of Execution Events'",
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T22:59:30.296Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'metric',
action: 'execution-metrics',
sequence: 1,
},
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
name: 'Lots of Execution Events',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
metrics: {
total_search_duration_ms: 12,
total_indexing_duration_ms: 0,
},
uuid: '7e62d859-aeaa-4e3d-8b9c-b3b737356383',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T22:59:30.296Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'event',
action: 'status-change',
sequence: 2,
},
message: 'succeeded',
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
name: 'Lots of Execution Events',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
status: 'succeeded',
status_order: 0,
uuid: '7e62d859-aeaa-4e3d-8b9c-b3b737356383',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T22:59:28.134Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'event',
action: 'status-change',
sequence: 0,
},
message: '',
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
name: 'Lots of Execution Events',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
status: 'running',
status_order: 15,
uuid: '7e62d859-aeaa-4e3d-8b9c-b3b737356383',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T22:59:28.100Z',
event: {
provider: 'alerting',
action: 'execute-start',
kind: 'alert',
category: ['siem'],
start: '2022-03-17T22:59:28.100Z',
},
kibana: {
alert: {
rule: {
execution: {
uuid: '7e62d859-aeaa-4e3d-8b9c-b3b737356383',
},
},
},
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
type_id: 'siem.queryRule',
},
],
task: {
scheduled: '2022-03-17T22:59:25.051Z',
schedule_delay: 3049000000,
},
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
license: 'basic',
category: 'siem.queryRule',
ruleset: 'siem',
},
message: 'rule execution start: "fb1fc150-a292-11ec-a2cf-c1b28b0392b0"',
ecs: {
version: '1.8.0',
},
},
];
export const failedGapExecution = [
{
'@timestamp': '2022-03-17T12:36:16.413Z',
event: {
provider: 'alerting',
action: 'execute',
kind: 'alert',
category: ['siem'],
start: '2022-03-17T12:36:14.868Z',
outcome: 'success',
end: '2022-03-17T12:36:16.413Z',
duration: 1545000000,
},
kibana: {
alert: {
rule: {
execution: {
uuid: '38fa2d4a-94d3-4ea3-80d6-d1284eb98357',
metrics: {
number_of_triggered_actions: 0,
number_of_searches: 6,
es_search_duration_ms: 2,
total_search_duration_ms: 15,
},
},
},
},
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
type_id: 'siem.queryRule',
},
],
task: {
scheduled: '2022-03-17T12:27:10.060Z',
schedule_delay: 544808000000,
},
alerting: {
status: 'ok',
},
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
license: 'basic',
category: 'siem.queryRule',
ruleset: 'siem',
name: 'Lots of Execution Events',
},
message:
"rule executed: siem.queryRule:fb1fc150-a292-11ec-a2cf-c1b28b0392b0: 'Lots of Execution Events'",
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T12:36:15.382Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'metric',
action: 'execution-metrics',
sequence: 1,
},
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
name: 'Lots of Execution Events',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
metrics: {
execution_gap_duration_s: 245,
},
uuid: '38fa2d4a-94d3-4ea3-80d6-d1284eb98357',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T12:36:15.382Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'event',
action: 'status-change',
sequence: 2,
},
message:
'4 minutes (244689ms) were not queried between this rule execution and the last execution, so signals may have been missed. Consider increasing your look behind time or adding more Kibana instances. name: "Lots of Execution Events" id: "fb1fc150-a292-11ec-a2cf-c1b28b0392b0" rule id: "7c44befd-f611-4994-b116-9861df75d0cb" execution id: "38fa2d4a-94d3-4ea3-80d6-d1284eb98357" space ID: "default"',
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
name: 'Lots of Execution Events',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
status: 'failed',
status_order: 30,
uuid: '38fa2d4a-94d3-4ea3-80d6-d1284eb98357',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T12:36:14.888Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'event',
action: 'status-change',
sequence: 0,
},
message: '',
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
name: 'Lots of Execution Events',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
status: 'running',
status_order: 15,
uuid: '38fa2d4a-94d3-4ea3-80d6-d1284eb98357',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-17T12:36:14.868Z',
event: {
provider: 'alerting',
action: 'execute-start',
kind: 'alert',
category: ['siem'],
start: '2022-03-17T12:36:14.868Z',
},
kibana: {
alert: {
rule: {
execution: {
uuid: '38fa2d4a-94d3-4ea3-80d6-d1284eb98357',
},
},
},
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
type_id: 'siem.queryRule',
},
],
task: {
scheduled: '2022-03-17T12:27:10.060Z',
schedule_delay: 544808000000,
},
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
rule: {
id: 'fb1fc150-a292-11ec-a2cf-c1b28b0392b0',
license: 'basic',
category: 'siem.queryRule',
ruleset: 'siem',
},
message: 'rule execution start: "fb1fc150-a292-11ec-a2cf-c1b28b0392b0"',
ecs: {
version: '1.8.0',
},
},
];
export const partialWarningExecution = [
{
'@timestamp': '2022-03-16T23:28:36.012Z',
event: {
provider: 'alerting',
action: 'execute',
kind: 'alert',
category: ['siem'],
start: '2022-03-16T23:28:34.365Z',
outcome: 'success',
end: '2022-03-16T23:28:36.012Z',
duration: 1647000000,
},
kibana: {
alert: {
rule: {
execution: {
uuid: 'ce37a09d-9359-4756-abbf-e319dd6b1336',
metrics: {
number_of_triggered_actions: 0,
number_of_searches: 2,
es_search_duration_ms: 0,
total_search_duration_ms: 3,
},
},
},
},
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
type_id: 'siem.queryRule',
},
],
task: {
scheduled: '2022-03-16T23:28:31.233Z',
schedule_delay: 3132000000,
},
alerting: {
status: 'ok',
},
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
rule: {
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
license: 'basic',
category: 'siem.queryRule',
ruleset: 'siem',
name: 'This Rule Makes Alerts, Actions, AND Moar!',
},
message:
"rule executed: siem.queryRule:f78f3550-a186-11ec-89a1-0bce95157aba: 'This Rule Makes Alerts, Actions, AND Moar!'",
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-16T23:28:34.998Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'event',
action: 'status-change',
sequence: 1,
},
message:
'Check privileges failed to execute ResponseError: index_not_found_exception: [index_not_found_exception] Reason: no such index [frank] name: "This Rule Makes Alerts, Actions, AND Moar!" id: "f78f3550-a186-11ec-89a1-0bce95157aba" rule id: "b64b4540-d035-4826-a1e7-f505bf4b9653" execution id: "ce37a09d-9359-4756-abbf-e319dd6b1336" space ID: "default"',
rule: {
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
name: 'This Rule Makes Alerts, Actions, AND Moar!',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
status: 'partial failure',
status_order: 20,
uuid: 'ce37a09d-9359-4756-abbf-e319dd6b1336',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-16T23:28:34.386Z',
event: {
provider: 'securitySolution.ruleExecution',
kind: 'event',
action: 'status-change',
sequence: 0,
},
message: '',
rule: {
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
name: 'This Rule Makes Alerts, Actions, AND Moar!',
category: 'siem.queryRule',
},
kibana: {
alert: {
rule: {
execution: {
status: 'running',
status_order: 15,
uuid: 'ce37a09d-9359-4756-abbf-e319dd6b1336',
},
},
},
space_ids: ['default'],
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
},
],
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
ecs: {
version: '1.8.0',
},
},
{
'@timestamp': '2022-03-16T23:28:34.365Z',
event: {
provider: 'alerting',
action: 'execute-start',
kind: 'alert',
category: ['siem'],
start: '2022-03-16T23:28:34.365Z',
},
kibana: {
alert: {
rule: {
execution: {
uuid: 'ce37a09d-9359-4756-abbf-e319dd6b1336',
},
},
},
saved_objects: [
{
rel: 'primary',
type: 'alert',
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
type_id: 'siem.queryRule',
},
],
task: {
scheduled: '2022-03-16T23:28:31.233Z',
schedule_delay: 3132000000,
},
server_uuid: '5b2de169-2785-441b-ae8c-186a1936b17d',
version: '8.2.0',
},
rule: {
id: 'f78f3550-a186-11ec-89a1-0bce95157aba',
license: 'basic',
category: 'siem.queryRule',
ruleset: 'siem',
},
message: 'rule execution start: "f78f3550-a186-11ec-89a1-0bce95157aba"',
ecs: {
version: '1.8.0',
},
},
];

119
x-pack/test/detection_engine_api_integration/utils.ts
View file

@ -5,6 +5,7 @@
* 2.0.
*/
import { SearchTotalHits } from '@elastic/elasticsearch/lib/api/types';
import { KbnClient } from '@kbn/test';
import { ALERT_RULE_RULE_ID, ALERT_RULE_UUID } from '@kbn/rule-data-utils';
@ -1466,6 +1467,124 @@ export const waitForSignalsToBePresent = async (
);
};
/**
* Waits for the event-log execution completed doc count to be greater than the
* supplied number before continuing with a default of at least one execution
* @param es The ES client
* @param log
* @param ruleId The id of rule to check execution logs for
* @param totalExecutions The number of executions to wait for, default is 1
*/
export const waitForEventLogExecuteComplete = async (
es: Client,
log: ToolingLog,
ruleId: string,
totalExecutions = 1
): Promise<void> => {
await waitFor(
async () => {
const executionCount = await getEventLogExecuteCompleteById(es, log, ruleId);
return executionCount >= totalExecutions;
},
'waitForEventLogExecuteComplete',
log
);
};
/**
* Given a single rule id this will return the number of event-log execution
* completed docs
* @param es The ES client
* @param log
* @param ruleId Rule id
*/
export const getEventLogExecuteCompleteById = async (
es: Client,
log: ToolingLog,
ruleId: string
): Promise<number> => {
const response = await es.search({
index: '.kibana-event-log*',
track_total_hits: true,
size: 0,
query: {
bool: {
must: [],
filter: [
{
match_phrase: {
'event.provider': 'alerting',
},
},
{
match_phrase: {
'event.action': 'execute',
},
},
{
match_phrase: {
'rule.id': ruleId,
},
},
],
should: [],
must_not: [],
},
},
});
return (response?.hits?.total as SearchTotalHits)?.value ?? 0;
};
/**
* Indexes provided execution events into .kibana-event-log-*
* @param es The ElasticSearch handle
* @param log The tooling logger
* @param events
*/
export const indexEventLogExecutionEvents = async (
es: Client,
log: ToolingLog,
events: object[]
): Promise<void> => {
const operations = events.flatMap((doc: object) => [
{ index: { _index: '.kibana-event-log-*' } },
doc,
]);
await es.bulk({ refresh: true, operations });
return;
};
/**
* Remove all .kibana-event-log-* documents with an execution.uuid
* This will retry 20 times before giving up and hopefully still not interfere with other tests
* @param es The ElasticSearch handle
* @param log The tooling logger
*/
export const deleteAllEventLogExecutionEvents = async (
es: Client,
log: ToolingLog
): Promise<void> => {
return countDownES(
async () => {
return es.deleteByQuery(
{
index: '.kibana-event-log-*',
q: '_exists_:kibana.alert.rule.execution.uuid',
wait_for_completion: true,
refresh: true,
body: {},
},
{ meta: true }
);
},
'deleteAllEventLogExecutionEvents',
log
);
};
/**
* Returns all signals both closed and opened by ruleId
* @param supertest Deps