github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 01:38:56 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Security Solution] Handle specific fields in /upgrade/_review endpoint and refactor diff logic to use Zod (#186615)

Browse source 
Fixes: https://github.com/elastic/kibana/issues/180393

## Summary

Handles specific fields in `/upgrade/_review` endpoint upgrade workflow,
as described in https://github.com/elastic/kibana/issues/180393.

Achieves this with two mechanisms:

1. Removing fields from the `PrebuiltRuleAsset` schema, which excludes
the field from the diff calculation completely.
2. Manually removing the diff calculation for certain fields, by
excluding them from
`/common/api/detection_engine/prebuilt_rules/model/diff/diffable_rule/diffable_rule.ts`

Also, refactors a part of the codebase from its prior usage of `io-ts`
schema types to use autogenerated Zod types.

With this refactor, most of the
`x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy`
could be deleted. Unluckily some of the types manually created there are
still used in some complex types elsewhere, so I added a note to that
file indicating that those should be migrated to Zod, so that the legacy
folder can finally be deleted.


### Checklist

Delete any items that are not applicable to this PR.

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
This commit is contained in:
Juan Pablo Djeredjian 2024-07-11 10:59:06 +02:00 • committed by GitHub
parent d0d3847c7e
commit 7950fb85ca
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
27 changed files with 712 additions and 1391 deletions
Download patch file Download diff file Expand all files Collapse all files

74
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/common_attributes.gen.ts
View file

@ -304,8 +304,29 @@ export type TimestampOverrideFallbackDisabled = z.infer<typeof TimestampOverride
export const TimestampOverrideFallbackDisabled = z.boolean();
/**
* Describes an Elasticsearch field that is needed for the rule to function
*/
* Describes an Elasticsearch field that is needed for the rule to function.
Almost all types of Security rules check source event documents for a match to some kind of
query or filter. If a document has certain field with certain values, then it's a match and
the rule will generate an alert.
Required field is an event field that must be present in the source indices of a given rule.
@example
const standardEcsField: RequiredField = {
name: 'event.action',
type: 'keyword',
ecs: true,
};
@example
const nonEcsField: RequiredField = {
name: 'winlog.event_data.AttributeLDAPDisplayName',
type: 'keyword',
ecs: false,
};
*/
export type RequiredField = z.infer<typeof RequiredField>;
export const RequiredField = z.object({
/**
@ -368,6 +389,39 @@ export const SavedObjectResolveAliasPurpose = z.enum([
export type SavedObjectResolveAliasPurposeEnum = typeof SavedObjectResolveAliasPurpose.enum;
export const SavedObjectResolveAliasPurposeEnum = SavedObjectResolveAliasPurpose.enum;
/**
* Related integration is a potential dependency of a rule. It's assumed that if the user installs
one of the related integrations of a rule, the rule might start to work properly because it will
have source events (generated by this integration) potentially matching the rule's query.
NOTE: Proper work is not guaranteed, because a related integration, if installed, can be
configured differently or generate data that is not necessarily relevant for this rule.
Related integration is a combination of a Fleet package and (optionally) one of the
package's "integrations" that this package contains. It is represented by 3 properties:
- `package`: name of the package (required, unique id)
- `version`: version of the package (required, semver-compatible)
- `integration`: name of the integration of this package (optional, id within the package)
There are Fleet packages like `windows` that contain only one integration; in this case,
`integration` should be unspecified. There are also packages like `aws` and `azure` that contain
several integrations; in this case, `integration` should be specified.
@example
const x: RelatedIntegration = {
package: 'windows',
version: '1.5.x',
};
@example
const x: RelatedIntegration = {
package: 'azure',
version: '~1.1.6',
integration: 'activitylogs',
};
*/
export type RelatedIntegration = z.infer<typeof RelatedIntegration>;
export const RelatedIntegration = z.object({
package: NonEmptyString,
@ -378,6 +432,22 @@ export const RelatedIntegration = z.object({
export type RelatedIntegrationArray = z.infer<typeof RelatedIntegrationArray>;
export const RelatedIntegrationArray = z.array(RelatedIntegration);
/**
* Schema for fields relating to investigation fields. These are user defined fields we use to highlight
in various features in the UI such as alert details flyout and exceptions auto-population from alert.
Added in PR #163235
Right now we only have a single field but anticipate adding more related fields to store various
configuration states such as `override` - where a user might say if they want only these fields to
display, or if they want these fields + the fields we select. When expanding this field, it may look
something like:
```typescript
const investigationFields = z.object({
field_names: NonEmptyArray(NonEmptyString),
override: z.boolean().optional(),
});
```
*/
export type InvestigationFields = z.infer<typeof InvestigationFields>;
export const InvestigationFields = z.object({
field_names: z.array(NonEmptyString).min(1),

70
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/common_attributes.schema.yaml
View file

@ -315,7 +315,28 @@ components:
RequiredField:
type: object
description: Describes an Elasticsearch field that is needed for the rule to function
description: |
Describes an Elasticsearch field that is needed for the rule to function.
Almost all types of Security rules check source event documents for a match to some kind of
query or filter. If a document has certain field with certain values, then it's a match and
the rule will generate an alert.
Required field is an event field that must be present in the source indices of a given rule.
@example
const standardEcsField: RequiredField = {
name: 'event.action',
type: 'keyword',
ecs: true,
};
@example
const nonEcsField: RequiredField = {
name: 'winlog.event_data.AttributeLDAPDisplayName',
type: 'keyword',
ecs: false,
};
properties:
name:
$ref: '../../../model/primitives.schema.yaml#/components/schemas/NonEmptyString'
@ -376,6 +397,37 @@ components:
RelatedIntegration:
type: object
description: |
Related integration is a potential dependency of a rule. It's assumed that if the user installs
one of the related integrations of a rule, the rule might start to work properly because it will
have source events (generated by this integration) potentially matching the rule's query.
NOTE: Proper work is not guaranteed, because a related integration, if installed, can be
configured differently or generate data that is not necessarily relevant for this rule.
Related integration is a combination of a Fleet package and (optionally) one of the
package's "integrations" that this package contains. It is represented by 3 properties:
- `package`: name of the package (required, unique id)
- `version`: version of the package (required, semver-compatible)
- `integration`: name of the integration of this package (optional, id within the package)
There are Fleet packages like `windows` that contain only one integration; in this case,
`integration` should be unspecified. There are also packages like `aws` and `azure` that contain
several integrations; in this case, `integration` should be specified.
@example
const x: RelatedIntegration = {
package: 'windows',
version: '1.5.x',
};
@example
const x: RelatedIntegration = {
package: 'azure',
version: '~1.1.6',
integration: 'activitylogs',
};
properties:
package:
$ref: '../../../model/primitives.schema.yaml#/components/schemas/NonEmptyString'
@ -392,10 +444,22 @@ components:
items:
$ref: '#/components/schemas/RelatedIntegration'
# Schema for fields relating to investigation fields, these are user defined fields we use to highlight in various features in the UI such as alert details flyout and exceptions auto-population from alert. Added in PR #163235
# Right now we only have a single field but anticipate adding more related fields to store various configuration states such as `override` - where a user might say if they want only these fields to display, or if they want these fields + the fields we select.
InvestigationFields:
type: object
description: |
Schema for fields relating to investigation fields. These are user defined fields we use to highlight
in various features in the UI such as alert details flyout and exceptions auto-population from alert.
Added in PR #163235
Right now we only have a single field but anticipate adding more related fields to store various
configuration states such as `override` - where a user might say if they want only these fields to
display, or if they want these fields + the fields we select. When expanding this field, it may look
something like:
```typescript
const investigationFields = z.object({
field_names: NonEmptyArray(NonEmptyString),
override: z.boolean().optional(),
});
```
properties:
field_names:
type: array

135
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/time_duration.test.ts Normal file
View file

@ -0,0 +1,135 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { expectParseError, expectParseSuccess, stringifyZodError } from '@kbn/zod-helpers';
import { TimeDuration } from './time_duration'; // Update with the actual path to your TimeDuration file
describe('TimeDuration schema', () => {
test('it should validate a correctly formed TimeDuration with time unit of seconds', () => {
const payload = '1s';
const schema = TimeDuration({ allowedUnits: ['s'] });
const result = schema.safeParse(payload);
expectParseSuccess(result);
expect(result.data).toEqual(payload);
});
test('it should validate a correctly formed TimeDuration with time unit of minutes', () => {
const payload = '100m';
const schema = TimeDuration({ allowedUnits: ['s', 'm'] });
const result = schema.safeParse(payload);
expectParseSuccess(result);
expect(result.data).toEqual(payload);
});
test('it should validate a correctly formed TimeDuration with time unit of hours', () => {
const payload = '10000000h';
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h'] });
const result = schema.safeParse(payload);
expectParseSuccess(result);
expect(result.data).toEqual(payload);
});
test('it should validate a correctly formed TimeDuration with time unit of days', () => {
const payload = '7d';
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h', 'd'] });
const result = schema.safeParse(payload);
expectParseSuccess(result);
expect(result.data).toEqual(payload);
});
test('it should NOT validate a correctly formed TimeDuration with time unit of seconds if it is not an allowed unit', () => {
const payload = '30s';
const schema = TimeDuration({ allowedUnits: ['m', 'h', 'd'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. \\"30s\\", \\"1m\\", \\"2h\\", \\"7d\\""`
);
});
test('it should NOT validate a negative TimeDuration', () => {
const payload = '-10s';
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h', 'd'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. \\"30s\\", \\"1m\\", \\"2h\\", \\"7d\\""`
);
});
test('it should NOT validate a fractional number', () => {
const payload = '1.5s';
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h', 'd'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. \\"30s\\", \\"1m\\", \\"2h\\", \\"7d\\""`
);
});
test('it should NOT validate a TimeDuration with an invalid time unit', () => {
const payload = '10000000days';
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h', 'd'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. \\"30s\\", \\"1m\\", \\"2h\\", \\"7d\\""`
);
});
test('it should NOT validate a TimeDuration with a time interval with incorrect format', () => {
const payload = '100ff0000w';
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. \\"30s\\", \\"1m\\", \\"2h\\", \\"7d\\""`
);
});
test('it should NOT validate an empty string', () => {
const payload = '';
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. \\"30s\\", \\"1m\\", \\"2h\\", \\"7d\\""`
);
});
test('it should NOT validate a number', () => {
const payload = 100;
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Expected string, received number"`
);
});
test('it should NOT validate a TimeDuration with a valid time unit but unsafe integer', () => {
const payload = `${Math.pow(2, 53)}h`;
const schema = TimeDuration({ allowedUnits: ['s', 'm', 'h'] });
const result = schema.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. \\"30s\\", \\"1m\\", \\"2h\\", \\"7d\\""`
);
});
});

53
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/time_duration.ts Normal file
View file

@ -0,0 +1,53 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { z } from 'zod';
type TimeUnits = 's' | 'm' | 'h' | 'd' | 'w' | 'y';
interface TimeDurationType {
allowedUnits: TimeUnits[];
}
const isTimeSafe = (time: number) => time >= 1 && Number.isSafeInteger(time);
/**
* Types the TimeDuration as:
* - A string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time
* - in the format {safe_integer}{timeUnit}, e.g. "30s", "1m", "2h", "7d"
*
* Example usage:
* ```
* const schedule: RuleSchedule = {
* interval: TimeDuration({ allowedUnits: ['s', 'm', 'h'] }).parse('3h'),
* };
* ```
*/
export const TimeDuration = ({ allowedUnits }: TimeDurationType) => {
return z.string().refine(
(input) => {
if (input.trim() === '') return false;
try {
const inputLength = input.length;
const time = Number(input.trim().substring(0, inputLength - 1));
const unit = input.trim().at(-1) as TimeUnits;
return isTimeSafe(time) && allowedUnits.includes(unit);
} catch (error) {
return false;
}
},
{
message:
'Invalid time duration format. Must be a string that is not empty, and composed of a positive integer greater than 0 followed by a unit of time in the format {safe_integer}{timeUnit}, e.g. "30s", "1m", "2h", "7d"',
}
);
};
export type TimeDurationSchema = ReturnType<typeof TimeDuration>;
export type TimeDuration = z.infer<TimeDurationSchema>;

264
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/common_attributes.ts
View file

@ -6,9 +6,24 @@
*/
import * as t from 'io-ts';
import { listArray } from '@kbn/securitysolution-io-ts-list-types';
import { NonEmptyString, version, UUID, NonEmptyArray } from '@kbn/securitysolution-io-ts-types';
import { max_signals, threat } from '@kbn/securitysolution-io-ts-alerting-types';
import { NonEmptyString, UUID } from '@kbn/securitysolution-io-ts-types';
/*
IMPORTANT NOTE ON THIS FILE:
This file contains the remaining rule schema types created manually via io-ts. They have been
migrated to Zod schemas created via code generation out of OpenAPI schemas
(found in x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/common_attributes.gen.ts)
The remaining types here couldn't easily be deleted/replaced because they are dependencies in
complex derived schemas in two files:
- x-pack/plugins/security_solution/common/api/detection_engine/rule_exceptions/find_exception_references/find_exception_references_route.ts
- x-pack/plugins/security_solution/common/api/timeline/model/api.ts
Once those two files are migrated to Zod, the /common/api/detection_engine/model/rule_schema_legacy
folder can be removed.
*/
export type RuleObjectId = t.TypeOf<typeof RuleObjectId>;
export const RuleObjectId = UUID;
@ -24,156 +39,6 @@ export const RuleSignatureId = t.string; // should be non-empty string?
export type RuleName = t.TypeOf<typeof RuleName>;
export const RuleName = NonEmptyString;
export type RuleDescription = t.TypeOf<typeof RuleDescription>;
export const RuleDescription = NonEmptyString;
export type RuleVersion = t.TypeOf<typeof RuleVersion>;
export const RuleVersion = version;
export type IsRuleImmutable = t.TypeOf<typeof IsRuleImmutable>;
export const IsRuleImmutable = t.boolean;
export type IsRuleEnabled = t.TypeOf<typeof IsRuleEnabled>;
export const IsRuleEnabled = t.boolean;
export type RuleTagArray = t.TypeOf<typeof RuleTagArray>;
export const RuleTagArray = t.array(t.string); // should be non-empty strings?
/**
* Note that this is a non-exact io-ts type as we allow extra meta information
* to be added to the meta object
*/
export type RuleMetadata = t.TypeOf<typeof RuleMetadata>;
export const RuleMetadata = t.UnknownRecord; // should be a more specific type?
export type RuleLicense = t.TypeOf<typeof RuleLicense>;
export const RuleLicense = t.string;
export type RuleAuthorArray = t.TypeOf<typeof RuleAuthorArray>;
export const RuleAuthorArray = t.array(t.string); // should be non-empty strings?
export type RuleFalsePositiveArray = t.TypeOf<typeof RuleFalsePositiveArray>;
export const RuleFalsePositiveArray = t.array(t.string); // should be non-empty strings?
export type RuleReferenceArray = t.TypeOf<typeof RuleReferenceArray>;
export const RuleReferenceArray = t.array(t.string); // should be non-empty strings?
export type InvestigationGuide = t.TypeOf<typeof InvestigationGuide>;
export const InvestigationGuide = t.string;
/**
* Any instructions for the user for setting up their environment in order to start receiving
* source events for a given rule.
*
* It's a multiline text. Markdown is supported.
*/
export type SetupGuide = t.TypeOf<typeof SetupGuide>;
export const SetupGuide = t.string;
export type BuildingBlockType = t.TypeOf<typeof BuildingBlockType>;
export const BuildingBlockType = t.string;
export type AlertsIndex = t.TypeOf<typeof AlertsIndex>;
export const AlertsIndex = t.string;
export type AlertsIndexNamespace = t.TypeOf<typeof AlertsIndexNamespace>;
export const AlertsIndexNamespace = t.string;
export type ExceptionListArray = t.TypeOf<typeof ExceptionListArray>;
export const ExceptionListArray = listArray;
export type MaxSignals = t.TypeOf<typeof MaxSignals>;
export const MaxSignals = max_signals;
export type ThreatArray = t.TypeOf<typeof ThreatArray>;
export const ThreatArray = t.array(threat);
export type IndexPatternArray = t.TypeOf<typeof IndexPatternArray>;
export const IndexPatternArray = t.array(t.string);
export type DataViewId = t.TypeOf<typeof DataViewId>;
export const DataViewId = t.string;
export type RuleQuery = t.TypeOf<typeof RuleQuery>;
export const RuleQuery = t.string;
/**
* TODO: Right now the filters is an "unknown", when it could more than likely
* become the actual ESFilter as a type.
*/
export type RuleFilterArray = t.TypeOf<typeof RuleFilterArray>; // Filters are not easily type-able yet
export const RuleFilterArray = t.array(t.unknown); // Filters are not easily type-able yet
export type RuleNameOverride = t.TypeOf<typeof RuleNameOverride>;
export const RuleNameOverride = t.string; // should be non-empty string?
export type TimestampOverride = t.TypeOf<typeof TimestampOverride>;
export const TimestampOverride = t.string; // should be non-empty string?
export type TimestampOverrideFallbackDisabled = t.TypeOf<typeof TimestampOverrideFallbackDisabled>;
export const TimestampOverrideFallbackDisabled = t.boolean;
/**
* Almost all types of Security rules check source event documents for a match to some kind of
* query or filter. If a document has certain field with certain values, then it's a match and
* the rule will generate an alert.
*
* Required field is an event field that must be present in the source indices of a given rule.
*
* @example
* const standardEcsField: RequiredField = {
* name: 'event.action',
* type: 'keyword',
* ecs: true,
* };
*
* @example
* const nonEcsField: RequiredField = {
* name: 'winlog.event_data.AttributeLDAPDisplayName',
* type: 'keyword',
* ecs: false,
* };
*/
export type RequiredField = t.TypeOf<typeof RequiredField>;
export const RequiredField = t.exact(
t.type({
name: NonEmptyString,
type: NonEmptyString,
ecs: t.boolean,
})
);
/**
* Array of event fields that must be present in the source indices of a given rule.
*
* @example
* const x: RequiredFieldArray = [
* {
* name: 'event.action',
* type: 'keyword',
* ecs: true,
* },
* {
* name: 'event.code',
* type: 'keyword',
* ecs: true,
* },
* {
* name: 'winlog.event_data.AttributeLDAPDisplayName',
* type: 'keyword',
* ecs: false,
* },
* ];
*/
export type RequiredFieldArray = t.TypeOf<typeof RequiredFieldArray>;
export const RequiredFieldArray = t.array(RequiredField);
export type TimelineTemplateId = t.TypeOf<typeof TimelineTemplateId>;
export const TimelineTemplateId = t.string; // should be non-empty string?
export type TimelineTemplateTitle = t.TypeOf<typeof TimelineTemplateTitle>;
export const TimelineTemplateTitle = t.string; // should be non-empty string?
/**
* Outcome is a property of the saved object resolve api
* will tell us info about the rule after 8.0 migrations
@ -193,96 +58,3 @@ export const SavedObjectResolveAliasPurpose = t.union([
t.literal('savedObjectConversion'),
t.literal('savedObjectImport'),
]);
/**
* Related integration is a potential dependency of a rule. It's assumed that if the user installs
* one of the related integrations of a rule, the rule might start to work properly because it will
* have source events (generated by this integration) potentially matching the rule's query.
*
* NOTE: Proper work is not guaranteed, because a related integration, if installed, can be
* configured differently or generate data that is not necessarily relevant for this rule.
*
* Related integration is a combination of a Fleet package and (optionally) one of the
* package's "integrations" that this package contains. It is represented by 3 properties:
*
* - `package`: name of the package (required, unique id)
* - `version`: version of the package (required, semver-compatible)
* - `integration`: name of the integration of this package (optional, id within the package)
*
* There are Fleet packages like `windows` that contain only one integration; in this case,
* `integration` should be unspecified. There are also packages like `aws` and `azure` that contain
* several integrations; in this case, `integration` should be specified.
*
* @example
* const x: RelatedIntegration = {
* package: 'windows',
* version: '1.5.x',
* };
*
* @example
* const x: RelatedIntegration = {
* package: 'azure',
* version: '~1.1.6',
* integration: 'activitylogs',
* };
*/
export type RelatedIntegration = t.TypeOf<typeof RelatedIntegration>;
export const RelatedIntegration = t.exact(
t.intersection([
t.type({
package: NonEmptyString,
version: NonEmptyString,
}),
t.partial({
integration: NonEmptyString,
}),
])
);
/**
* Array of related integrations.
*
* @example
* const x: RelatedIntegrationArray = [
* {
* package: 'windows',
* version: '1.5.x',
* },
* {
* package: 'azure',
* version: '~1.1.6',
* integration: 'activitylogs',
* },
* ];
*/
export type RelatedIntegrationArray = t.TypeOf<typeof RelatedIntegrationArray>;
export const RelatedIntegrationArray = t.array(RelatedIntegration);
/**
* Schema for fields relating to investigation fields, these are user defined fields we use to highlight
* in various features in the UI such as alert details flyout and exceptions auto-population from alert.
* Added in PR #163235
* Right now we only have a single field but anticipate adding more related fields to store various
* configuration states such as `override` - where a user might say if they want only these fields to
* display, or if they want these fields + the fields we select. When expanding this field, it may look
* something like:
* export const investigationFields = t.intersection([
* t.exact(
* t.type({
* field_names: NonEmptyArray(NonEmptyString),
* })
* ),
* t.exact(
* t.partial({
* overide: t.boolean,
* })
* ),
* ]);
*
*/
export type InvestigationFields = t.TypeOf<typeof InvestigationFields>;
export const InvestigationFields = t.exact(
t.type({
field_names: NonEmptyArray(NonEmptyString),
})
);

19
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/eql_attributes.ts
View file

@ -1,19 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as t from 'io-ts';
// Attributes specific to EQL rules
export type EventCategoryOverride = t.TypeOf<typeof EventCategoryOverride>;
export const EventCategoryOverride = t.string; // should be non-empty string?
export type TimestampField = t.TypeOf<typeof TimestampField>;
export const TimestampField = t.string; // should be non-empty string?
export type TiebreakerField = t.TypeOf<typeof TiebreakerField>;
export const TiebreakerField = t.string; // should be non-empty string?

7
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/index.ts
View file

@ -6,10 +6,3 @@
*/
export * from './common_attributes';
export * from './eql_attributes';
export * from './new_terms_attributes';
export * from './query_attributes';
export * from './threshold_attributes';
export * from './rule_schemas';

25
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/new_terms_attributes.ts
View file

@ -1,25 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as t from 'io-ts';
import { LimitedSizeArray, NonEmptyString } from '@kbn/securitysolution-io-ts-types';
import { MAX_NUMBER_OF_NEW_TERMS_FIELDS } from '../../../../constants';
// Attributes specific to New Terms rules
/**
* New terms rule type supports a limited number of fields. Max number of fields is 3 and defined in common constants as MAX_NUMBER_OF_NEW_TERMS_FIELDS
*/
export type NewTermsFields = t.TypeOf<typeof NewTermsFields>;
export const NewTermsFields = LimitedSizeArray({
codec: t.string,
minSize: 1,
maxSize: MAX_NUMBER_OF_NEW_TERMS_FIELDS,
});
export type HistoryWindowStart = t.TypeOf<typeof HistoryWindowStart>;
export const HistoryWindowStart = NonEmptyString;

70
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/query_attributes.ts
View file

@ -1,70 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as t from 'io-ts';
import {
LimitedSizeArray,
PositiveIntegerGreaterThanZero,
enumeration,
} from '@kbn/securitysolution-io-ts-types';
import { AlertSuppressionMissingFieldsStrategyEnum } from '../rule_schema/common_attributes.gen';
export type AlertSuppressionMissingFields = t.TypeOf<typeof AlertSuppressionMissingFields>;
export const AlertSuppressionMissingFields = enumeration(
'AlertSuppressionMissingFields',
AlertSuppressionMissingFieldsStrategyEnum
);
export const AlertSuppressionGroupBy = LimitedSizeArray({
codec: t.string,
minSize: 1,
maxSize: 3,
});
export const AlertSuppressionDuration = t.type({
value: PositiveIntegerGreaterThanZero,
unit: t.keyof({
s: null,
m: null,
h: null,
}),
});
/**
* Schema for fields relating to alert suppression, which enables limiting the number of alerts per entity.
* e.g. group_by: ['host.name'] would create only one alert per value of host.name. The created alert
* contains metadata about how many other candidate alerts with the same host.name value were suppressed.
*/
export type AlertSuppression = t.TypeOf<typeof AlertSuppression>;
export const AlertSuppression = t.intersection([
t.exact(
t.type({
group_by: AlertSuppressionGroupBy,
})
),
t.exact(
t.partial({
duration: AlertSuppressionDuration,
missing_fields_strategy: AlertSuppressionMissingFields,
})
),
]);
export type AlertSuppressionCamel = t.TypeOf<typeof AlertSuppressionCamel>;
export const AlertSuppressionCamel = t.intersection([
t.exact(
t.type({
groupBy: AlertSuppressionGroupBy,
})
),
t.exact(
t.partial({
duration: AlertSuppressionDuration,
missingFieldsStrategy: AlertSuppressionMissingFields,
})
),
]);

74
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/response_actions.ts
View file

@ -1,74 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { arrayQueries, ecsMapping } from '@kbn/osquery-io-ts-types';
import * as t from 'io-ts';
// to enable using RESPONSE_ACTION_API_COMMANDS_NAMES as a type
function keyObject<T extends readonly string[]>(arr: T): { [K in T[number]]: null } {
return Object.fromEntries(arr.map((v) => [v, null])) as never;
}
export type EndpointParams = t.TypeOf<typeof EndpointParams>;
export const EndpointParams = t.type({
// TODO: TC- change these when we go GA with automated process actions
command: t.keyof(keyObject(['isolate', 'kill-process', 'suspend-process'])),
// command: t.keyof(keyObject(ENABLED_AUTOMATED_RESPONSE_ACTION_COMMANDS)),
comment: t.union([t.string, t.undefined]),
});
export const OsqueryParams = t.type({
query: t.union([t.string, t.undefined]),
ecs_mapping: t.union([ecsMapping, t.undefined]),
queries: t.union([arrayQueries, t.undefined]),
pack_id: t.union([t.string, t.undefined]),
saved_query_id: t.union([t.string, t.undefined]),
});
export const OsqueryParamsCamelCase = t.type({
query: t.union([t.string, t.undefined]),
ecsMapping: t.union([ecsMapping, t.undefined]),
queries: t.union([arrayQueries, t.undefined]),
packId: t.union([t.string, t.undefined]),
savedQueryId: t.union([t.string, t.undefined]),
});
// When we create new response action types, create a union of types
export type RuleResponseOsqueryAction = t.TypeOf<typeof RuleResponseOsqueryAction>;
export const RuleResponseOsqueryAction = t.strict({
actionTypeId: t.literal('.osquery'),
params: OsqueryParamsCamelCase,
});
export type RuleResponseEndpointAction = t.TypeOf<typeof RuleResponseEndpointAction>;
export const RuleResponseEndpointAction = t.strict({
actionTypeId: t.literal('.endpoint'),
params: EndpointParams,
});
export type RuleResponseAction = t.TypeOf<typeof ResponseActionRuleParam>;
const ResponseActionRuleParam = t.union([RuleResponseOsqueryAction, RuleResponseEndpointAction]);
export const ResponseActionRuleParamsOrUndefined = t.union([
t.array(ResponseActionRuleParam),
t.undefined,
]);
// When we create new response action types, create a union of types
const OsqueryResponseAction = t.strict({
action_type_id: t.literal('.osquery'),
params: OsqueryParams,
});
const EndpointResponseAction = t.strict({
action_type_id: t.literal('.endpoint'),
params: EndpointParams,
});
export type ResponseAction = t.TypeOf<typeof ResponseAction>;
export const ResponseAction = t.union([OsqueryResponseAction, EndpointResponseAction]);
export const ResponseActionArray = t.array(ResponseAction);

388
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/rule_schemas.ts
View file

@ -1,388 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as t from 'io-ts';
import {
concurrent_searches,
items_per_search,
machine_learning_job_id,
RiskScore,
RiskScoreMapping,
RuleActionArray,
RuleActionThrottle,
RuleInterval,
RuleIntervalFrom,
RuleIntervalTo,
Severity,
SeverityMapping,
threat_filters,
threat_index,
threat_indicator_path,
threat_mapping,
threat_query,
} from '@kbn/securitysolution-io-ts-alerting-types';
import { PositiveInteger } from '@kbn/securitysolution-io-ts-types';
import { ResponseActionArray } from './response_actions';
import { anomaly_threshold, saved_id } from '../schemas';
import {
AlertsIndex,
AlertsIndexNamespace,
BuildingBlockType,
DataViewId,
ExceptionListArray,
IndexPatternArray,
InvestigationFields,
InvestigationGuide,
IsRuleEnabled,
MaxSignals,
RuleAuthorArray,
RuleDescription,
RuleFalsePositiveArray,
RuleFilterArray,
RuleLicense,
RuleMetadata,
RuleName,
RuleNameOverride,
RuleQuery,
RuleReferenceArray,
RuleSignatureId,
RuleTagArray,
RuleVersion,
SavedObjectResolveAliasPurpose,
SavedObjectResolveAliasTargetId,
SavedObjectResolveOutcome,
ThreatArray,
TimelineTemplateId,
TimelineTemplateTitle,
TimestampOverride,
TimestampOverrideFallbackDisabled,
} from './common_attributes';
import { EventCategoryOverride, TiebreakerField, TimestampField } from './eql_attributes';
import { HistoryWindowStart, NewTermsFields } from './new_terms_attributes';
import { AlertSuppression } from './query_attributes';
import { Threshold } from './threshold_attributes';
export const buildRuleSchemas = <
Required extends t.Props,
Optional extends t.Props,
Defaultable extends t.Props
>({
required,
optional,
defaultable,
}: {
required: Required;
optional: Optional;
defaultable: Defaultable;
}) => ({
create: t.intersection([
t.exact(t.type(required)),
t.exact(t.partial(optional)),
t.exact(t.partial(defaultable)),
]),
patch: t.intersection([t.partial(required), t.partial(optional), t.partial(defaultable)]),
response: t.intersection([
t.exact(t.type(required)),
// This bit of logic is to force all fields to be accounted for in conversions from the internal
// rule schema to the response schema. Rather than use `t.partial`, which makes each field optional,
// we make each field required but possibly undefined. The result is that if a field is forgotten in
// the conversion from internal schema to response schema TS will report an error. If we just used t.partial
// instead, then optional fields can be accidentally omitted from the conversion - and any actual values
// in those fields internally will be stripped in the response.
t.exact(t.type(orUndefined(optional))),
t.exact(t.type(defaultable)),
]),
});
export type OrUndefined<P extends t.Props> = {
[K in keyof P]: P[K] | t.UndefinedC;
};
export const orUndefined = <P extends t.Props>(props: P): OrUndefined<P> => {
return Object.keys(props).reduce<t.Props>((acc, key) => {
acc[key] = t.union([props[key], t.undefined]);
return acc;
}, {}) as OrUndefined<P>;
};
// -------------------------------------------------------------------------------------------------
// Base schema
export const baseSchema = buildRuleSchemas({
required: {
name: RuleName,
description: RuleDescription,
risk_score: RiskScore,
severity: Severity,
},
optional: {
// Field overrides
rule_name_override: RuleNameOverride,
timestamp_override: TimestampOverride,
timestamp_override_fallback_disabled: TimestampOverrideFallbackDisabled,
// Timeline template
timeline_id: TimelineTemplateId,
timeline_title: TimelineTemplateTitle,
// Attributes related to SavedObjectsClient.resolve API
outcome: SavedObjectResolveOutcome,
alias_target_id: SavedObjectResolveAliasTargetId,
alias_purpose: SavedObjectResolveAliasPurpose,
// Misc attributes
license: RuleLicense,
note: InvestigationGuide,
building_block_type: BuildingBlockType,
output_index: AlertsIndex,
namespace: AlertsIndexNamespace,
meta: RuleMetadata,
investigation_fields: InvestigationFields,
// Throttle
throttle: RuleActionThrottle,
},
defaultable: {
// Main attributes
version: RuleVersion,
tags: RuleTagArray,
enabled: IsRuleEnabled,
// Field overrides
risk_score_mapping: RiskScoreMapping,
severity_mapping: SeverityMapping,
// Rule schedule
interval: RuleInterval,
from: RuleIntervalFrom,
to: RuleIntervalTo,
// Rule actions
actions: RuleActionArray,
// Rule exceptions
exceptions_list: ExceptionListArray,
// Misc attributes
author: RuleAuthorArray,
false_positives: RuleFalsePositiveArray,
references: RuleReferenceArray,
// maxSignals not used in ML rules but probably should be used
max_signals: MaxSignals,
threat: ThreatArray,
},
});
export type DurationMetric = t.TypeOf<typeof DurationMetric>;
export const DurationMetric = PositiveInteger;
export type RuleExecutionMetrics = t.TypeOf<typeof RuleExecutionMetrics>;
/**
@property total_search_duration_ms - "total time spent performing ES searches as measured by Kibana;
includes network latency and time spent serializing/deserializing request/response",
@property total_indexing_duration_ms - "total time spent indexing documents during current rule execution cycle",
@property total_enrichment_duration_ms - total time spent enriching documents during current rule execution cycle
@property execution_gap_duration_s - "duration in seconds of execution gap"
*/
export const RuleExecutionMetrics = t.partial({
total_search_duration_ms: DurationMetric,
total_indexing_duration_ms: DurationMetric,
total_enrichment_duration_ms: DurationMetric,
execution_gap_duration_s: DurationMetric,
});
export type BaseCreateProps = t.TypeOf<typeof BaseCreateProps>;
export const BaseCreateProps = baseSchema.create;
// -------------------------------------------------------------------------------------------------
// Shared schemas
// "Shared" types are the same across all rule types, and built from "baseSchema" above
// with some variations for each route. These intersect with type specific schemas below
// to create the full schema for each route.
export type SharedCreateProps = t.TypeOf<typeof SharedCreateProps>;
export const SharedCreateProps = t.intersection([
baseSchema.create,
t.exact(t.partial({ rule_id: RuleSignatureId })),
]);
// -------------------------------------------------------------------------------------------------
// EQL rule schema
export type KqlQueryLanguage = t.TypeOf<typeof KqlQueryLanguage>;
export const KqlQueryLanguage = t.keyof({ kuery: null, lucene: null });
export type EqlQueryLanguage = t.TypeOf<typeof EqlQueryLanguage>;
export const EqlQueryLanguage = t.literal('eql');
const eqlSchema = buildRuleSchemas({
required: {
type: t.literal('eql'),
language: EqlQueryLanguage,
query: RuleQuery,
},
optional: {
index: IndexPatternArray,
data_view_id: DataViewId,
filters: RuleFilterArray,
timestamp_field: TimestampField,
event_category_override: EventCategoryOverride,
tiebreaker_field: TiebreakerField,
},
defaultable: {},
});
// -------------------------------------------------------------------------------------------------
// ES|QL rule schema
export type EsqlQueryLanguage = t.TypeOf<typeof EsqlQueryLanguage>;
export const EsqlQueryLanguage = t.literal('esql');
const esqlSchema = buildRuleSchemas({
required: {
type: t.literal('esql'),
language: EsqlQueryLanguage,
query: RuleQuery,
},
optional: {},
defaultable: {},
});
// -------------------------------------------------------------------------------------------------
// Indicator Match rule schema
const threatMatchSchema = buildRuleSchemas({
required: {
type: t.literal('threat_match'),
query: RuleQuery,
threat_query,
threat_mapping,
threat_index,
},
optional: {
index: IndexPatternArray,
data_view_id: DataViewId,
filters: RuleFilterArray,
saved_id,
threat_filters,
threat_indicator_path,
threat_language: KqlQueryLanguage,
concurrent_searches,
items_per_search,
},
defaultable: {
language: KqlQueryLanguage,
},
});
// -------------------------------------------------------------------------------------------------
// Custom Query rule schema
const querySchema = buildRuleSchemas({
required: {
type: t.literal('query'),
},
optional: {
index: IndexPatternArray,
data_view_id: DataViewId,
filters: RuleFilterArray,
saved_id,
response_actions: ResponseActionArray,
alert_suppression: AlertSuppression,
},
defaultable: {
query: RuleQuery,
language: KqlQueryLanguage,
},
});
// -------------------------------------------------------------------------------------------------
// Saved Query rule schema
const savedQuerySchema = buildRuleSchemas({
required: {
type: t.literal('saved_query'),
saved_id,
},
optional: {
// Having language, query, and filters possibly defined adds more code confusion and probably user confusion
// if the saved object gets deleted for some reason
index: IndexPatternArray,
data_view_id: DataViewId,
query: RuleQuery,
filters: RuleFilterArray,
response_actions: ResponseActionArray,
alert_suppression: AlertSuppression,
},
defaultable: {
language: KqlQueryLanguage,
},
});
// -------------------------------------------------------------------------------------------------
// Threshold rule schema
const thresholdSchema = buildRuleSchemas({
required: {
type: t.literal('threshold'),
query: RuleQuery,
threshold: Threshold,
},
optional: {
index: IndexPatternArray,
data_view_id: DataViewId,
filters: RuleFilterArray,
saved_id,
},
defaultable: {
language: KqlQueryLanguage,
},
});
// -------------------------------------------------------------------------------------------------
// Machine Learning rule schema
const machineLearningSchema = buildRuleSchemas({
required: {
type: t.literal('machine_learning'),
anomaly_threshold,
machine_learning_job_id,
},
optional: {},
defaultable: {},
});
// -------------------------------------------------------------------------------------------------
// New Terms rule schema
const newTermsSchema = buildRuleSchemas({
required: {
type: t.literal('new_terms'),
query: RuleQuery,
new_terms_fields: NewTermsFields,
history_window_start: HistoryWindowStart,
},
optional: {
index: IndexPatternArray,
data_view_id: DataViewId,
filters: RuleFilterArray,
},
defaultable: {
language: KqlQueryLanguage,
},
});
// -------------------------------------------------------------------------------------------------
// Combined type specific schemas
export type TypeSpecificCreateProps = t.TypeOf<typeof TypeSpecificCreateProps>;
export const TypeSpecificCreateProps = t.union([
eqlSchema.create,
esqlSchema.create,
threatMatchSchema.create,
querySchema.create,
savedQuerySchema.create,
thresholdSchema.create,
machineLearningSchema.create,
newTermsSchema.create,
]);

62
x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema_legacy/threshold_attributes.ts
View file

@ -1,62 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as t from 'io-ts';
import { PositiveInteger, PositiveIntegerGreaterThanZero } from '@kbn/securitysolution-io-ts-types';
// Attributes specific to Threshold rules
const thresholdField = t.exact(
t.type({
field: t.union([t.string, t.array(t.string)]), // Covers pre- and post-7.12
value: PositiveIntegerGreaterThanZero,
})
);
const thresholdFieldNormalized = t.exact(
t.type({
field: t.array(t.string),
value: PositiveIntegerGreaterThanZero,
})
);
const thresholdCardinalityField = t.exact(
t.type({
field: t.string,
value: PositiveInteger,
})
);
export type Threshold = t.TypeOf<typeof Threshold>;
export const Threshold = t.intersection([
thresholdField,
t.exact(
t.partial({
cardinality: t.array(thresholdCardinalityField),
})
),
]);
export type ThresholdNormalized = t.TypeOf<typeof ThresholdNormalized>;
export const ThresholdNormalized = t.intersection([
thresholdFieldNormalized,
t.exact(
t.partial({
cardinality: t.array(thresholdCardinalityField),
})
),
]);
export type ThresholdWithCardinality = t.TypeOf<typeof ThresholdWithCardinality>;
export const ThresholdWithCardinality = t.intersection([
thresholdFieldNormalized,
t.exact(
t.type({
cardinality: t.array(thresholdCardinalityField),
})
),
]);

121
x-pack/plugins/security_solution/common/api/detection_engine/model/schemas.ts
View file

@ -7,89 +7,76 @@
/* eslint-disable @typescript-eslint/naming-convention */
import * as t from 'io-ts';
import { PositiveInteger } from '@kbn/securitysolution-io-ts-types';
import { z } from 'zod';
export const file_name = t.string;
export type FileName = t.TypeOf<typeof file_name>;
export type FileName = z.infer<typeof file_name>;
export const file_name = z.string();
export const exclude_export_details = t.boolean;
export type ExcludeExportDetails = t.TypeOf<typeof exclude_export_details>;
export type ExcludeExportDetails = z.infer<typeof exclude_export_details>;
export const exclude_export_details = z.boolean();
export const saved_id = t.string;
export const saved_id = z.string();
export const savedIdOrUndefined = t.union([saved_id, t.undefined]);
export type SavedIdOrUndefined = t.TypeOf<typeof savedIdOrUndefined>;
export type SavedIdOrUndefined = z.infer<typeof savedIdOrUndefined>;
export const savedIdOrUndefined = saved_id.optional();
export const anomaly_threshold = PositiveInteger;
export const status = z.enum(['open', 'closed', 'acknowledged', 'in-progress']);
export type Status = z.infer<typeof status>;
export const status = t.keyof({
open: null,
closed: null,
acknowledged: null,
'in-progress': null,
});
export type Status = t.TypeOf<typeof status>;
export const signal_ids = z.array(z.string());
export type SignalIds = z.infer<typeof signal_ids>;
export const conflicts = t.keyof({ abort: null, proceed: null });
export const alert_tag_ids = z.array(z.string());
export type AlertTagIds = z.infer<typeof alert_tag_ids>;
export const signal_ids = t.array(t.string);
export type SignalIds = t.TypeOf<typeof signal_ids>;
// TODO: Can this be more strict or is this is the set of all Elastic Queries?
export const signal_status_query = t.object;
export const alert_tag_ids = t.array(t.string);
export type AlertTagIds = t.TypeOf<typeof alert_tag_ids>;
export const indexRecord = t.record(
t.string,
t.type({
all: t.boolean,
maintenance: t.boolean,
read: t.boolean,
create_index: t.boolean,
index: t.boolean,
monitor: t.boolean,
delete: t.boolean,
manage: t.boolean,
delete_index: t.boolean,
create_doc: t.boolean,
view_index_metadata: t.boolean,
create: t.boolean,
write: t.boolean,
export const indexRecord = z.record(
z.string(),
z.object({
all: z.boolean(),
maintenance: z.boolean(),
read: z.boolean(),
create_index: z.boolean(),
index: z.boolean(),
monitor: z.boolean(),
delete: z.boolean(),
manage: z.boolean(),
delete_index: z.boolean(),
create_doc: z.boolean(),
view_index_metadata: z.boolean(),
create: z.boolean(),
write: z.boolean(),
})
);
export const privilege = t.type({
username: t.string,
has_all_requested: t.boolean,
cluster: t.type({
monitor_ml: t.boolean,
manage_index_templates: t.boolean,
monitor_transform: t.boolean,
manage_security: t.boolean,
manage_own_api_key: t.boolean,
all: t.boolean,
monitor: t.boolean,
manage: t.boolean,
manage_transform: t.boolean,
manage_ml: t.boolean,
manage_pipeline: t.boolean,
export const privilege = z.object({
username: z.string(),
has_all_requested: z.boolean(),
cluster: z.object({
monitor_ml: z.boolean(),
manage_index_templates: z.boolean(),
monitor_transform: z.boolean(),
manage_security: z.boolean(),
manage_own_api_key: z.boolean(),
all: z.boolean(),
monitor: z.boolean(),
manage: z.boolean(),
manage_transform: z.boolean(),
manage_ml: z.boolean(),
manage_pipeline: z.boolean(),
}),
index: indexRecord,
is_authenticated: t.boolean,
has_encryption_key: t.boolean,
is_authenticated: z.boolean(),
has_encryption_key: z.boolean(),
});
export type Privilege = t.TypeOf<typeof privilege>;
export type Privilege = z.infer<typeof privilege>;
export const alert_tags = t.type({
tags_to_add: t.array(t.string),
tags_to_remove: t.array(t.string),
export const alert_tags = z.object({
tags_to_add: z.array(z.string()),
tags_to_remove: z.array(z.string()),
});
export type AlertTags = t.TypeOf<typeof alert_tags>;
export type AlertTags = z.infer<typeof alert_tags>;
export const user_search_term = t.string;
export type UserSearchTerm = t.TypeOf<typeof user_search_term>;
export const user_search_term = z.string();
export type UserSearchTerm = z.infer<typeof user_search_term>;

1
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/index.ts
View file

@ -14,7 +14,6 @@ export * from './review_rule_installation/review_rule_installation_route';
export * from './review_rule_upgrade/review_rule_upgrade_route';
export * from './urls';
export * from './model/aggregated_prebuilt_rules_error';
export * from './model/diff/diffable_rule/build_schema';
export * from './model/diff/diffable_rule/diffable_field_types';
export * from './model/diff/diffable_rule/diffable_rule';
export * from './model/diff/rule_diff/fields_diff';

25
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/model/diff/diffable_rule/build_schema.ts
View file

@ -1,25 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as t from 'io-ts';
// TODO https://github.com/elastic/security-team/issues/7491
// eslint-disable-next-line no-restricted-imports
import { orUndefined } from '../../../../model/rule_schema_legacy';
interface RuleFields<TRequired extends t.Props, TOptional extends t.Props> {
required: TRequired;
optional: TOptional;
}
export const buildSchema = <TRequired extends t.Props, TOptional extends t.Props>(
fields: RuleFields<TRequired, TOptional>
) => {
return t.intersection([
t.exact(t.type(fields.required)),
t.exact(t.type(orUndefined(fields.optional))),
]);
};

159
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/model/diff/diffable_rule/diffable_field_types.ts
View file

@ -4,25 +4,23 @@
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { z } from 'zod';
import * as t from 'io-ts';
import { TimeDuration } from '@kbn/securitysolution-io-ts-types';
// TODO https://github.com/elastic/security-team/issues/7491
// eslint-disable-next-line no-restricted-imports
import {
BuildingBlockType,
DataViewId,
IndexPatternArray,
KqlQueryLanguage,
RuleFilterArray,
RuleNameOverride as RuleNameOverrideFieldName,
RuleNameOverride,
RuleQuery,
SavedQueryId,
TimelineTemplateId,
TimelineTemplateTitle,
TimestampOverride as TimestampOverrideFieldName,
TimestampOverride,
TimestampOverrideFallbackDisabled,
} from '../../../../model/rule_schema_legacy';
import { saved_id } from '../../../../model/schemas';
} from '../../../../model/rule_schema';
import { TimeDuration } from '../../../../model/rule_schema/time_duration';
// -------------------------------------------------------------------------------------------------
// Rule data source
@ -32,24 +30,23 @@ export enum DataSourceType {
'data_view' = 'data_view',
}
export type DataSourceIndexPatterns = t.TypeOf<typeof DataSourceIndexPatterns>;
export const DataSourceIndexPatterns = t.exact(
t.type({
type: t.literal(DataSourceType.index_patterns),
index_patterns: IndexPatternArray,
})
);
export type DataSourceIndexPatterns = z.infer<typeof DataSourceIndexPatterns>;
export const DataSourceIndexPatterns = z.object({
type: z.literal(DataSourceType.index_patterns),
index_patterns: IndexPatternArray,
});
export type DataSourceDataView = t.TypeOf<typeof DataSourceDataView>;
export const DataSourceDataView = t.exact(
t.type({
type: t.literal(DataSourceType.data_view),
data_view_id: DataViewId,
})
);
export type DataSourceDataView = z.infer<typeof DataSourceDataView>;
export const DataSourceDataView = z.object({
type: z.literal(DataSourceType.data_view),
data_view_id: DataViewId,
});
export type RuleDataSource = t.TypeOf<typeof RuleDataSource>;
export const RuleDataSource = t.union([DataSourceIndexPatterns, DataSourceDataView]);
export type RuleDataSource = z.infer<typeof RuleDataSource>;
export const RuleDataSource = z.discriminatedUnion('type', [
DataSourceIndexPatterns,
DataSourceDataView,
]);
// -------------------------------------------------------------------------------------------------
// Rule data query
@ -59,93 +56,75 @@ export enum KqlQueryType {
'saved_query' = 'saved_query',
}
export type InlineKqlQuery = t.TypeOf<typeof InlineKqlQuery>;
export const InlineKqlQuery = t.exact(
t.type({
type: t.literal(KqlQueryType.inline_query),
query: RuleQuery,
language: KqlQueryLanguage,
filters: RuleFilterArray,
})
);
export type InlineKqlQuery = z.infer<typeof InlineKqlQuery>;
export const InlineKqlQuery = z.object({
type: z.literal(KqlQueryType.inline_query),
query: RuleQuery,
language: KqlQueryLanguage,
filters: RuleFilterArray,
});
export type SavedKqlQuery = t.TypeOf<typeof SavedKqlQuery>;
export const SavedKqlQuery = t.exact(
t.type({
type: t.literal(KqlQueryType.saved_query),
saved_query_id: saved_id,
})
);
export type SavedKqlQuery = z.infer<typeof SavedKqlQuery>;
export const SavedKqlQuery = z.object({
type: z.literal(KqlQueryType.saved_query),
saved_query_id: SavedQueryId,
});
export type RuleKqlQuery = t.TypeOf<typeof RuleKqlQuery>;
export const RuleKqlQuery = t.union([InlineKqlQuery, SavedKqlQuery]);
export type RuleKqlQuery = z.infer<typeof RuleKqlQuery>;
export const RuleKqlQuery = z.discriminatedUnion('type', [InlineKqlQuery, SavedKqlQuery]);
export type RuleEqlQuery = t.TypeOf<typeof RuleEqlQuery>;
export const RuleEqlQuery = t.exact(
t.type({
query: RuleQuery,
language: t.literal('eql'),
filters: RuleFilterArray,
})
);
export type RuleEqlQuery = z.infer<typeof RuleEqlQuery>;
export const RuleEqlQuery = z.object({
query: RuleQuery,
language: z.literal('eql'),
filters: RuleFilterArray,
});
export type RuleEsqlQuery = t.TypeOf<typeof RuleEsqlQuery>;
export const RuleEsqlQuery = t.exact(
t.type({
query: RuleQuery,
language: t.literal('esql'),
})
);
export type RuleEsqlQuery = z.infer<typeof RuleEsqlQuery>;
export const RuleEsqlQuery = z.object({
query: RuleQuery,
language: z.literal('esql'),
});
// -------------------------------------------------------------------------------------------------
// Rule schedule
export type RuleSchedule = t.TypeOf<typeof RuleSchedule>;
export const RuleSchedule = t.exact(
t.type({
interval: TimeDuration({ allowedUnits: ['s', 'm', 'h'] }),
lookback: TimeDuration({ allowedUnits: ['s', 'm', 'h'] }),
})
);
export type RuleSchedule = z.infer<typeof RuleSchedule>;
export const RuleSchedule = z.object({
interval: TimeDuration({ allowedUnits: ['s', 'm', 'h'] }),
lookback: TimeDuration({ allowedUnits: ['s', 'm', 'h'] }),
});
// -------------------------------------------------------------------------------------------------
// Rule name override
export type RuleNameOverrideObject = t.TypeOf<typeof RuleNameOverrideObject>;
export const RuleNameOverrideObject = t.exact(
t.type({
field_name: RuleNameOverrideFieldName,
})
);
export type RuleNameOverrideObject = z.infer<typeof RuleNameOverrideObject>;
export const RuleNameOverrideObject = z.object({
field_name: RuleNameOverride,
});
// -------------------------------------------------------------------------------------------------
// Timestamp override
export type TimestampOverrideObject = t.TypeOf<typeof TimestampOverrideObject>;
export const TimestampOverrideObject = t.exact(
t.type({
field_name: TimestampOverrideFieldName,
fallback_disabled: TimestampOverrideFallbackDisabled,
})
);
export type TimestampOverrideObject = z.infer<typeof TimestampOverrideObject>;
export const TimestampOverrideObject = z.object({
field_name: TimestampOverride,
fallback_disabled: TimestampOverrideFallbackDisabled,
});
// -------------------------------------------------------------------------------------------------
// Reference to a timeline template
export type TimelineTemplateReference = t.TypeOf<typeof TimelineTemplateReference>;
export const TimelineTemplateReference = t.exact(
t.type({
timeline_id: TimelineTemplateId,
timeline_title: TimelineTemplateTitle,
})
);
export type TimelineTemplateReference = z.infer<typeof TimelineTemplateReference>;
export const TimelineTemplateReference = z.object({
timeline_id: TimelineTemplateId,
timeline_title: TimelineTemplateTitle,
});
// -------------------------------------------------------------------------------------------------
// Building block
export type BuildingBlockObject = t.TypeOf<typeof BuildingBlockObject>;
export const BuildingBlockObject = t.exact(
t.type({
type: BuildingBlockType,
})
);
export type BuildingBlockObject = z.infer<typeof BuildingBlockObject>;
export const BuildingBlockObject = z.object({
type: BuildingBlockType,
});

264
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/model/diff/diffable_rule/diffable_rule.ts
View file

@ -5,212 +5,164 @@
* 2.0.
*/
import * as t from 'io-ts';
import { z } from 'zod';
import {
concurrent_searches,
items_per_search,
machine_learning_job_id,
RiskScore,
RiskScoreMapping,
RuleActionArray,
RuleActionThrottle,
Severity,
SeverityMapping,
threat_index,
threat_indicator_path,
threat_mapping,
} from '@kbn/securitysolution-io-ts-alerting-types';
// TODO https://github.com/elastic/security-team/issues/7491
// eslint-disable-next-line no-restricted-imports
import {
AlertSuppression,
AnomalyThreshold,
ConcurrentSearches,
EventCategoryOverride,
ExceptionListArray,
HistoryWindowStart,
InvestigationGuide,
ItemsPerSearch,
MachineLearningJobId,
MaxSignals,
NewTermsFields,
RelatedIntegrationArray,
RequiredFieldArray,
RiskScore,
RiskScoreMapping,
RuleAuthorArray,
RuleDescription,
RuleExceptionList,
RuleFalsePositiveArray,
RuleLicense,
RuleMetadata,
RuleName,
RuleReferenceArray,
RuleSignatureId,
RuleTagArray,
RuleVersion,
SetupGuide,
Severity,
SeverityMapping,
ThreatArray,
ThreatIndex,
ThreatIndicatorPath,
ThreatMapping,
Threshold,
TiebreakerField,
TimestampField,
} from '../../../../model/rule_schema_legacy';
} from '../../../../model/rule_schema';
import {
BuildingBlockObject,
InlineKqlQuery,
RuleDataSource,
RuleEqlQuery,
RuleEsqlQuery,
InlineKqlQuery,
RuleKqlQuery,
RuleDataSource,
RuleNameOverrideObject,
RuleSchedule,
TimelineTemplateReference,
TimestampOverrideObject,
} from './diffable_field_types';
import { buildSchema } from './build_schema';
import { anomaly_threshold } from '../../../../model/schemas';
export type DiffableCommonFields = z.infer<typeof DiffableCommonFields>;
export const DiffableCommonFields = z.object({
// Technical fields
// NOTE: We might consider removing them from the schema and returning from the API
// not via the fields diff, but via dedicated properties in the response body.
rule_id: RuleSignatureId,
version: RuleVersion,
export type DiffableCommonFields = t.TypeOf<typeof DiffableCommonFields>;
export const DiffableCommonFields = buildSchema({
required: {
// Technical fields
// NOTE: We might consider removing them from the schema and returning from the API
// not via the fields diff, but via dedicated properties in the response body.
rule_id: RuleSignatureId,
version: RuleVersion,
meta: RuleMetadata,
// Main domain fields
name: RuleName,
tags: RuleTagArray,
description: RuleDescription,
severity: Severity,
severity_mapping: SeverityMapping,
risk_score: RiskScore,
risk_score_mapping: RiskScoreMapping,
// Main domain fields
name: RuleName,
tags: RuleTagArray,
description: RuleDescription,
severity: Severity,
severity_mapping: SeverityMapping,
risk_score: RiskScore,
risk_score_mapping: RiskScoreMapping,
// About -> Advanced settings
references: RuleReferenceArray,
false_positives: RuleFalsePositiveArray,
threat: ThreatArray,
note: InvestigationGuide,
setup: SetupGuide,
related_integrations: RelatedIntegrationArray,
required_fields: RequiredFieldArray,
author: RuleAuthorArray,
license: RuleLicense,
// About -> Advanced settings
references: RuleReferenceArray,
false_positives: RuleFalsePositiveArray,
threat: ThreatArray,
note: InvestigationGuide,
setup: SetupGuide,
related_integrations: RelatedIntegrationArray,
required_fields: RequiredFieldArray,
author: RuleAuthorArray,
license: RuleLicense,
// Other domain fields
rule_schedule: RuleSchedule, // NOTE: new field
exceptions_list: z.array(RuleExceptionList),
max_signals: MaxSignals,
// Other domain fields
rule_schedule: RuleSchedule, // NOTE: new field
actions: RuleActionArray,
throttle: RuleActionThrottle,
exceptions_list: ExceptionListArray,
max_signals: MaxSignals,
},
optional: {
rule_name_override: RuleNameOverrideObject, // NOTE: new field
timestamp_override: TimestampOverrideObject, // NOTE: new field
timeline_template: TimelineTemplateReference, // NOTE: new field
building_block: BuildingBlockObject, // NOTE: new field
},
// Optional fields
rule_name_override: RuleNameOverrideObject.optional(), // NOTE: new field
timestamp_override: TimestampOverrideObject.optional(), // NOTE: new field
timeline_template: TimelineTemplateReference.optional(), // NOTE: new field
building_block: BuildingBlockObject.optional(), // NOTE: new field
});
export type DiffableCustomQueryFields = t.TypeOf<typeof DiffableCustomQueryFields>;
export const DiffableCustomQueryFields = buildSchema({
required: {
type: t.literal('query'),
kql_query: RuleKqlQuery, // NOTE: new field
},
optional: {
data_source: RuleDataSource, // NOTE: new field
alert_suppression: AlertSuppression,
},
export type DiffableCustomQueryFields = z.infer<typeof DiffableCustomQueryFields>;
export const DiffableCustomQueryFields = z.object({
type: z.literal('query'),
kql_query: RuleKqlQuery, // NOTE: new field
data_source: RuleDataSource.optional(), // NOTE: new field
});
export type DiffableSavedQueryFields = t.TypeOf<typeof DiffableSavedQueryFields>;
export const DiffableSavedQueryFields = buildSchema({
required: {
type: t.literal('saved_query'),
kql_query: RuleKqlQuery, // NOTE: new field
},
optional: {
data_source: RuleDataSource, // NOTE: new field
alert_suppression: AlertSuppression,
},
export type DiffableSavedQueryFields = z.infer<typeof DiffableSavedQueryFields>;
export const DiffableSavedQueryFields = z.object({
type: z.literal('saved_query'),
kql_query: RuleKqlQuery, // NOTE: new field
data_source: RuleDataSource.optional(), // NOTE: new field
});
export type DiffableEqlFields = t.TypeOf<typeof DiffableEqlFields>;
export const DiffableEqlFields = buildSchema({
required: {
type: t.literal('eql'),
eql_query: RuleEqlQuery, // NOTE: new field
},
optional: {
data_source: RuleDataSource, // NOTE: new field
event_category_override: EventCategoryOverride,
timestamp_field: TimestampField,
tiebreaker_field: TiebreakerField,
},
export type DiffableEqlFields = z.infer<typeof DiffableEqlFields>;
export const DiffableEqlFields = z.object({
type: z.literal('eql'),
eql_query: RuleEqlQuery, // NOTE: new field
data_source: RuleDataSource.optional(), // NOTE: new field
event_category_override: EventCategoryOverride.optional(),
timestamp_field: TimestampField.optional(),
tiebreaker_field: TiebreakerField.optional(),
});
export type DiffableEsqlFields = t.TypeOf<typeof DiffableEsqlFields>;
export const DiffableEsqlFields = buildSchema({
required: {
type: t.literal('esql'),
esql_query: RuleEsqlQuery, // NOTE: new field
},
// this is a new type of rule, no prebuilt rules created yet.
// new properties might be added here during further rule type development
optional: {},
// this is a new type of rule, no prebuilt rules created yet.
// new properties might be added here during further rule type development
export type DiffableEsqlFields = z.infer<typeof DiffableEsqlFields>;
export const DiffableEsqlFields = z.object({
type: z.literal('esql'),
esql_query: RuleEsqlQuery, // NOTE: new field
});
export type DiffableThreatMatchFields = t.TypeOf<typeof DiffableThreatMatchFields>;
export const DiffableThreatMatchFields = buildSchema({
required: {
type: t.literal('threat_match'),
kql_query: RuleKqlQuery, // NOTE: new field
threat_query: InlineKqlQuery, // NOTE: new field
threat_index,
threat_mapping,
},
optional: {
data_source: RuleDataSource, // NOTE: new field
threat_indicator_path,
concurrent_searches, // Should combine concurrent_searches and items_per_search?
items_per_search,
},
export type DiffableThreatMatchFields = z.infer<typeof DiffableThreatMatchFields>;
export const DiffableThreatMatchFields = z.object({
type: z.literal('threat_match'),
kql_query: RuleKqlQuery, // NOTE: new field
threat_query: InlineKqlQuery, // NOTE: new field
threat_index: ThreatIndex,
threat_mapping: ThreatMapping,
data_source: RuleDataSource.optional(), // NOTE: new field
threat_indicator_path: ThreatIndicatorPath.optional(),
concurrent_searches: ConcurrentSearches.optional(),
items_per_search: ItemsPerSearch.optional(),
});
export type DiffableThresholdFields = t.TypeOf<typeof DiffableThresholdFields>;
export const DiffableThresholdFields = buildSchema({
required: {
type: t.literal('threshold'),
kql_query: RuleKqlQuery, // NOTE: new field
threshold: Threshold,
},
optional: {
data_source: RuleDataSource, // NOTE: new field
},
export type DiffableThresholdFields = z.infer<typeof DiffableThresholdFields>;
export const DiffableThresholdFields = z.object({
type: z.literal('threshold'),
kql_query: RuleKqlQuery, // NOTE: new field
threshold: Threshold,
data_source: RuleDataSource.optional(), // NOTE: new field
});
export type DiffableMachineLearningFields = t.TypeOf<typeof DiffableMachineLearningFields>;
export const DiffableMachineLearningFields = buildSchema({
required: {
type: t.literal('machine_learning'),
machine_learning_job_id,
anomaly_threshold,
},
optional: {},
export type DiffableMachineLearningFields = z.infer<typeof DiffableMachineLearningFields>;
export const DiffableMachineLearningFields = z.object({
type: z.literal('machine_learning'),
machine_learning_job_id: MachineLearningJobId,
anomaly_threshold: AnomalyThreshold,
});
export type DiffableNewTermsFields = t.TypeOf<typeof DiffableNewTermsFields>;
export const DiffableNewTermsFields = buildSchema({
required: {
type: t.literal('new_terms'),
kql_query: InlineKqlQuery, // NOTE: new field
new_terms_fields: NewTermsFields,
history_window_start: HistoryWindowStart,
},
optional: {
data_source: RuleDataSource, // NOTE: new field
},
export type DiffableNewTermsFields = z.infer<typeof DiffableNewTermsFields>;
export const DiffableNewTermsFields = z.object({
type: z.literal('new_terms'),
kql_query: InlineKqlQuery, // NOTE: new field
new_terms_fields: NewTermsFields,
history_window_start: HistoryWindowStart,
data_source: RuleDataSource.optional(), // NOTE: new field
});
/**
@ -240,10 +192,10 @@ export const DiffableNewTermsFields = buildSchema({
* top-level fields.
*/
export type DiffableRule = t.TypeOf<typeof DiffableRule>;
export const DiffableRule = t.intersection([
export type DiffableRule = z.infer<typeof DiffableRule>;
const DiffableRule = z.intersection(
DiffableCommonFields,
t.union([
z.discriminatedUnion('type', [
DiffableCustomQueryFields,
DiffableSavedQueryFields,
DiffableEqlFields,
@ -252,8 +204,8 @@ export const DiffableRule = t.intersection([
DiffableThresholdFields,
DiffableMachineLearningFields,
DiffableNewTermsFields,
]),
]);
])
);
/**
* This is a merge of all fields from all rule types into a single TS type.

8
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/model/diff/rule_diff/fields_diff.ts
View file

@ -7,10 +7,10 @@
import type { ThreeWayDiff, ThreeWayDiffAlgorithm } from '../three_way_diff/three_way_diff';
export type FieldsDiff<TObject> = {
export type FieldsDiff<TObject> = Required<{
[Field in keyof TObject]: ThreeWayDiff<TObject[Field]>;
};
}>;
export type FieldsDiffAlgorithmsFor<TObject> = {
export type FieldsDiffAlgorithmsFor<TObject> = Required<{
[Field in keyof TObject]: ThreeWayDiffAlgorithm<TObject[Field]>;
};
}>;

1
x-pack/plugins/security_solution/common/api/detection_engine/prebuilt_rules/model/index.ts
View file

@ -6,7 +6,6 @@
*/
export * from './aggregated_prebuilt_rules_error';
export * from './diff/diffable_rule/build_schema';
export * from './diff/diffable_rule/diffable_field_types';
export * from './diff/diffable_rule/diffable_rule';
export * from './diff/rule_diff/fields_diff';

1
x-pack/plugins/security_solution/public/detection_engine/rule_management/components/rule_details/constants.ts
View file

@ -51,7 +51,6 @@ export const DEFINITION_UPGRADE_FIELD_ORDER: Array<keyof DiffableAllFields> = [
'threat_indicator_path',
'concurrent_searches',
'items_per_search',
'alert_suppression',
'new_terms_fields',
'history_window_start',
'max_signals',

13
x-pack/plugins/security_solution/public/detection_engine/rule_management/components/rule_details/per_field_diff/get_field_diffs_for_grouped_fields.ts
View file

@ -8,6 +8,9 @@
import stringify from 'json-stable-stringify';
import type {
AllFieldsDiff,
RuleFieldsDiffWithDataSource,
RuleFieldsDiffWithEqlQuery,
RuleFieldsDiffWithEsqlQuery,
RuleFieldsDiffWithKqlQuery,
} from '../../../../../../common/api/detection_engine';
import type { FieldDiff } from '../../../model/rule_details/rule_field_diff';
@ -24,7 +27,7 @@ export const sortAndStringifyJson = (fieldValue: unknown): string => {
};
export const getFieldDiffsForDataSource = (
dataSourceThreeWayDiff: AllFieldsDiff['data_source']
dataSourceThreeWayDiff: RuleFieldsDiffWithDataSource['data_source']
): FieldDiff[] => {
const currentType = sortAndStringifyJson(dataSourceThreeWayDiff.current_version?.type);
const targetType = sortAndStringifyJson(dataSourceThreeWayDiff.target_version?.type);
@ -171,7 +174,9 @@ export const getFieldDiffsForKqlQuery = (
];
};
export const getFieldDiffsForEqlQuery = (eqlQuery: AllFieldsDiff['eql_query']): FieldDiff[] => {
export const getFieldDiffsForEqlQuery = (
eqlQuery: RuleFieldsDiffWithEqlQuery['eql_query']
): FieldDiff[] => {
const currentQuery = sortAndStringifyJson(eqlQuery.current_version?.query);
const targetQuery = sortAndStringifyJson(eqlQuery.target_version?.query);
@ -199,7 +204,9 @@ export const getFieldDiffsForEqlQuery = (eqlQuery: AllFieldsDiff['eql_query']):
];
};
export const getFieldDiffsForEsqlQuery = (esqlQuery: AllFieldsDiff['esql_query']): FieldDiff[] => {
export const getFieldDiffsForEsqlQuery = (
esqlQuery: RuleFieldsDiffWithEsqlQuery['esql_query']
): FieldDiff[] => {
const currentQuery = sortAndStringifyJson(esqlQuery.current_version?.query);
const targetQuery = sortAndStringifyJson(esqlQuery.target_version?.query);

26
x-pack/plugins/security_solution/public/detection_engine/rule_management/components/rule_details/rule_diff_tab.tsx
View file

@ -28,7 +28,7 @@ import * as i18n from './json_diff/translations';
import { getHumanizedDuration } from '../../../../detections/pages/detection_engine/rules/helpers';
/* Inclding these properties in diff display might be confusing to users. */
const HIDDEN_PROPERTIES = [
const HIDDEN_PROPERTIES: Array<keyof RuleResponse> = [
/*
By default, prebuilt rules don't have any actions or exception lists. So if a user has defined actions or exception lists for a rule, it'll show up as diff. This looks confusing as the user might think that their actions and exceptions lists will get removed after the upgrade, which is not the case - they will be preserved.
*/
@ -44,13 +44,13 @@ const HIDDEN_PROPERTIES = [
'revision',
/*
"updated_at" value is regenerated on every '/upgrade/_review' endpoint run
"updated_at" value is regenerated on every '/upgrade/_review' endpoint run
and will therefore always show a diff. It adds no value to display it to the user.
*/
'updated_at',
/*
These values make sense only for installed prebuilt rules.
These values make sense only for installed prebuilt rules.
They are not present in the prebuilt rule package.
So, showing them in the diff doesn't add value.
*/
@ -76,11 +76,11 @@ const normalizeRule = (originalRule: RuleResponse): RuleResponse => {
const rule = { ...originalRule };
/*
Convert the "from" property value to a humanized duration string, like 'now-1m' or 'now-2h'.
Conversion is needed to skip showing the diff for the "from" property when the same
duration is represented in different time units. For instance, 'now-1h' and 'now-3600s'
Convert the "from" property value to a humanized duration string, like 'now-1m' or 'now-2h'.
Conversion is needed to skip showing the diff for the "from" property when the same
duration is represented in different time units. For instance, 'now-1h' and 'now-3600s'
indicate a one-hour duration.
The same helper is used in the rule editing UI to format "from" before submitting the edits.
The same helper is used in the rule editing UI to format "from" before submitting the edits.
So, after the rule is saved, the "from" property unit/value might differ from what's in the package.
*/
rule.from = formatScheduleStepData({
@ -91,8 +91,8 @@ const normalizeRule = (originalRule: RuleResponse): RuleResponse => {
/*
Default "note" to an empty string if it's not present.
Sometimes, in a new version of a rule, the "note" value equals an empty string, while
in the old version, it wasn't specified at all (undefined becomes ''). In this case,
Sometimes, in a new version of a rule, the "note" value equals an empty string, while
in the old version, it wasn't specified at all (undefined becomes ''). In this case,
it doesn't make sense to show diff, so we default falsy values to ''.
*/
rule.note = rule.note ?? '';
@ -104,9 +104,9 @@ const normalizeRule = (originalRule: RuleResponse): RuleResponse => {
rule.threat = filterEmptyThreats(rule.threat);
/*
The "machine_learning_job_id" property is converted from the legacy string format
to the new array format during installation and upgrade. Thus, all installed rules
use the new format. For correct comparison, we must ensure that the rule update is
The "machine_learning_job_id" property is converted from the legacy string format
to the new array format during installation and upgrade. Thus, all installed rules
use the new format. For correct comparison, we must ensure that the rule update is
also in the new format before showing the diff.
*/
if ('machine_learning_job_id' in rule) {
@ -115,7 +115,7 @@ const normalizeRule = (originalRule: RuleResponse): RuleResponse => {
/*
Default the "alias" property to null for all threat filters that don't have it.
Setting a default is needed to match the behavior of the rule editing UI,
Setting a default is needed to match the behavior of the rule editing UI,
which also defaults the "alias" property to null.
*/
if (rule.type === 'threat_match' && Array.isArray(rule.threat_filters)) {

5
x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/logic/diff/calculation/calculate_rule_fields_diff.ts
View file

@ -173,7 +173,6 @@ const calculateCommonFieldsDiff = (
const commonFieldsDiffAlgorithms: FieldsDiffAlgorithmsFor<DiffableCommonFields> = {
rule_id: simpleDiffAlgorithm,
version: numberDiffAlgorithm,
meta: simpleDiffAlgorithm,
name: singleLineStringDiffAlgorithm,
tags: simpleDiffAlgorithm,
description: simpleDiffAlgorithm,
@ -191,8 +190,6 @@ const commonFieldsDiffAlgorithms: FieldsDiffAlgorithmsFor<DiffableCommonFields>
author: simpleDiffAlgorithm,
license: singleLineStringDiffAlgorithm,
rule_schedule: simpleDiffAlgorithm,
actions: simpleDiffAlgorithm,
throttle: simpleDiffAlgorithm,
exceptions_list: simpleDiffAlgorithm,
max_signals: numberDiffAlgorithm,
rule_name_override: simpleDiffAlgorithm,
@ -211,7 +208,6 @@ const customQueryFieldsDiffAlgorithms: FieldsDiffAlgorithmsFor<DiffableCustomQue
type: simpleDiffAlgorithm,
kql_query: simpleDiffAlgorithm,
data_source: simpleDiffAlgorithm,
alert_suppression: simpleDiffAlgorithm,
};
const calculateSavedQueryFieldsDiff = (
@ -224,7 +220,6 @@ const savedQueryFieldsDiffAlgorithms: FieldsDiffAlgorithmsFor<DiffableSavedQuery
type: simpleDiffAlgorithm,
kql_query: simpleDiffAlgorithm,
data_source: simpleDiffAlgorithm,
alert_suppression: simpleDiffAlgorithm,
};
const calculateEqlFieldsDiff = (

25
x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/logic/diff/normalization/convert_rule_to_diffable.ts
View file

@ -5,7 +5,7 @@
* 2.0.
*/
import type { RuleActionArray } from '@kbn/securitysolution-io-ts-alerting-types';
import type { RequiredOptional } from '@kbn/zod-helpers';
import { requiredOptional } from '@kbn/zod-helpers';
import { DEFAULT_MAX_SIGNALS } from '../../../../../../../common/constants';
import { assertUnreachable } from '../../../../../../../common/utility_types';
@ -110,13 +110,12 @@ export const convertRuleToDiffable = (rule: RuleResponse | PrebuiltRuleAsset): D
const extractDiffableCommonFields = (
rule: RuleResponse | PrebuiltRuleAsset
): DiffableCommonFields => {
): RequiredOptional<DiffableCommonFields> => {
return {
// --------------------- REQUIRED FIELDS
// Technical fields
rule_id: rule.rule_id,
version: rule.version,
meta: rule.meta ?? {},
// Main domain fields
name: rule.name,
@ -140,8 +139,6 @@ const extractDiffableCommonFields = (
// Other domain fields
rule_schedule: extractRuleSchedule(rule),
actions: (rule.actions ?? []) as RuleActionArray,
throttle: rule.throttle ?? 'no_actions',
exceptions_list: rule.exceptions_list ?? [],
max_signals: rule.max_signals ?? DEFAULT_MAX_SIGNALS,
@ -155,29 +152,27 @@ const extractDiffableCommonFields = (
const extractDiffableCustomQueryFields = (
rule: QueryRule | QueryRuleCreateProps
): DiffableCustomQueryFields => {
): RequiredOptional<DiffableCustomQueryFields> => {
return {
type: rule.type,
kql_query: extractRuleKqlQuery(rule.query, rule.language, rule.filters, rule.saved_id),
data_source: extractRuleDataSource(rule.index, rule.data_view_id),
alert_suppression: rule.alert_suppression,
};
};
const extractDiffableSavedQueryFieldsFromRuleObject = (
rule: SavedQueryRule | SavedQueryRuleCreateProps
): DiffableSavedQueryFields => {
): RequiredOptional<DiffableSavedQueryFields> => {
return {
type: rule.type,
kql_query: extractRuleKqlQuery(rule.query, rule.language, rule.filters, rule.saved_id),
data_source: extractRuleDataSource(rule.index, rule.data_view_id),
alert_suppression: rule.alert_suppression,
};
};
const extractDiffableEqlFieldsFromRuleObject = (
rule: EqlRule | EqlRuleCreateProps
): DiffableEqlFields => {
): RequiredOptional<DiffableEqlFields> => {
return {
type: rule.type,
eql_query: extractRuleEqlQuery(rule.query, rule.language, rule.filters),
@ -190,7 +185,7 @@ const extractDiffableEqlFieldsFromRuleObject = (
const extractDiffableEsqlFieldsFromRuleObject = (
rule: EsqlRule | EsqlRuleCreateProps
): DiffableEsqlFields => {
): RequiredOptional<DiffableEsqlFields> => {
return {
type: rule.type,
esql_query: extractRuleEsqlQuery(rule.query, rule.language),
@ -199,7 +194,7 @@ const extractDiffableEsqlFieldsFromRuleObject = (
const extractDiffableThreatMatchFieldsFromRuleObject = (
rule: ThreatMatchRule | ThreatMatchRuleCreateProps
): DiffableThreatMatchFields => {
): RequiredOptional<DiffableThreatMatchFields> => {
return {
type: rule.type,
kql_query: extractRuleKqlQuery(rule.query, rule.language, rule.filters, rule.saved_id),
@ -219,7 +214,7 @@ const extractDiffableThreatMatchFieldsFromRuleObject = (
const extractDiffableThresholdFieldsFromRuleObject = (
rule: ThresholdRule | ThresholdRuleCreateProps
): DiffableThresholdFields => {
): RequiredOptional<DiffableThresholdFields> => {
return {
type: rule.type,
kql_query: extractRuleKqlQuery(rule.query, rule.language, rule.filters, rule.saved_id),
@ -230,7 +225,7 @@ const extractDiffableThresholdFieldsFromRuleObject = (
const extractDiffableMachineLearningFieldsFromRuleObject = (
rule: MachineLearningRule | MachineLearningRuleCreateProps
): DiffableMachineLearningFields => {
): RequiredOptional<DiffableMachineLearningFields> => {
return {
type: rule.type,
machine_learning_job_id: rule.machine_learning_job_id,
@ -240,7 +235,7 @@ const extractDiffableMachineLearningFieldsFromRuleObject = (
const extractDiffableNewTermsFieldsFromRuleObject = (
rule: NewTermsRule | NewTermsRuleCreateProps
): DiffableNewTermsFields => {
): RequiredOptional<DiffableNewTermsFields> => {
return {
type: rule.type,
kql_query: extractInlineKqlQuery(rule.query, rule.language, rule.filters),

8
x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/logic/diff/normalization/extract_rule_schedule.ts
View file

@ -9,7 +9,10 @@ import moment from 'moment';
import dateMath from '@elastic/datemath';
import { parseDuration } from '@kbn/alerting-plugin/common';
import type { RuleResponse } from '../../../../../../../common/api/detection_engine/model/rule_schema';
import type {
RuleMetadata,
RuleResponse,
} from '../../../../../../../common/api/detection_engine/model/rule_schema';
import type { RuleSchedule } from '../../../../../../../common/api/detection_engine/prebuilt_rules';
import type { PrebuiltRuleAsset } from '../../../model/rule_assets/prebuilt_rule_asset';
@ -18,8 +21,7 @@ export const extractRuleSchedule = (rule: RuleResponse | PrebuiltRuleAsset): Rul
const from = rule.from ?? 'now-6m';
const to = rule.to ?? 'now';
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const ruleMeta = (rule.meta ?? {}) as any;
const ruleMeta: RuleMetadata = ('meta' in rule ? rule.meta : undefined) ?? {};
const lookbackFromMeta = String(ruleMeta.from ?? '');
const intervalDuration = parseInterval(interval);

129
x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/model/rule_assets/prebuilt_rule_asset.test.ts
View file

@ -32,6 +32,33 @@ describe('Prebuilt rule asset schema', () => {
expect(result.data).toEqual(getPrebuiltRuleMock());
});
describe('ommited fields from the rule schema are ignored', () => {
// The PrebuiltRuleAsset schema is built out of the rule schema,
// but the following fields are manually omitted.
// See: detection_engine/prebuilt_rules/model/rule_assets/prebuilt_rule_asset.ts
const omittedFields = [
'actions',
'throttle',
'meta',
'output_index',
'namespace',
'alias_purpose',
'alias_target_id',
'outcome',
];
test.each(omittedFields)('ignores %s since it`s an omitted field', (field) => {
const payload: Partial<PrebuiltRuleAsset> & Record<string, unknown> = {
...getPrebuiltRuleMock(),
[field]: 'some value',
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseSuccess(result);
expect(result.data).toEqual(getPrebuiltRuleMock());
});
});
test('[rule_id] does not validate', () => {
const payload: Partial<PrebuiltRuleAsset> = {
rule_id: 'rule-1',
@ -64,17 +91,6 @@ describe('Prebuilt rule asset schema', () => {
expect(result.data).toEqual(payload);
});
test('You can send in a namespace', () => {
const payload: PrebuiltRuleAsset = {
...getPrebuiltRuleMock(),
namespace: 'a namespace',
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseSuccess(result);
expect(result.data).toEqual(payload);
});
test('You can send in an empty array to threat', () => {
const payload: PrebuiltRuleAsset = {
...getPrebuiltRuleMock(),
@ -449,32 +465,6 @@ describe('Prebuilt rule asset schema', () => {
expect(result.data).toEqual(payload);
});
test('You can set meta to any object you want', () => {
const payload: PrebuiltRuleAsset = {
...getPrebuiltRuleMock(),
meta: {
somethingMadeUp: { somethingElse: true },
},
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseSuccess(result);
expect(result.data).toEqual(payload);
});
test('You cannot create meta as a string', () => {
const payload: Omit<PrebuiltRuleAsset, 'meta'> & { meta: string } = {
...getPrebuiltRuleMock(),
meta: 'should not work',
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"meta: Expected object, received string"`
);
});
test('validates with timeline_id and timeline_title', () => {
const payload: PrebuiltRuleAsset = {
...getPrebuiltRuleMock(),
@ -500,71 +490,6 @@ describe('Prebuilt rule asset schema', () => {
);
});
test('You cannot send in an array of actions that are missing "group"', () => {
const payload: Omit<PrebuiltRuleAsset['actions'], 'group'> = {
...getPrebuiltRuleMock(),
actions: [{ id: 'id', action_type_id: 'action_type_id', params: {} }],
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(`"actions.0.group: Required"`);
});
test('You cannot send in an array of actions that are missing "id"', () => {
const payload: Omit<PrebuiltRuleAsset['actions'], 'id'> = {
...getPrebuiltRuleMock(),
actions: [{ group: 'group', action_type_id: 'action_type_id', params: {} }],
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(`"actions.0.id: Required"`);
});
test('You cannot send in an array of actions that are missing "action_type_id"', () => {
const payload: Omit<PrebuiltRuleAsset['actions'], 'action_type_id'> = {
...getPrebuiltRuleMock(),
actions: [{ group: 'group', id: 'id', params: {} }],
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"actions.0.action_type_id: Required"`
);
});
test('You cannot send in an array of actions that are missing "params"', () => {
const payload: Omit<PrebuiltRuleAsset['actions'], 'params'> = {
...getPrebuiltRuleMock(),
actions: [{ group: 'group', id: 'id', action_type_id: 'action_type_id' }],
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(`"actions.0.params: Required"`);
});
test('You cannot send in an array of actions that are including "actionTypeId"', () => {
const payload: Omit<PrebuiltRuleAsset['actions'], 'actions'> = {
...getPrebuiltRuleMock(),
actions: [
{
group: 'group',
id: 'id',
actionTypeId: 'actionTypeId',
params: {},
},
],
};
const result = PrebuiltRuleAsset.safeParse(payload);
expectParseError(result);
expect(stringifyZodError(result.error)).toMatchInlineSnapshot(
`"actions.0.action_type_id: Required"`
);
});
describe('note', () => {
test('You can set note to a string', () => {
const payload: PrebuiltRuleAsset = {

76
x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/model/rule_assets/prebuilt_rule_asset.ts
View file

@ -10,9 +10,64 @@ import {
RuleSignatureId,
RuleVersion,
BaseCreateProps,
TypeSpecificCreateProps,
EqlRuleCreateFields,
EsqlRuleCreateFields,
MachineLearningRuleCreateFields,
NewTermsRuleCreateFields,
QueryRuleCreateFields,
SavedQueryRuleCreateFields,
ThreatMatchRuleCreateFields,
ThresholdRuleCreateFields,
} from '../../../../../../common/api/detection_engine/model/rule_schema';
/**
* The PrebuiltRuleAsset schema is created based on the rule schema defined in our OpenAPI specs.
* However, we don't need all the rule schema fields to be present in the PrebuiltRuleAsset.
* We omit some of them because they are not present in https://github.com/elastic/detection-rules.
* Context: https://github.com/elastic/kibana/issues/180393
*/
const BASE_PROPS_REMOVED_FROM_PREBUILT_RULE_ASSET = zodMaskFor<BaseCreateProps>()([
'actions',
'throttle',
'meta',
'output_index',
'namespace',
'alias_purpose',
'alias_target_id',
'outcome',
]);
// `response_actions` is only part of the optional fields in QueryRuleCreateFields and SavedQueryRuleCreateFields
const TYPE_SPECIFIC_PROPS_REMOVED_FROM_PREBUILT_RULE_ASSET = zodMaskFor<
QueryRuleCreateFields | SavedQueryRuleCreateFields
>()(['response_actions']);
const QueryRuleAssetFields = QueryRuleCreateFields.omit(
TYPE_SPECIFIC_PROPS_REMOVED_FROM_PREBUILT_RULE_ASSET
);
const SavedQueryRuleAssetFields = SavedQueryRuleCreateFields.omit(
TYPE_SPECIFIC_PROPS_REMOVED_FROM_PREBUILT_RULE_ASSET
);
export const RuleAssetTypeSpecificCreateProps = z.discriminatedUnion('type', [
EqlRuleCreateFields,
QueryRuleAssetFields,
SavedQueryRuleAssetFields,
ThresholdRuleCreateFields,
ThreatMatchRuleCreateFields,
MachineLearningRuleCreateFields,
NewTermsRuleCreateFields,
EsqlRuleCreateFields,
]);
function zodMaskFor<T>() {
return function <U extends keyof T>(props: U[]): Record<U, true> {
type PropObject = Record<string, boolean>;
const propObjects: PropObject[] = props.map((p: U) => ({ [p]: true }));
return Object.assign({}, ...propObjects);
};
}
/**
* Asset containing source content of a prebuilt Security detection rule.
* Is defined for each prebuilt rule in https://github.com/elastic/detection-rules.
@ -24,13 +79,16 @@ import {
* - Data Exfiltration Detection
*
* Big differences between this schema and RuleCreateProps:
* - rule_id is required here
* - version is a required field that must exist
* - rule_id is a required field
* - version is a required field
* - some fields are omitted because they are not present in https://github.com/elastic/detection-rules
*/
export type PrebuiltRuleAsset = z.infer<typeof PrebuiltRuleAsset>;
export const PrebuiltRuleAsset = BaseCreateProps.and(TypeSpecificCreateProps).and(
z.object({
rule_id: RuleSignatureId,
version: RuleVersion,
})
);
export const PrebuiltRuleAsset = BaseCreateProps.omit(BASE_PROPS_REMOVED_FROM_PREBUILT_RULE_ASSET)
.and(RuleAssetTypeSpecificCreateProps)
.and(
z.object({
rule_id: RuleSignatureId,
version: RuleVersion,
})
);