github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 09:48:58 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Entity Analytics][Risk Engine] Risk Scoring Task (#163216)

Browse source 
## What this PR does
* Adds a new Task Manager task, `risk_engine:risk_scoring`, responsible
for invoking the `calculateAndPersistRiskScores` API defined in the risk
scoring service.
* Unlike an alerting task, we do not encrypt/persist an API key for the
user. Instead, we use the internal kibana user to query all alerts in
the current space.
* The task configuration is stored as part of the existing
`risk-engine-configuration` Saved Object
* Extends the `risk-engine-configuration` SO to include more
configuration fields
* Management of this configuration is not currently exposed to the user.
They can only enable/disable the entire "Risk Engine" on the `Settings
-> Entity Risk Score` page
* The settings currently serve mainly as the "default" values for task
execution, but also as a way for a customer/SA to modify task execution
if necessary.
* We expect to be modifying these default values before release, as part
of our planned "tuning" stage.

### How to Review
* Setup:
* The risk engine acts on Detection engine alerts, and so you will need
to create:
      1. some "source" data (logs, filebeat, auditbeat, etc)
2. Rules looking for the above "source" data, and generating alerts
* The risk engine requires two feature flags, currently:
`riskScoringPersistence` and `riskScoringRoutesEnabled`
  * You will also need a Platinum or greater license.
1. Test that the task executes correctly
1. With the above data set up, navigate to `Settings -> Entity Risk
Score` page, and enable the task by toggling `Entity risk scoring` to
`On`
1. Within a few minutes, risk scores should be written to the risk score
datastream:
        * `GET risk-score.risk-score-default/_search`
* Replace `default` with the name of your current space, as necessary.
1. Disabling/re-enabling the risk engine should trigger another
execution of the task (similar to disabling/enabling a DE rule)
1. Enable the risk engine in another space
    * The engine (and task) can be enabled/executed in any kibana space.
* Because the engine only acts upon alerts in the current space, you
will need to first ensure alerts exist in that space.
1. Validate the data/mappings of persisted risk scores
* Scores are based on the Stage 1 [ECS
RFC](https://github.com/elastic/ecs/pull/2236)
* There is no UI reading from these scores, currently (but that is
introduced in https://github.com/elastic/kibana/pull/163237)
  
  

### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Ryland Herrick 2023-08-24 16:51:57 -05:00 • committed by GitHub
parent 6c2cd6048d
commit 43b0fab35c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
48 changed files with 2065 additions and 162 deletions
Download patch file Download diff file Expand all files Collapse all files

26
packages/kbn-check-mappings-update-cli/current_mappings.json
View file

@ -2956,8 +2956,34 @@
"risk-engine-configuration": {
"dynamic": false,
"properties": {
"dataViewId": {
"type": "keyword"
},
"enabled": {
"type": "boolean"
},
"filter": {
"dynamic": false,
"properties": {}
},
"identifierType": {
"type": "keyword"
},
"interval": {
"type": "keyword"
},
"pageSize": {
"type": "integer"
},
"range": {
"properties": {
"start": {
"type": "keyword"
},
"end": {
"type": "keyword"
}
}
}
}
},

2
src/core/server/integration_tests/saved_objects/migrations/group2/check_registered_types.test.ts
View file

@ -126,7 +126,7 @@ describe('checking migration metadata changes on all registered SO types', () =>
"osquery-pack-asset": "b14101d3172c4b60eb5404696881ce5275c84152",
"osquery-saved-query": "44f1161e165defe3f9b6ad643c68c542a765fcdb",
"query": "21cbbaa09abb679078145ce90087b1e88b7eae95",
"risk-engine-configuration": "1b8b175e29ea5311408125c92c6247f502b2d79d",
"risk-engine-configuration": "b105d4a3c6adce40708d729d12e5ef3c8fbd9508",
"rules-settings": "892a2918ebaeba809a612b8d97cec0b07c800b5f",
"sample-data-telemetry": "37441b12f5b0159c2d6d5138a494c9f440e950b5",
"search": "8d5184dd5b986d57250b6ffd9ae48a1925e4c7a3",

1
x-pack/plugins/security_solution/common/risk_engine/index.ts
View file

@ -8,6 +8,7 @@
export * from './after_keys';
export * from './risk_weights';
export * from './identifier_types';
export * from './range';
export * from './types';
export * from './indices';
export * from './constants';

15
x-pack/plugins/security_solution/common/risk_engine/range.ts Normal file
View file

@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import * as t from 'io-ts';
export const rangeSchema = t.type({
start: t.string,
end: t.string,
});
export type RangeSchema = t.TypeOf<typeof rangeSchema>;
export type Range = RangeSchema;

6
x-pack/plugins/security_solution/common/risk_engine/risk_score_preview/request_schema.ts
View file

@ -9,6 +9,7 @@ import * as t from 'io-ts';
import { DataViewId } from '../../api/detection_engine/model/rule_schema';
import { afterKeysSchema } from '../after_keys';
import { identifierTypeSchema } from '../identifier_types';
import { rangeSchema } from '../range';
import { riskWeightsSchema } from '../risk_weights/schema';
export const riskScorePreviewRequestSchema = t.exact(
@ -22,10 +23,7 @@ export const riskScorePreviewRequestSchema = t.exact(
filter: t.unknown,
page_size: t.number,
identifier_type: identifierTypeSchema,
range: t.type({
start: t.string,
end: t.string,
}),
range: rangeSchema,
weights: riskWeightsSchema,
}),
])

2
x-pack/plugins/security_solution/server/lib/detection_engine/routes/__mocks__/request_context.ts
View file

@ -34,7 +34,7 @@ import type {
import { getEndpointAuthzInitialStateMock } from '../../../../../common/endpoint/service/authz/mocks';
import type { EndpointAuthz } from '../../../../../common/endpoint/types/authz';
import { riskEngineDataClientMock } from '../../../risk_engine/__mocks__/risk_engine_data_client_mock';
import { riskEngineDataClientMock } from '../../../risk_engine/risk_engine_data_client.mock';
export const createMockClients = () => {
const core = coreMock.createRequestHandlerContext();

28
x-pack/plugins/security_solution/server/lib/risk_engine/calculate_risk_scores.test.ts
View file

@ -48,6 +48,34 @@ describe('calculateRiskScores()', () => {
);
});
it('drops an empty object filter if specified by the caller', async () => {
params.filter = {};
await calculateRiskScores(params);
expect(esClient.search).toHaveBeenCalledWith(
expect.objectContaining({
query: {
bool: {
filter: expect.not.arrayContaining([{}]),
},
},
})
);
});
it('drops an empty array filter if specified by the caller', async () => {
params.filter = [];
await calculateRiskScores(params);
expect(esClient.search).toHaveBeenCalledWith(
expect.objectContaining({
query: {
bool: {
filter: expect.not.arrayContaining([[]]),
},
},
})
);
});
describe('identifierType', () => {
it('creates aggs for both host and user by default', async () => {
await calculateRiskScores(params);

3
x-pack/plugins/security_solution/server/lib/risk_engine/calculate_risk_scores.ts
View file

@ -5,6 +5,7 @@
* 2.0.
*/
import { isEmpty } from 'lodash';
import type {
AggregationsAggregationContainer,
QueryDslQueryContainer,
@ -213,7 +214,7 @@ export const calculateRiskScores = async ({
const now = new Date().toISOString();
const filter = [{ exists: { field: ALERT_RISK_SCORE } }, filterFromRange(range)];
if (userFilter) {
if (!isEmpty(userFilter)) {
filter.push(userFilter as QueryDslQueryContainer);
}
const identifierTypes: IdentifierType[] = identifierType ? [identifierType] : ['host', 'user'];

2
x-pack/plugins/security_solution/server/lib/risk_engine/get_risk_inputs_index.ts
View file

@ -9,7 +9,7 @@ import type { MappingRuntimeFields } from '@elastic/elasticsearch/lib/api/types'
import type { Logger, SavedObjectsClientContract } from '@kbn/core/server';
import type { DataViewAttributes } from '@kbn/data-views-plugin/common';
interface RiskInputsIndexResponse {
export interface RiskInputsIndexResponse {
index: string;
runtimeMappings: MappingRuntimeFields;
}

53
x-pack/plugins/security_solution/server/lib/risk_engine/helpers.test.ts Normal file
View file

@ -0,0 +1,53 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { isRiskScoreCalculationComplete } from './helpers';
describe('isRiskScoreCalculationComplete', () => {
it('is true if both after_keys.host and after_keys.user are empty', () => {
const result = {
after_keys: {
host: {},
user: {},
},
};
// @ts-expect-error using a minimal result object for testing
expect(isRiskScoreCalculationComplete(result)).toEqual(true);
});
it('is true if after_keys is an empty object', () => {
const result = {
after_keys: {},
};
// @ts-expect-error using a minimal result object for testing
expect(isRiskScoreCalculationComplete(result)).toEqual(true);
});
it('is false if the host key has a key/value', () => {
const result = {
after_keys: {
host: {
key: 'value',
},
},
};
// @ts-expect-error using a minimal result object for testing
expect(isRiskScoreCalculationComplete(result)).toEqual(false);
});
it('is false if the user key has a key/value', () => {
const result = {
after_keys: {
user: {
key: 'value',
},
},
};
// @ts-expect-error using a minimal result object for testing
expect(isRiskScoreCalculationComplete(result)).toEqual(false);
});
});

7
x-pack/plugins/security_solution/server/lib/risk_engine/helpers.ts
View file

@ -6,6 +6,7 @@
*/
import type { AfterKey, AfterKeys, IdentifierType } from '../../../common/risk_engine';
import type { CalculateAndPersistScoresResponse } from './types';
export const getFieldForIdentifierAgg = (identifierType: IdentifierType): string =>
identifierType === 'host' ? 'host.name' : 'user.name';
@ -17,3 +18,9 @@ export const getAfterKeyForIdentifierType = ({
afterKeys: AfterKeys;
identifierType: IdentifierType;
}): AfterKey | undefined => afterKeys[identifierType];
export const isRiskScoreCalculationComplete = (
result: CalculateAndPersistScoresResponse
): boolean =>
Object.keys(result.after_keys.host ?? {}).length === 0 &&
Object.keys(result.after_keys.user ?? {}).length === 0;

10
x-pack/plugins/security_solution/server/lib/risk_engine/__mocks__/risk_engine_data_client_mock.ts → x-pack/plugins/security_solution/server/lib/risk_engine/risk_engine_data_client.mock.ts
View file

@ -5,13 +5,19 @@
* 2.0.
*/
import type { RiskEngineDataClient } from '../risk_engine_data_client';
import type { RiskEngineDataClient } from './risk_engine_data_client';
const createRiskEngineDataClientMock = () =>
({
disableLegacyRiskEngine: jest.fn(),
disableRiskEngine: jest.fn(),
enableRiskEngine: jest.fn(),
getConfiguration: jest.fn(),
getRiskInputsIndex: jest.fn(),
getStatus: jest.fn(),
getWriter: jest.fn(),
initializeResources: jest.fn(),
init: jest.fn(),
initializeResources: jest.fn(),
} as unknown as jest.Mocked<RiskEngineDataClient>);
export const riskEngineDataClientMock = { create: createRiskEngineDataClientMock };

96
x-pack/plugins/security_solution/server/lib/risk_engine/risk_engine_data_client.test.ts
View file

@ -15,7 +15,10 @@ import {
elasticsearchServiceMock,
savedObjectsClientMock,
} from '@kbn/core/server/mocks';
import { taskManagerMock } from '@kbn/task-manager-plugin/server/mocks';
import type { SavedObject } from '@kbn/core/server';
import { RiskEngineDataClient } from './risk_engine_data_client';
import type { RiskEngineConfiguration } from './types';
import { createDataStream } from './utils/create_datastream';
import * as savedObjectConfig from './utils/saved_object_configuration';
import * as transforms from './utils/transforms';
@ -537,10 +540,6 @@ describe('RiskEngineDataClient', () => {
mockSavedObjectClient.find.mockResolvedValue(getSavedObjectConfiguration());
});
afterEach(() => {
mockSavedObjectClient.find.mockReset();
});
it('should return status with enabled true', async () => {
mockSavedObjectClient.find.mockResolvedValue(
getSavedObjectConfiguration({
@ -611,31 +610,43 @@ describe('RiskEngineDataClient', () => {
});
});
describe('#getConfiguration', () => {
it('retrieves configuration from the saved object', async () => {
mockSavedObjectClient.find.mockResolvedValueOnce(getSavedObjectConfiguration());
const configuration = await riskEngineDataClient.getConfiguration();
expect(mockSavedObjectClient.find).toHaveBeenCalledTimes(1);
expect(configuration).toEqual({
enabled: false,
});
});
});
describe('enableRiskEngine', () => {
afterEach(() => {
mockSavedObjectClient.find.mockReset();
let mockTaskManagerStart: ReturnType<typeof taskManagerMock.createStart>;
beforeEach(() => {
mockSavedObjectClient.find.mockResolvedValue(getSavedObjectConfiguration());
mockTaskManagerStart = taskManagerMock.createStart();
});
it('should return error if saved object not exist', async () => {
mockSavedObjectClient.find.mockResolvedValueOnce({
it('returns an error if saved object does not exist', async () => {
mockSavedObjectClient.find.mockResolvedValue({
page: 1,
per_page: 20,
total: 0,
saved_objects: [],
});
expect.assertions(1);
try {
await riskEngineDataClient.enableRiskEngine();
} catch (e) {
expect(e.message).toEqual('There no saved object configuration for risk engine');
}
await expect(
riskEngineDataClient.enableRiskEngine({ taskManager: mockTaskManagerStart })
).rejects.toThrow('Risk engine configuration not found');
});
it('should update saved object attrubute', async () => {
mockSavedObjectClient.find.mockResolvedValueOnce(getSavedObjectConfiguration());
await riskEngineDataClient.enableRiskEngine();
it('should update saved object attribute', async () => {
await riskEngineDataClient.enableRiskEngine({ taskManager: mockTaskManagerStart });
expect(mockSavedObjectClient.update).toHaveBeenCalledWith(
'risk-engine-configuration',
@ -648,11 +659,36 @@ describe('RiskEngineDataClient', () => {
}
);
});
describe('if task manager throws an error', () => {
beforeEach(() => {
mockTaskManagerStart.ensureScheduled.mockRejectedValueOnce(new Error('Task Manager error'));
});
it('disables the risk engine and re-throws the error', async () => {
await expect(
riskEngineDataClient.enableRiskEngine({ taskManager: mockTaskManagerStart })
).rejects.toThrow('Task Manager error');
expect(mockSavedObjectClient.update).toHaveBeenCalledWith(
'risk-engine-configuration',
'de8ca330-2d26-11ee-bc86-f95bf6192ee6',
{
enabled: false,
},
{
refresh: 'wait_for',
}
);
});
});
});
describe('disableRiskEngine', () => {
afterEach(() => {
mockSavedObjectClient.find.mockReset();
let mockTaskManagerStart: ReturnType<typeof taskManagerMock.createStart>;
beforeEach(() => {
mockTaskManagerStart = taskManagerMock.createStart();
});
it('should return error if saved object not exist', async () => {
@ -665,16 +701,16 @@ describe('RiskEngineDataClient', () => {
expect.assertions(1);
try {
await riskEngineDataClient.disableRiskEngine();
await riskEngineDataClient.disableRiskEngine({ taskManager: mockTaskManagerStart });
} catch (e) {
expect(e.message).toEqual('There no saved object configuration for risk engine');
expect(e.message).toEqual('Risk engine configuration not found');
}
});
it('should update saved object attrubute', async () => {
mockSavedObjectClient.find.mockResolvedValueOnce(getSavedObjectConfiguration());
await riskEngineDataClient.disableRiskEngine();
await riskEngineDataClient.disableRiskEngine({ taskManager: mockTaskManagerStart });
expect(mockSavedObjectClient.update).toHaveBeenCalledWith(
'risk-engine-configuration',
@ -690,6 +726,7 @@ describe('RiskEngineDataClient', () => {
});
describe('init', () => {
let mockTaskManagerStart: ReturnType<typeof taskManagerMock.createStart>;
const initializeResourcesMock = jest.spyOn(
RiskEngineDataClient.prototype,
'initializeResources'
@ -701,6 +738,7 @@ describe('RiskEngineDataClient', () => {
'disableLegacyRiskEngine'
);
beforeEach(() => {
mockTaskManagerStart = taskManagerMock.createStart();
disableLegacyRiskEngineMock.mockImplementation(() => Promise.resolve(true));
initializeResourcesMock.mockImplementation(() => {
@ -711,9 +749,9 @@ describe('RiskEngineDataClient', () => {
return Promise.resolve(getSavedObjectConfiguration().saved_objects[0]);
});
jest.spyOn(savedObjectConfig, 'initSavedObjects').mockImplementation(() => {
return Promise.resolve(getSavedObjectConfiguration().saved_objects[0]);
});
jest
.spyOn(savedObjectConfig, 'initSavedObjects')
.mockResolvedValue({} as unknown as SavedObject<RiskEngineConfiguration>);
});
afterEach(() => {
@ -725,6 +763,7 @@ describe('RiskEngineDataClient', () => {
it('success', async () => {
const initResult = await riskEngineDataClient.init({
namespace: 'default',
taskManager: mockTaskManagerStart,
});
expect(initResult).toEqual({
@ -742,6 +781,7 @@ describe('RiskEngineDataClient', () => {
});
const initResult = await riskEngineDataClient.init({
namespace: 'default',
taskManager: mockTaskManagerStart,
});
expect(initResult).toEqual({
@ -760,6 +800,7 @@ describe('RiskEngineDataClient', () => {
const initResult = await riskEngineDataClient.init({
namespace: 'default',
taskManager: mockTaskManagerStart,
});
expect(initResult).toEqual({
@ -778,6 +819,7 @@ describe('RiskEngineDataClient', () => {
const initResult = await riskEngineDataClient.init({
namespace: 'default',
taskManager: mockTaskManagerStart,
});
expect(initResult).toEqual({
@ -796,6 +838,7 @@ describe('RiskEngineDataClient', () => {
const initResult = await riskEngineDataClient.init({
namespace: 'default',
taskManager: mockTaskManagerStart,
});
expect(initResult).toEqual({
@ -814,6 +857,7 @@ describe('RiskEngineDataClient', () => {
const initResult = await riskEngineDataClient.init({
namespace: 'default',
taskManager: mockTaskManagerStart,
});
expect(initResult).toEqual({

67
x-pack/plugins/security_solution/server/lib/risk_engine/risk_engine_data_client.ts
View file

@ -15,6 +15,7 @@ import {
import { mappingFromFieldMap } from '@kbn/alerting-plugin/common';
import { DEFAULT_NAMESPACE_STRING } from '@kbn/core-saved-objects-utils-server';
import type { Logger, ElasticsearchClient, SavedObjectsClientContract } from '@kbn/core/server';
import type { TaskManagerStartContract } from '@kbn/task-manager-plugin/server';
import {
riskScoreFieldMap,
@ -47,10 +48,13 @@ import {
initSavedObjects,
getEnabledRiskEngineAmount,
} from './utils/saved_object_configuration';
import { getRiskInputsIndex } from './get_risk_inputs_index';
import { removeRiskScoringTask, startRiskScoringTask } from './tasks';
import { createIndex } from './utils/create_index';
interface InitOpts {
namespace: string;
taskManager: TaskManagerStartContract;
}
interface InitializeRiskEngineResourcesOpts {
@ -69,7 +73,7 @@ export class RiskEngineDataClient {
private writerCache: Map<string, Writer> = new Map();
constructor(private readonly options: RiskEngineDataClientOpts) {}
public async init({ namespace }: InitOpts) {
public async init({ namespace, taskManager }: InitOpts) {
const result: InitRiskEngineResult = {
legacyRiskEngineDisabled: false,
riskEngineResourcesInstalled: false,
@ -94,7 +98,10 @@ export class RiskEngineDataClient {
}
try {
await initSavedObjects({ savedObjectsClient: this.options.soClient });
await initSavedObjects({
savedObjectsClient: this.options.soClient,
namespace,
});
result.riskEngineConfigurationCreated = true;
} catch (e) {
result.errors.push(e.message);
@ -102,7 +109,7 @@ export class RiskEngineDataClient {
}
try {
await this.enableRiskEngine();
await this.enableRiskEngine({ taskManager });
result.riskEngineEnabled = true;
} catch (e) {
result.errors.push(e.message);
@ -133,6 +140,18 @@ export class RiskEngineDataClient {
return writer;
}
public getConfiguration = () =>
getConfiguration({
savedObjectsClient: this.options.soClient,
});
public getRiskInputsIndex = ({ dataViewId }: { dataViewId: string }) =>
getRiskInputsIndex({
dataViewId,
logger: this.options.logger,
soClient: this.options.soClient,
});
public async getStatus({ namespace }: { namespace: string }) {
const riskEngineStatus = await this.getCurrentStatus();
const legacyRiskEngineStatus = await this.getLegacyStatus({ namespace });
@ -140,18 +159,38 @@ export class RiskEngineDataClient {
return { riskEngineStatus, legacyRiskEngineStatus, isMaxAmountOfRiskEnginesReached };
}
public async enableRiskEngine() {
// code to run task
return updateSavedObjectAttribute({
savedObjectsClient: this.options.soClient,
attributes: {
enabled: true,
},
});
public async enableRiskEngine({ taskManager }: { taskManager: TaskManagerStartContract }) {
try {
const configurationResult = await updateSavedObjectAttribute({
savedObjectsClient: this.options.soClient,
attributes: {
enabled: true,
},
});
await startRiskScoringTask({
logger: this.options.logger,
namespace: this.options.namespace,
riskEngineDataClient: this,
taskManager,
});
return configurationResult;
} catch (e) {
this.options.logger.error(`Error while enabling risk engine: ${e.message}`);
await this.disableRiskEngine({ taskManager });
throw e;
}
}
public async disableRiskEngine() {
// code to stop task
public async disableRiskEngine({ taskManager }: { taskManager: TaskManagerStartContract }) {
await removeRiskScoringTask({
namespace: this.options.namespace,
taskManager,
logger: this.options.logger,
});
return updateSavedObjectAttribute({
savedObjectsClient: this.options.soClient,
@ -179,7 +218,7 @@ export class RiskEngineDataClient {
}
private async getCurrentStatus() {
const configuration = await getConfiguration({ savedObjectsClient: this.options.soClient });
const configuration = await this.getConfiguration();
if (configuration) {
return configuration.enabled ? RiskEngineStatus.ENABLED : RiskEngineStatus.DISABLED;

2
x-pack/plugins/security_solution/server/lib/risk_engine/risk_score_service.mock.ts
View file

@ -25,6 +25,8 @@ const createRiskScoreMock = (overrides: Partial<RiskScore> = {}): RiskScore => (
const createRiskScoreServiceMock = (): jest.Mocked<RiskScoreService> => ({
calculateScores: jest.fn(),
calculateAndPersistScores: jest.fn(),
getConfiguration: jest.fn(),
getRiskInputsIndex: jest.fn(),
});
export const riskScoreServiceMock = {

6
x-pack/plugins/security_solution/server/lib/risk_engine/risk_score_service.ts
View file

@ -11,16 +11,20 @@ import type {
CalculateAndPersistScoresResponse,
CalculateScoresParams,
CalculateScoresResponse,
RiskEngineConfiguration,
} from './types';
import { calculateRiskScores } from './calculate_risk_scores';
import { calculateAndPersistRiskScores } from './calculate_and_persist_risk_scores';
import type { RiskEngineDataClient } from './risk_engine_data_client';
import type { RiskInputsIndexResponse } from './get_risk_inputs_index';
export interface RiskScoreService {
calculateScores: (params: CalculateScoresParams) => Promise<CalculateScoresResponse>;
calculateAndPersistScores: (
params: CalculateAndPersistScoresParams
) => Promise<CalculateAndPersistScoresResponse>;
getConfiguration: () => Promise<RiskEngineConfiguration | null>;
getRiskInputsIndex: ({ dataViewId }: { dataViewId: string }) => Promise<RiskInputsIndexResponse>;
}
export interface RiskScoreServiceFactoryParams {
@ -39,4 +43,6 @@ export const riskScoreServiceFactory = ({
calculateScores: (params) => calculateRiskScores({ ...params, esClient, logger }),
calculateAndPersistScores: (params) =>
calculateAndPersistRiskScores({ ...params, esClient, logger, riskEngineDataClient, spaceId }),
getConfiguration: async () => riskEngineDataClient.getConfiguration(),
getRiskInputsIndex: async (params) => riskEngineDataClient.getRiskInputsIndex(params),
});

105
x-pack/plugins/security_solution/server/lib/risk_engine/routes/risk_engine_disable_route.test.ts Normal file
View file

@ -0,0 +1,105 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { taskManagerMock } from '@kbn/task-manager-plugin/server/mocks';
import { riskEngineDisableRoute } from './risk_engine_disable_route';
import { RISK_ENGINE_DISABLE_URL } from '../../../../common/constants';
import {
serverMock,
requestContextMock,
requestMock,
} from '../../detection_engine/routes/__mocks__';
import { riskEngineDataClientMock } from '../risk_engine_data_client.mock';
describe('risk score calculation route', () => {
let server: ReturnType<typeof serverMock.create>;
let context: ReturnType<typeof requestContextMock.convertContext>;
let mockTaskManagerStart: ReturnType<typeof taskManagerMock.createStart>;
let mockRiskEngineDataClient: ReturnType<typeof riskEngineDataClientMock.create>;
let getStartServicesMock: jest.Mock;
beforeEach(() => {
jest.resetAllMocks();
server = serverMock.create();
const { clients } = requestContextMock.createTools();
mockRiskEngineDataClient = riskEngineDataClientMock.create();
context = requestContextMock.convertContext(
requestContextMock.create({
...clients,
riskEngineDataClient: mockRiskEngineDataClient,
})
);
mockTaskManagerStart = taskManagerMock.createStart();
});
const buildRequest = () => {
return requestMock.create({
method: 'post',
path: RISK_ENGINE_DISABLE_URL,
body: {},
});
};
describe('when task manager is available', () => {
beforeEach(() => {
getStartServicesMock = jest
.fn()
.mockResolvedValue([{}, { taskManager: mockTaskManagerStart }]);
riskEngineDisableRoute(server.router, getStartServicesMock);
});
it('invokes the risk score data client', async () => {
const request = buildRequest();
await server.inject(request, context);
expect(mockRiskEngineDataClient.disableRiskEngine).toHaveBeenCalled();
});
it('returns a 200 when disabling is successful', async () => {
// @ts-expect-error response is not used in the route nor this test
mockRiskEngineDataClient.disableRiskEngine.mockResolvedValue({ enabled: false });
const request = buildRequest();
const response = await server.inject(request, context);
expect(response.status).toEqual(200);
});
it('returns a 500 if disabling fails', async () => {
mockRiskEngineDataClient.disableRiskEngine.mockRejectedValue(
new Error('something went wrong')
);
const request = buildRequest();
const response = await server.inject(request, context);
expect(response.status).toEqual(500);
expect(response.body.message.message).toEqual('something went wrong');
});
});
describe('when task manager is unavailable', () => {
beforeEach(() => {
getStartServicesMock = jest.fn().mockResolvedValueOnce([{}, { taskManager: undefined }]);
riskEngineDisableRoute(server.router, getStartServicesMock);
});
it('returns a 400 response', async () => {
const request = buildRequest();
const response = await server.inject(request, context);
expect(response.status).toEqual(400);
expect(response.body).toEqual({
message: {
message:
'Task Manager is unavailable, but is required to disable the risk engine. Please enable the taskManager plugin and try again.',
},
status_code: 400,
});
});
});
});

20
x-pack/plugins/security_solution/server/lib/risk_engine/routes/risk_engine_disable_route.ts
View file

@ -5,17 +5,16 @@
* 2.0.
*/
import type { Logger } from '@kbn/core/server';
import type { StartServicesAccessor } from '@kbn/core/server';
import { buildSiemResponse } from '@kbn/lists-plugin/server/routes/utils';
import { transformError } from '@kbn/securitysolution-es-utils';
import { RISK_ENGINE_DISABLE_URL, APP_ID } from '../../../../common/constants';
import type { SetupPlugins } from '../../../plugin';
import type { StartPlugins } from '../../../plugin';
import type { SecuritySolutionPluginRouter } from '../../../types';
export const riskEngineDisableRoute = (
router: SecuritySolutionPluginRouter,
logger: Logger,
security: SetupPlugins['security']
getStartServices: StartServicesAccessor<StartPlugins>
) => {
router.post(
{
@ -28,11 +27,22 @@ export const riskEngineDisableRoute = (
async (context, request, response) => {
const siemResponse = buildSiemResponse(response);
const [_, { taskManager }] = await getStartServices();
const securitySolution = await context.securitySolution;
const riskEngineClient = securitySolution.getRiskEngineDataClient();
if (!taskManager) {
return siemResponse.error({
statusCode: 400,
body: {
message:
'Task Manager is unavailable, but is required to disable the risk engine. Please enable the taskManager plugin and try again.',
},
});
}
try {
await riskEngineClient.disableRiskEngine();
await riskEngineClient.disableRiskEngine({ taskManager });
return response.ok({ body: { success: true } });
} catch (e) {
const error = transformError(e);

105
x-pack/plugins/security_solution/server/lib/risk_engine/routes/risk_engine_enable_route.test.ts Normal file
View file

@ -0,0 +1,105 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { taskManagerMock } from '@kbn/task-manager-plugin/server/mocks';
import { riskEngineEnableRoute } from './risk_engine_enable_route';
import { RISK_ENGINE_ENABLE_URL } from '../../../../common/constants';
import {
serverMock,
requestContextMock,
requestMock,
} from '../../detection_engine/routes/__mocks__';
import { riskEngineDataClientMock } from '../risk_engine_data_client.mock';
describe('risk score calculation route', () => {
let server: ReturnType<typeof serverMock.create>;
let context: ReturnType<typeof requestContextMock.convertContext>;
let mockTaskManagerStart: ReturnType<typeof taskManagerMock.createStart>;
let mockRiskEngineDataClient: ReturnType<typeof riskEngineDataClientMock.create>;
let getStartServicesMock: jest.Mock;
beforeEach(() => {
jest.resetAllMocks();
server = serverMock.create();
const { clients } = requestContextMock.createTools();
mockRiskEngineDataClient = riskEngineDataClientMock.create();
context = requestContextMock.convertContext(
requestContextMock.create({
...clients,
riskEngineDataClient: mockRiskEngineDataClient,
})
);
mockTaskManagerStart = taskManagerMock.createStart();
});
const buildRequest = () => {
return requestMock.create({
method: 'post',
path: RISK_ENGINE_ENABLE_URL,
body: {},
});
};
describe('when task manager is available', () => {
beforeEach(() => {
getStartServicesMock = jest
.fn()
.mockResolvedValue([{}, { taskManager: mockTaskManagerStart }]);
riskEngineEnableRoute(server.router, getStartServicesMock);
});
it('invokes the risk score service', async () => {
const request = buildRequest();
await server.inject(request, context);
expect(mockRiskEngineDataClient.enableRiskEngine).toHaveBeenCalled();
});
it('returns a 200 when enablement is successful', async () => {
// @ts-expect-error response is not used in the route nor this test
mockRiskEngineDataClient.enableRiskEngine.mockResolvedValue({ enabled: true });
const request = buildRequest();
const response = await server.inject(request, context);
expect(response.status).toEqual(200);
});
it('returns a 500 if enabling fails', async () => {
mockRiskEngineDataClient.enableRiskEngine.mockRejectedValue(
new Error('something went wrong')
);
const request = buildRequest();
const response = await server.inject(request, context);
expect(response.status).toEqual(500);
expect(response.body.message.message).toEqual('something went wrong');
});
});
describe('when task manager is unavailable', () => {
beforeEach(() => {
getStartServicesMock = jest.fn().mockResolvedValueOnce([{}, { taskManager: undefined }]);
riskEngineEnableRoute(server.router, getStartServicesMock);
});
it('returns a 400 response', async () => {
const request = buildRequest();
const response = await server.inject(request, context);
expect(response.status).toEqual(400);
expect(response.body).toEqual({
message: {
message:
'Task Manager is unavailable, but is required to enable the risk engine. Please enable the taskManager plugin and try again.',
},
status_code: 400,
});
});
});
});

20
x-pack/plugins/security_solution/server/lib/risk_engine/routes/risk_engine_enable_route.ts
View file

@ -5,17 +5,16 @@
* 2.0.
*/
import type { Logger } from '@kbn/core/server';
import type { StartServicesAccessor } from '@kbn/core/server';
import { buildSiemResponse } from '@kbn/lists-plugin/server/routes/utils';
import { transformError } from '@kbn/securitysolution-es-utils';
import { RISK_ENGINE_ENABLE_URL, APP_ID } from '../../../../common/constants';
import type { SetupPlugins } from '../../../plugin';
import type { StartPlugins } from '../../../plugin';
import type { SecuritySolutionPluginRouter } from '../../../types';
export const riskEngineEnableRoute = (
router: SecuritySolutionPluginRouter,
logger: Logger,
security: SetupPlugins['security']
getStartServices: StartServicesAccessor<StartPlugins>
) => {
router.post(
{
@ -27,11 +26,22 @@ export const riskEngineEnableRoute = (
},
async (context, request, response) => {
const siemResponse = buildSiemResponse(response);
const [_, { taskManager }] = await getStartServices();
const securitySolution = await context.securitySolution;
const riskEngineClient = securitySolution.getRiskEngineDataClient();
if (!taskManager) {
return siemResponse.error({
statusCode: 400,
body: {
message:
'Task Manager is unavailable, but is required to enable the risk engine. Please enable the taskManager plugin and try again.',
},
});
}
try {
await riskEngineClient.enableRiskEngine();
await riskEngineClient.enableRiskEngine({ taskManager });
return response.ok({ body: { success: true } });
} catch (e) {
const error = transformError(e);

23
x-pack/plugins/security_solution/server/lib/risk_engine/routes/risk_engine_init_route.ts
View file

@ -5,18 +5,17 @@
* 2.0.
*/
import type { Logger } from '@kbn/core/server';
import type { StartServicesAccessor } from '@kbn/core/server';
import { buildSiemResponse } from '@kbn/lists-plugin/server/routes/utils';
import { transformError } from '@kbn/securitysolution-es-utils';
import { RISK_ENGINE_INIT_URL, APP_ID } from '../../../../common/constants';
import type { SetupPlugins } from '../../../plugin';
import type { StartPlugins } from '../../../plugin';
import type { SecuritySolutionPluginRouter } from '../../../types';
export const riskEngineInitRoute = (
router: SecuritySolutionPluginRouter,
logger: Logger,
security: SetupPlugins['security']
getStartServices: StartServicesAccessor<StartPlugins>
) => {
router.post(
{
@ -29,11 +28,23 @@ export const riskEngineInitRoute = (
async (context, request, response) => {
const siemResponse = buildSiemResponse(response);
const securitySolution = await context.securitySolution;
const riskEngineClient = securitySolution.getRiskEngineDataClient();
const [_, { taskManager }] = await getStartServices();
const riskEngineDataClient = securitySolution.getRiskEngineDataClient();
const spaceId = securitySolution.getSpaceId();
try {
const initResult = await riskEngineClient.init({
if (!taskManager) {
return siemResponse.error({
statusCode: 400,
body: {
message:
'Task Manager is unavailable, but is required to initialize the risk engine. Please enable the taskManager plugin and try again.',
},
});
}
const initResult = await riskEngineDataClient.init({
taskManager,
namespace: spaceId,
});

3
x-pack/plugins/security_solution/server/lib/risk_engine/routes/risk_engine_status_route.ts
View file

@ -5,14 +5,13 @@
* 2.0.
*/
import type { Logger } from '@kbn/core/server';
import { buildSiemResponse } from '@kbn/lists-plugin/server/routes/utils';
import { transformError } from '@kbn/securitysolution-es-utils';
import { RISK_ENGINE_STATUS_URL, APP_ID } from '../../../../common/constants';
import type { SecuritySolutionPluginRouter } from '../../../types';
export const riskEngineStatusRoute = (router: SecuritySolutionPluginRouter, logger: Logger) => {
export const riskEngineStatusRoute = (router: SecuritySolutionPluginRouter) => {
router.get(
{
path: RISK_ENGINE_STATUS_URL,

26
x-pack/plugins/security_solution/server/lib/risk_engine/saved_object/risk_engine_configuration_type.ts
View file

@ -13,9 +13,35 @@ export const riskEngineConfigurationTypeName = 'risk-engine-configuration';
export const riskEngineConfigurationTypeMappings: SavedObjectsType['mappings'] = {
dynamic: false,
properties: {
dataViewId: {
type: 'keyword',
},
enabled: {
type: 'boolean',
},
filter: {
dynamic: false,
properties: {},
},
identifierType: {
type: 'keyword',
},
interval: {
type: 'keyword',
},
pageSize: {
type: 'integer',
},
range: {
properties: {
start: {
type: 'keyword',
},
end: {
type: 'keyword',
},
},
},
},
};

20
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/constants.ts Normal file
View file

@ -0,0 +1,20 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export const SCOPE = ['securitySolution'];
export const TYPE = 'risk_engine:risk_scoring';
export const VERSION = '0.0.1';
export const INTERVAL = '1h';
export const TIMEOUT = '5m';
export const RISK_SCORING_TASK_CONSTANTS = {
SCOPE,
TYPE,
VERSION,
INTERVAL,
TIMEOUT,
};

51
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/helpers.test.ts Normal file
View file

@ -0,0 +1,51 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import moment from 'moment';
import { convertDateToISOString } from './helpers';
moment.suppressDeprecationWarnings = true;
describe('convertDateToISOString', () => {
const ISO_8601_PATTERN = /\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}Z/;
it('converts a datemath expression to an ISO string', () => {
const date = 'now-1d';
const result = convertDateToISOString(date);
expect(result).toMatch(ISO_8601_PATTERN);
});
it('converts a rounded datemath expression to an ISO string', () => {
const date = 'now-30d/d';
const result = convertDateToISOString(date);
expect(result).toMatch(ISO_8601_PATTERN);
});
it('converts a regular date string to an ISO string', () => {
const date = '2023-08-03T12:34:56.789Z';
const result = convertDateToISOString(date);
expect(result).toMatch(ISO_8601_PATTERN);
});
it('does nothing to an ISO string', () => {
const date = '2023-08-03T12:34:56.789Z';
const result = convertDateToISOString(date);
expect(result).toEqual(date);
});
it('throws an error if the date string is invalid', () => {
const date = 'hi mom';
expect(() => {
convertDateToISOString(date);
}).toThrowErrorMatchingInlineSnapshot(`"Could not convert string \\"hi mom\\" to ISO string"`);
});
});

74
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/helpers.ts Normal file
View file

@ -0,0 +1,74 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import datemath from '@kbn/datemath';
import {
CoreKibanaRequest,
type KibanaRequest,
SECURITY_EXTENSION_ID,
type CoreStart,
} from '@kbn/core/server';
import { addSpaceIdToPath } from '@kbn/spaces-plugin/server';
import type { Range } from '../../../../common/risk_engine';
export const convertDateToISOString = (dateString: string): string => {
const date = datemath.parse(dateString);
if (date?.isValid()) {
return date.toISOString();
} else {
throw new Error(`Could not convert string "${dateString}" to ISO string`);
}
};
export const convertRangeToISO = (range: Range): Range => ({
start: convertDateToISOString(range.start),
end: convertDateToISOString(range.end),
});
const buildFakeScopedRequest = ({
coreStart,
namespace,
}: {
coreStart: CoreStart;
namespace: string;
}): KibanaRequest => {
const rawRequest = {
headers: {},
path: '/',
};
const request = CoreKibanaRequest.from(rawRequest);
const scopedPath = addSpaceIdToPath('/', namespace);
coreStart.http.basePath.set(request, scopedPath);
return request;
};
/**
* Builds a SavedObjectsClient scoped to the given namespace. This should be used with caution, and only in cases where a real kibana request is not available to build a proper scoped client (e.g. a task manager task).
*
__Note__: Because the kibana system user cannot access SavedObjects itself, this client does not have the security extension enabled, which has (negative) implications both for logging and for security.
* @param coreStart CoreStart plugin context
* @param namespace the namespace to which the client should be scoped
* @returns a SavedObjectsClient scoped to the given namespace
*/
export const buildScopedInternalSavedObjectsClientUnsafe = ({
coreStart,
namespace,
}: {
coreStart: CoreStart;
namespace: string;
}) => {
const fakeScopedRequest = buildFakeScopedRequest({ coreStart, namespace });
return coreStart.savedObjects.getScopedClient(fakeScopedRequest, {
excludedExtensions: [SECURITY_EXTENSION_ID],
});
};

12
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/index.ts Normal file
View file

@ -0,0 +1,12 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export {
registerRiskScoringTask,
startRiskScoringTask,
removeRiskScoringTask,
} from './risk_scoring_task';

33
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/risk_scoring_task.mock.ts Normal file
View file

@ -0,0 +1,33 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { type ConcreteTaskInstance, TaskStatus } from '@kbn/task-manager-plugin/server';
import { taskManagerMock } from '@kbn/task-manager-plugin/server/mocks';
import { TYPE, VERSION } from './constants';
import { defaultState } from './state';
const createRiskScoringTaskInstanceMock = (
overrides: Partial<ConcreteTaskInstance> = {}
): ConcreteTaskInstance =>
taskManagerMock.createTask({
id: `${TYPE}:default:${VERSION}`,
runAt: new Date(),
attempts: 0,
ownerId: '',
status: TaskStatus.Running,
startedAt: new Date(),
scheduledAt: new Date(),
retryAt: new Date(),
params: {},
state: defaultState,
taskType: TYPE,
...overrides,
});
export const riskScoringTaskMock = {
createInstance: createRiskScoringTaskInstanceMock,
};

398
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/risk_scoring_task.test.ts Normal file
View file

@ -0,0 +1,398 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { SavedObjectsErrorHelpers } from '@kbn/core/server';
import { coreMock } from '@kbn/core/server/mocks';
import { taskManagerMock } from '@kbn/task-manager-plugin/server/mocks';
import { loggerMock } from '@kbn/logging-mocks';
import type { RiskScoreService } from '../risk_score_service';
import { riskScoreServiceMock } from '../risk_score_service.mock';
import { riskScoringTaskMock } from './risk_scoring_task.mock';
import { riskEngineDataClientMock } from '../risk_engine_data_client.mock';
import {
registerRiskScoringTask,
startRiskScoringTask,
removeRiskScoringTask,
runTask,
} from './risk_scoring_task';
const ISO_8601_PATTERN = /\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}Z/;
describe('Risk Scoring Task', () => {
let mockRiskEngineDataClient: ReturnType<typeof riskEngineDataClientMock.create>;
let mockRiskScoreService: ReturnType<typeof riskScoreServiceMock.create>;
let mockCore: ReturnType<typeof coreMock.createSetup>;
let mockTaskManagerSetup: ReturnType<typeof taskManagerMock.createSetup>;
let mockTaskManagerStart: ReturnType<typeof taskManagerMock.createStart>;
let mockLogger: ReturnType<typeof loggerMock.create>;
beforeEach(() => {
mockCore = coreMock.createSetup();
mockRiskEngineDataClient = riskEngineDataClientMock.create();
mockRiskScoreService = riskScoreServiceMock.create();
mockTaskManagerSetup = taskManagerMock.createSetup();
mockTaskManagerStart = taskManagerMock.createStart();
mockLogger = loggerMock.create();
});
describe('registerRiskScoringTask()', () => {
it('registers the task with TaskManager', () => {
expect(mockTaskManagerSetup.registerTaskDefinitions).not.toHaveBeenCalled();
registerRiskScoringTask({
getStartServices: mockCore.getStartServices,
kibanaVersion: '8.10.0',
taskManager: mockTaskManagerSetup,
logger: mockLogger,
});
expect(mockTaskManagerSetup.registerTaskDefinitions).toHaveBeenCalled();
});
it('does nothing if TaskManager is not available', () => {
expect(mockTaskManagerSetup.registerTaskDefinitions).not.toHaveBeenCalled();
registerRiskScoringTask({
getStartServices: mockCore.getStartServices,
kibanaVersion: '8.10.0',
taskManager: undefined,
logger: mockLogger,
});
expect(mockTaskManagerSetup.registerTaskDefinitions).not.toHaveBeenCalled();
});
});
describe('startRiskScoringTask()', () => {
it('schedules the task', async () => {
await startRiskScoringTask({
logger: mockLogger,
namespace: 'default',
taskManager: mockTaskManagerStart,
riskEngineDataClient: mockRiskEngineDataClient,
});
expect(mockTaskManagerStart.ensureScheduled).toHaveBeenCalledWith(
expect.objectContaining({
schedule: { interval: '1h' },
taskType: 'risk_engine:risk_scoring',
})
);
});
it('schedules the task for a particular namespace', async () => {
await startRiskScoringTask({
logger: mockLogger,
namespace: 'other',
taskManager: mockTaskManagerStart,
riskEngineDataClient: mockRiskEngineDataClient,
});
expect(mockTaskManagerStart.ensureScheduled).toHaveBeenCalledWith(
expect.objectContaining({
schedule: { interval: '1h' },
taskType: 'risk_engine:risk_scoring',
state: expect.objectContaining({ namespace: 'other' }),
})
);
});
it('rethrows an error from taskManager', async () => {
mockTaskManagerStart.ensureScheduled.mockRejectedValueOnce(new Error('whoops'));
await expect(
startRiskScoringTask({
logger: mockLogger,
namespace: 'other',
taskManager: mockTaskManagerStart,
riskEngineDataClient: mockRiskEngineDataClient,
})
).rejects.toThrowError('whoops');
});
});
describe('removeRiskScoringTask()', () => {
it('removes the task', async () => {
await removeRiskScoringTask({
namespace: 'default',
logger: mockLogger,
taskManager: mockTaskManagerStart,
});
expect(mockTaskManagerStart.remove).toHaveBeenCalledWith(
'risk_engine:risk_scoring:default:0.0.1'
);
});
it('removes the task for a non-default namespace', async () => {
await removeRiskScoringTask({
namespace: 'other',
logger: mockLogger,
taskManager: mockTaskManagerStart,
});
expect(mockTaskManagerStart.remove).toHaveBeenCalledWith(
'risk_engine:risk_scoring:other:0.0.1'
);
});
it('does nothing if task was not found', async () => {
mockTaskManagerStart.remove.mockRejectedValueOnce(
SavedObjectsErrorHelpers.createGenericNotFoundError('type', 'id')
);
await removeRiskScoringTask({
namespace: 'default',
logger: mockLogger,
taskManager: mockTaskManagerStart,
});
expect(mockLogger.error).not.toHaveBeenCalled();
});
it('rethrows errors other than "not found"', async () => {
mockTaskManagerStart.remove.mockRejectedValueOnce(new Error('whoops'));
await expect(
removeRiskScoringTask({
namespace: 'default',
logger: mockLogger,
taskManager: mockTaskManagerStart,
})
).rejects.toThrowError('whoops');
expect(mockLogger.error).toHaveBeenCalledWith('Failed to remove risk scoring task: whoops');
});
});
describe('runTask()', () => {
let riskScoringTaskInstanceMock: ReturnType<typeof riskScoringTaskMock.createInstance>;
let getRiskScoreService: (namespace: string) => Promise<RiskScoreService>;
beforeEach(async () => {
await startRiskScoringTask({
logger: mockLogger,
namespace: 'default',
taskManager: mockTaskManagerStart,
riskEngineDataClient: mockRiskEngineDataClient,
});
riskScoringTaskInstanceMock = riskScoringTaskMock.createInstance();
mockRiskScoreService.getRiskInputsIndex.mockResolvedValueOnce({
index: 'index',
runtimeMappings: {},
});
mockRiskScoreService.getConfiguration.mockResolvedValue({
dataViewId: 'data_view_id',
enabled: true,
filter: {},
identifierType: 'host',
interval: '1h',
pageSize: 10_000,
range: { start: 'now-30d', end: 'now' },
});
getRiskScoreService = jest.fn().mockResolvedValueOnce(mockRiskScoreService);
});
describe('when there are no scores to calculate', () => {
beforeEach(() => {
mockRiskScoreService.calculateAndPersistScores.mockResolvedValueOnce({
after_keys: {},
scores_written: 0,
errors: [],
});
});
it('invokes the risk score service only once', async () => {
await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.calculateAndPersistScores).toHaveBeenCalledTimes(1);
});
});
describe('when there are scores to calculate', () => {
beforeEach(() => {
mockRiskScoreService.calculateAndPersistScores
.mockResolvedValueOnce({
after_keys: { host: { 'host.name': 'value' } },
scores_written: 5,
errors: [],
})
.mockResolvedValueOnce({
after_keys: {},
scores_written: 5,
errors: [],
});
});
it('retrieves configuration from the saved object', async () => {
await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.getConfiguration).toHaveBeenCalledTimes(1);
});
it('invokes the risk score service once for each page of scores', async () => {
await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.calculateAndPersistScores).toHaveBeenCalledTimes(2);
});
it('invokes the risk score service with the persisted configuration', async () => {
mockRiskScoreService.getConfiguration.mockResolvedValueOnce({
dataViewId: 'data_view_id',
enabled: true,
filter: {
term: { 'host.name': 'SUSPICIOUS' },
},
identifierType: 'host',
interval: '2h',
pageSize: 11_111,
range: { start: 'now-30d', end: 'now' },
});
await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.calculateAndPersistScores).toHaveBeenCalledWith(
expect.objectContaining({
filter: {
term: { 'host.name': 'SUSPICIOUS' },
},
identifierType: 'host',
pageSize: 11_111,
range: {
start: expect.stringMatching(ISO_8601_PATTERN),
end: expect.stringMatching(ISO_8601_PATTERN),
},
})
);
});
describe('when no identifier type is configured', () => {
beforeEach(() => {
mockRiskScoreService.getConfiguration.mockResolvedValue({
dataViewId: 'data_view_id',
enabled: true,
filter: {},
identifierType: undefined,
interval: '1h',
pageSize: 10_000,
range: { start: 'now-30d', end: 'now' },
});
// add additional mock responses for the additional identifier calls
mockRiskScoreService.calculateAndPersistScores
.mockResolvedValueOnce({
after_keys: { host: { 'user.name': 'value' } },
scores_written: 5,
errors: [],
})
.mockResolvedValueOnce({
after_keys: {},
scores_written: 5,
errors: [],
});
});
it('invokes the risk score service once for type of identifier', async () => {
await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.calculateAndPersistScores).toHaveBeenCalledTimes(4);
expect(mockRiskScoreService.calculateAndPersistScores).toHaveBeenCalledWith(
expect.objectContaining({
identifierType: 'host',
})
);
expect(mockRiskScoreService.calculateAndPersistScores).toHaveBeenCalledWith(
expect.objectContaining({
identifierType: 'user',
})
);
});
});
it('updates the task state', async () => {
const { state: initialState } = riskScoringTaskInstanceMock;
const { state: nextState } = await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(initialState).not.toEqual(nextState);
expect(nextState).toEqual(
expect.objectContaining({
runs: 1,
scoresWritten: 10,
})
);
});
describe('short-circuiting', () => {
it('does not execute if the risk engine is not enabled', async () => {
mockRiskScoreService.getConfiguration.mockResolvedValueOnce({
dataViewId: 'data_view_id',
enabled: false,
filter: {
term: { 'host.name': 'SUSPICIOUS' },
},
identifierType: undefined,
interval: '2h',
pageSize: 11_111,
range: { start: 'now-30d', end: 'now' },
});
await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.calculateAndPersistScores).not.toHaveBeenCalled();
expect(mockLogger.info).toHaveBeenCalledWith(expect.stringContaining('not enabled'));
});
it('does not execute if the configuration is not found', async () => {
mockRiskScoreService.getConfiguration.mockResolvedValueOnce(null);
await runTask({
getRiskScoreService,
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.calculateAndPersistScores).not.toHaveBeenCalled();
expect(mockLogger.info).toHaveBeenCalledWith(
expect.stringContaining('configuration not found')
);
});
it('does not execute if the riskScoreService is not available', async () => {
await runTask({
getRiskScoreService: jest.fn().mockResolvedValueOnce(undefined),
logger: mockLogger,
taskInstance: riskScoringTaskInstanceMock,
});
expect(mockRiskScoreService.calculateAndPersistScores).not.toHaveBeenCalled();
expect(mockLogger.info).toHaveBeenCalledWith(
expect.stringContaining('service is not available')
);
});
});
});
});
});

246
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/risk_scoring_task.ts Normal file
View file

@ -0,0 +1,246 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import moment from 'moment';
import { asyncForEach } from '@kbn/std';
import {
type Logger,
SavedObjectsErrorHelpers,
type StartServicesAccessor,
} from '@kbn/core/server';
import type {
ConcreteTaskInstance,
TaskManagerSetupContract,
TaskManagerStartContract,
} from '@kbn/task-manager-plugin/server';
import type { AfterKeys, IdentifierType } from '../../../../common/risk_engine';
import type { StartPlugins } from '../../../plugin';
import { type RiskScoreService, riskScoreServiceFactory } from '../risk_score_service';
import { RiskEngineDataClient } from '../risk_engine_data_client';
import { isRiskScoreCalculationComplete } from '../helpers';
import {
defaultState,
stateSchemaByVersion,
type LatestTaskStateSchema as RiskScoringTaskState,
} from './state';
import { INTERVAL, SCOPE, TIMEOUT, TYPE, VERSION } from './constants';
import { buildScopedInternalSavedObjectsClientUnsafe, convertRangeToISO } from './helpers';
import { RiskScoreEntity } from '../../../../common/risk_engine/types';
const logFactory =
(logger: Logger, taskId: string) =>
(message: string): void =>
logger.info(`[task ${taskId}]: ${message}`);
const getTaskName = (): string => TYPE;
const getTaskId = (namespace: string): string => `${TYPE}:${namespace}:${VERSION}`;
type GetRiskScoreService = (namespace: string) => Promise<RiskScoreService>;
export const registerRiskScoringTask = ({
getStartServices,
kibanaVersion,
logger,
taskManager,
}: {
getStartServices: StartServicesAccessor<StartPlugins>;
kibanaVersion: string;
logger: Logger;
taskManager: TaskManagerSetupContract | undefined;
}): void => {
if (!taskManager) {
logger.info('Task Manager is unavailable; skipping risk engine task registration.');
return;
}
const getRiskScoreService: GetRiskScoreService = (namespace) =>
getStartServices().then(([coreStart, _]) => {
const esClient = coreStart.elasticsearch.client.asInternalUser;
const soClient = buildScopedInternalSavedObjectsClientUnsafe({ coreStart, namespace });
const riskEngineDataClient = new RiskEngineDataClient({
logger,
kibanaVersion,
esClient,
namespace,
soClient,
});
return riskScoreServiceFactory({
esClient,
logger,
riskEngineDataClient,
spaceId: namespace,
});
});
taskManager.registerTaskDefinitions({
[getTaskName()]: {
title: 'Entity Analytics Risk Engine - Risk Scoring Task',
timeout: TIMEOUT,
stateSchemaByVersion,
createTaskRunner: createTaskRunnerFactory({ logger, getRiskScoreService }),
},
});
};
export const startRiskScoringTask = async ({
logger,
namespace,
riskEngineDataClient,
taskManager,
}: {
logger: Logger;
namespace: string;
riskEngineDataClient: RiskEngineDataClient;
taskManager: TaskManagerStartContract;
}) => {
const taskId = getTaskId(namespace);
const log = logFactory(logger, taskId);
log('starting task');
const interval = (await riskEngineDataClient.getConfiguration())?.interval ?? INTERVAL;
log('attempting to schedule');
try {
await taskManager.ensureScheduled({
id: taskId,
taskType: getTaskName(),
scope: SCOPE,
schedule: {
interval,
},
state: { ...defaultState, namespace },
params: { version: VERSION },
});
} catch (e) {
logger.warn(`[task ${taskId}]: error scheduling task, received ${e.message}`);
throw e;
}
};
export const removeRiskScoringTask = async ({
logger,
namespace,
taskManager,
}: {
logger: Logger;
namespace: string;
taskManager: TaskManagerStartContract;
}) => {
try {
await taskManager.remove(getTaskId(namespace));
} catch (err) {
if (!SavedObjectsErrorHelpers.isNotFoundError(err)) {
logger.error(`Failed to remove risk scoring task: ${err.message}`);
throw err;
}
}
};
export const runTask = async ({
getRiskScoreService,
logger,
taskInstance,
}: {
logger: Logger;
getRiskScoreService: GetRiskScoreService;
taskInstance: ConcreteTaskInstance;
}): Promise<{
state: RiskScoringTaskState;
}> => {
const state = taskInstance.state as RiskScoringTaskState;
const taskId = taskInstance.id;
const log = logFactory(logger, taskId);
const taskExecutionTime = moment().utc().toISOString();
log('running task');
let scoresWritten = 0;
const updatedState = {
lastExecutionTimestamp: taskExecutionTime,
namespace: state.namespace,
runs: state.runs + 1,
scoresWritten,
};
if (taskId !== getTaskId(state.namespace)) {
log('outdated task; exiting');
return { state: updatedState };
}
const riskScoreService = await getRiskScoreService(state.namespace);
if (!riskScoreService) {
log('risk score service is not available; exiting task');
return { state: updatedState };
}
const configuration = await riskScoreService.getConfiguration();
if (configuration == null) {
log(
'Risk engine configuration not found; exiting task. Please reinitialize the risk engine and try again'
);
return { state: updatedState };
}
const {
dataViewId,
enabled,
filter,
identifierType: configuredIdentifierType,
range: configuredRange,
pageSize,
} = configuration;
if (!enabled) {
log('risk engine is not enabled, exiting task');
return { state: updatedState };
}
const range = convertRangeToISO(configuredRange);
const { index, runtimeMappings } = await riskScoreService.getRiskInputsIndex({
dataViewId,
});
const identifierTypes: IdentifierType[] = configuredIdentifierType
? [configuredIdentifierType]
: [RiskScoreEntity.host, RiskScoreEntity.user];
await asyncForEach(identifierTypes, async (identifierType) => {
let isWorkComplete = false;
let afterKeys: AfterKeys = {};
while (!isWorkComplete) {
const result = await riskScoreService.calculateAndPersistScores({
afterKeys,
index,
filter,
identifierType,
pageSize,
range,
runtimeMappings,
weights: [],
});
isWorkComplete = isRiskScoreCalculationComplete(result);
afterKeys = result.after_keys;
scoresWritten += result.scores_written;
}
});
updatedState.scoresWritten = scoresWritten;
log('task run completed');
return {
state: updatedState,
};
};
const createTaskRunnerFactory =
({ logger, getRiskScoreService }: { logger: Logger; getRiskScoreService: GetRiskScoreService }) =>
({ taskInstance }: { taskInstance: ConcreteTaskInstance }) => {
return {
run: async () => runTask({ getRiskScoreService, logger, taskInstance }),
cancel: async () => {},
};
};

44
x-pack/plugins/security_solution/server/lib/risk_engine/tasks/state.ts Normal file
View file

@ -0,0 +1,44 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { schema, type TypeOf } from '@kbn/config-schema';
/**
* WARNING: Do not modify the existing versioned schema(s) below, instead define a new version (ex: 2, 3, 4).
* This is required to support zero-downtime upgrades and rollbacks. See https://github.com/elastic/kibana/issues/155764.
*
* As you add a new schema version, don't forget to change latestTaskStateSchema variable to reference the latest schema.
* For example, changing stateSchemaByVersion[1].schema to stateSchemaByVersion[2].schema.
*/
export const stateSchemaByVersion = {
1: {
// A task that was created < 8.10 will go through this "up" migration
// to ensure it matches the v1 schema.
up: (state: Record<string, unknown>) => ({
lastExecutionTimestamp: state.lastExecutionTimestamp || undefined,
runs: state.runs || 0,
namespace: typeof state.namespace === 'string' ? state.namespace : 'default',
scoresWritten: typeof state.scoresWritten === 'number' ? state.scoresWritten : undefined,
}),
schema: schema.object({
lastExecutionTimestamp: schema.maybe(schema.string()),
namespace: schema.string(),
runs: schema.number(),
scoresWritten: schema.maybe(schema.number()),
}),
},
};
const latestTaskStateSchema = stateSchemaByVersion[1].schema;
export type LatestTaskStateSchema = TypeOf<typeof latestTaskStateSchema>;
export const defaultState: LatestTaskStateSchema = {
lastExecutionTimestamp: undefined,
namespace: 'default',
runs: 0,
scoresWritten: undefined,
};

9
x-pack/plugins/security_solution/server/lib/risk_engine/types.ts
View file

@ -11,6 +11,7 @@ import type {
AfterKeys,
IdentifierType,
RiskWeights,
Range,
RiskEngineStatus,
RiskScore,
} from '../../../common/risk_engine';
@ -34,7 +35,7 @@ export interface CalculateAndPersistScoresParams {
filter?: unknown;
identifierType: IdentifierType;
pageSize: number;
range: { start: string; end: string };
range: Range;
runtimeMappings: MappingRuntimeFields;
weights?: RiskWeights;
}
@ -129,5 +130,11 @@ export interface RiskScoreBucket {
}
export interface RiskEngineConfiguration {
dataViewId: string;
enabled: boolean;
filter: unknown;
identifierType: IdentifierType | undefined;
interval: string;
pageSize: number;
range: Range;
}

30
x-pack/plugins/security_solution/server/lib/risk_engine/utils/saved_object_configuration.ts
View file

@ -6,6 +6,7 @@
*/
import type { SavedObject, SavedObjectsClientContract } from '@kbn/core/server';
import { getAlertsIndex } from '../../../../common/utils/risk_score_modules';
import type { RiskEngineConfiguration } from '../types';
import { riskEngineConfigurationTypeName } from '../saved_object';
@ -13,6 +14,20 @@ export interface SavedObjectsClientArg {
savedObjectsClient: SavedObjectsClientContract;
}
const getDefaultRiskEngineConfiguration = ({
namespace,
}: {
namespace: string;
}): RiskEngineConfiguration => ({
dataViewId: getAlertsIndex(namespace),
enabled: false,
filter: {},
identifierType: undefined,
interval: '1h',
pageSize: 10_000,
range: { start: 'now-30d', end: 'now' },
});
const getConfigurationSavedObject = async ({
savedObjectsClient,
}: SavedObjectsClientArg): Promise<SavedObject<RiskEngineConfiguration> | undefined> => {
@ -47,7 +62,7 @@ export const updateSavedObjectAttribute = async ({
});
if (!savedObjectConfiguration) {
throw new Error('There no saved object configuration for risk engine');
throw new Error('Risk engine configuration not found');
}
const result = await savedObjectsClient.update(
@ -64,14 +79,19 @@ export const updateSavedObjectAttribute = async ({
return result;
};
export const initSavedObjects = async ({ savedObjectsClient }: SavedObjectsClientArg) => {
export const initSavedObjects = async ({
namespace,
savedObjectsClient,
}: SavedObjectsClientArg & { namespace: string }) => {
const configuration = await getConfigurationSavedObject({ savedObjectsClient });
if (configuration) {
return configuration;
}
const result = await savedObjectsClient.create(riskEngineConfigurationTypeName, {
enabled: false,
});
const result = await savedObjectsClient.create(
riskEngineConfigurationTypeName,
getDefaultRiskEngineConfiguration({ namespace }),
{}
);
return result;
};

10
x-pack/plugins/security_solution/server/plugin.ts
View file

@ -97,6 +97,7 @@ import {
} from '../common/endpoint/constants';
import { AppFeatures } from './lib/app_features';
import { registerRiskScoringTask } from './lib/risk_engine/tasks/risk_scoring_task';
export type { SetupPlugins, StartPlugins, PluginSetup, PluginStart } from './plugin_contract';
@ -161,6 +162,15 @@ export class Plugin implements ISecuritySolutionPlugin {
this.ruleMonitoringService.setup(core, plugins);
if (experimentalFeatures.riskScoringPersistence) {
registerRiskScoringTask({
getStartServices: core.getStartServices,
kibanaVersion: pluginContext.env.packageInfo.version,
logger: this.logger,
taskManager: plugins.taskManager,
});
}
const requestContextFactory = new RequestContextFactory({
config,
logger,

8
x-pack/plugins/security_solution/server/routes/index.ts
View file

@ -183,9 +183,9 @@ export const initRoutes = (
if (config.experimentalFeatures.riskScoringRoutesEnabled) {
riskScorePreviewRoute(router, logger);
riskScoreCalculationRoute(router, logger);
riskEngineInitRoute(router, logger, security);
riskEngineEnableRoute(router, logger, security);
riskEngineStatusRoute(router, logger);
riskEngineDisableRoute(router, logger, security);
riskEngineStatusRoute(router);
riskEngineInitRoute(router, getStartServices);
riskEngineEnableRoute(router, getStartServices);
riskEngineDisableRoute(router, getStartServices);
}
};

3
x-pack/test/detection_engine_api_integration/security_and_spaces/group10/index.ts
View file

@ -37,9 +37,10 @@ export default ({ loadTestFile }: FtrProviderContext): void => {
loadTestFile(require.resolve('./throttle'));
loadTestFile(require.resolve('./ignore_fields'));
loadTestFile(require.resolve('./migrations'));
loadTestFile(require.resolve('./risk_engine/risk_engine_status'));
loadTestFile(require.resolve('./risk_engine/init_and_status_apis'));
loadTestFile(require.resolve('./risk_engine/risk_score_preview'));
loadTestFile(require.resolve('./risk_engine/risk_score_calculation'));
loadTestFile(require.resolve('./risk_engine/risk_scoring_task_execution'));
loadTestFile(require.resolve('./set_alert_tags'));
});
};

67
x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/risk_engine_status.ts → x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/init_and_status_apis.ts
View file

@ -6,19 +6,14 @@
*/
import expect from '@kbn/expect';
import {
RISK_ENGINE_INIT_URL,
RISK_ENGINE_DISABLE_URL,
RISK_ENGINE_ENABLE_URL,
RISK_ENGINE_STATUS_URL,
} from '@kbn/security-solution-plugin/common/constants';
import { riskEngineConfigurationTypeName } from '@kbn/security-solution-plugin/server/lib/risk_engine/saved_object';
import { FtrProviderContext } from '../../../common/ftr_provider_context';
import {
cleanRiskEngineConfig,
legacyTransformIds,
createTransforms,
createLegacyTransforms,
clearLegacyTransforms,
riskEngineRouteHelpersFactory,
clearTransforms,
} from './utils';
@ -27,6 +22,7 @@ export default ({ getService }: FtrProviderContext) => {
const es = getService('es');
const supertest = getService('supertest');
const kibanaServer = getService('kibanaServer');
const riskEngineRoutes = riskEngineRouteHelpersFactory(supertest);
const log = getService('log');
describe('Risk Engine', () => {
@ -44,21 +40,9 @@ export default ({ getService }: FtrProviderContext) => {
});
});
const initRiskEngine = async () =>
await supertest.post(RISK_ENGINE_INIT_URL).set('kbn-xsrf', 'true').send().expect(200);
const getRiskEngineStatus = async () =>
await supertest.get(RISK_ENGINE_STATUS_URL).set('kbn-xsrf', 'true').send().expect(200);
const enableRiskEngine = async () =>
await supertest.post(RISK_ENGINE_ENABLE_URL).set('kbn-xsrf', 'true').send().expect(200);
const disableRiskEngine = async () =>
await supertest.post(RISK_ENGINE_DISABLE_URL).set('kbn-xsrf', 'true').send().expect(200);
describe('init api', () => {
it('should return response with success status', async () => {
const response = await initRiskEngine();
const response = await riskEngineRoutes.init();
expect(response.body).to.eql({
result: {
errors: [],
@ -78,7 +62,7 @@ export default ({ getService }: FtrProviderContext) => {
const latestIndexName = 'risk-score.risk-score-latest-default';
const transformId = 'risk_score_latest_transform_default';
await initRiskEngine();
await riskEngineRoutes.init();
const ilmPolicy = await es.ilm.getLifecycle({
name: ilmPolicyName,
@ -308,23 +292,31 @@ export default ({ getService }: FtrProviderContext) => {
});
it('should create configuration saved object', async () => {
await initRiskEngine();
await riskEngineRoutes.init();
const response = await kibanaServer.savedObjects.find({
type: riskEngineConfigurationTypeName,
});
expect(response?.saved_objects?.[0]?.attributes).to.eql({
dataViewId: '.alerts-security.alerts-default',
enabled: true,
filter: {},
interval: '1h',
pageSize: 10000,
range: {
end: 'now',
start: 'now-30d',
},
});
});
it('should create configuration saved object only once', async () => {
await initRiskEngine();
await riskEngineRoutes.init();
const firstResponse = await kibanaServer.savedObjects.find({
type: riskEngineConfigurationTypeName,
});
await initRiskEngine();
await riskEngineRoutes.init();
const secondResponse = await kibanaServer.savedObjects.find({
type: riskEngineConfigurationTypeName,
});
@ -336,7 +328,7 @@ export default ({ getService }: FtrProviderContext) => {
});
it('should remove legacy risk score transform if it exists', async () => {
await createTransforms({ es });
await createLegacyTransforms({ es });
for (const transformId of legacyTransformIds) {
const tr = await es.transform.getTransform({
@ -346,7 +338,7 @@ export default ({ getService }: FtrProviderContext) => {
expect(tr?.transforms?.[0]?.id).to.eql(transformId);
}
await initRiskEngine();
await riskEngineRoutes.init();
for (const transformId of legacyTransformIds) {
try {
@ -362,7 +354,8 @@ export default ({ getService }: FtrProviderContext) => {
describe('status api', () => {
it('should disable / enable risk engige', async () => {
const status1 = await getRiskEngineStatus();
const status1 = await riskEngineRoutes.getStatus();
await riskEngineRoutes.init();
expect(status1.body).to.eql({
risk_engine_status: 'NOT_INSTALLED',
@ -370,9 +363,9 @@ export default ({ getService }: FtrProviderContext) => {
is_max_amount_of_risk_engines_reached: false,
});
await initRiskEngine();
await riskEngineRoutes.init();
const status2 = await getRiskEngineStatus();
const status2 = await riskEngineRoutes.getStatus();
expect(status2.body).to.eql({
risk_engine_status: 'ENABLED',
@ -380,8 +373,8 @@ export default ({ getService }: FtrProviderContext) => {
is_max_amount_of_risk_engines_reached: false,
});
await disableRiskEngine();
const status3 = await getRiskEngineStatus();
await riskEngineRoutes.disable();
const status3 = await riskEngineRoutes.getStatus();
expect(status3.body).to.eql({
risk_engine_status: 'DISABLED',
@ -389,8 +382,8 @@ export default ({ getService }: FtrProviderContext) => {
is_max_amount_of_risk_engines_reached: false,
});
await enableRiskEngine();
const status4 = await getRiskEngineStatus();
await riskEngineRoutes.enable();
const status4 = await riskEngineRoutes.getStatus();
expect(status4.body).to.eql({
risk_engine_status: 'ENABLED',
@ -400,8 +393,8 @@ export default ({ getService }: FtrProviderContext) => {
});
it('should return status of legacy risk engine', async () => {
await createTransforms({ es });
const status1 = await getRiskEngineStatus();
await createLegacyTransforms({ es });
const status1 = await riskEngineRoutes.getStatus();
expect(status1.body).to.eql({
risk_engine_status: 'NOT_INSTALLED',
@ -409,9 +402,9 @@ export default ({ getService }: FtrProviderContext) => {
is_max_amount_of_risk_engines_reached: false,
});
await initRiskEngine();
await riskEngineRoutes.init();
const status2 = await getRiskEngineStatus();
const status2 = await riskEngineRoutes.getStatus();
expect(status2.body).to.eql({
risk_engine_status: 'ENABLED',

13
x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/risk_score_calculation.ts
View file

@ -62,7 +62,7 @@ export default ({ getService }: FtrProviderContext): void => {
});
};
describe('Risk Engine Scoring - Calculation', () => {
describe('Risk Engine - Risk Scoring Calculation API', () => {
context('with auditbeat data', () => {
const { indexListOfDocuments } = dataGeneratorFactory({
es,
@ -106,7 +106,7 @@ export default ({ getService }: FtrProviderContext): void => {
scores_written: 1,
});
await waitForRiskScoresToBePresent(es, log);
await waitForRiskScoresToBePresent({ es, log });
const scores = await readRiskScores(es);
expect(scores.length).to.eql(1);
@ -123,8 +123,7 @@ export default ({ getService }: FtrProviderContext): void => {
]);
});
// FLAKY: https://github.com/elastic/kibana/issues/162736
describe.skip('paging through calculationss', () => {
describe('paging through calculations', () => {
let documentId: string;
beforeEach(async () => {
documentId = uuidv4();
@ -163,7 +162,7 @@ export default ({ getService }: FtrProviderContext): void => {
scores_written: 10,
});
await waitForRiskScoresToBePresent(es, log);
await waitForRiskScoresToBePresent({ es, log, scoreCount: 10 });
const scores = await readRiskScores(es);
expect(scores.length).to.eql(10);
@ -212,7 +211,7 @@ export default ({ getService }: FtrProviderContext): void => {
scores_written: 5,
});
await waitForRiskScoresToBePresent(es, log);
await waitForRiskScoresToBePresent({ es, log, scoreCount: 10 });
const scores = await readRiskScores(es);
expect(scores.length).to.eql(10);
@ -258,7 +257,7 @@ export default ({ getService }: FtrProviderContext): void => {
scores_written: 0,
});
await waitForRiskScoresToBePresent(es, log);
await waitForRiskScoresToBePresent({ es, log, scoreCount: 10 });
const scores = await readRiskScores(es);
expect(scores.length).to.eql(10);

2
x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/risk_score_preview.ts
View file

@ -55,7 +55,7 @@ export default ({ getService }: FtrProviderContext): void => {
return await previewRiskScores({ body: {} });
};
describe('Risk Engine Scoring - Preview', () => {
describe('Risk Engine - Risk Scoring Preview API', () => {
context('with auditbeat data', () => {
const { indexListOfDocuments } = dataGeneratorFactory({
es,

293
x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/risk_scoring_task_execution.ts Normal file
View file

@ -0,0 +1,293 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import expect from '@kbn/expect';
import { v4 as uuidv4 } from 'uuid';
import { FtrProviderContext } from '../../../common/ftr_provider_context';
import { deleteAllAlerts, deleteAllRules } from '../../../utils';
import { dataGeneratorFactory } from '../../../utils/data_generator';
import {
buildDocument,
createAndSyncRuleAndAlertsFactory,
deleteRiskEngineTask,
deleteAllRiskScores,
readRiskScores,
waitForRiskScoresToBePresent,
normalizeScores,
riskEngineRouteHelpersFactory,
updateRiskEngineConfigSO,
getRiskEngineTask,
cleanRiskEngineConfig,
waitForRiskEngineTaskToBeGone,
} from './utils';
// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext): void => {
const supertest = getService('supertest');
const esArchiver = getService('esArchiver');
const es = getService('es');
const log = getService('log');
const kibanaServer = getService('kibanaServer');
const createAndSyncRuleAndAlerts = createAndSyncRuleAndAlertsFactory({ supertest, log });
const riskEngineRoutes = riskEngineRouteHelpersFactory(supertest);
describe('Risk Engine - Risk Scoring Task', () => {
context('with auditbeat data', () => {
const { indexListOfDocuments } = dataGeneratorFactory({
es,
index: 'ecs_compliant',
log,
});
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/security_solution/ecs_compliant');
});
after(async () => {
await esArchiver.unload(
'x-pack/test/functional/es_archives/security_solution/ecs_compliant'
);
});
beforeEach(async () => {
await cleanRiskEngineConfig({ kibanaServer });
await deleteRiskEngineTask({ es, log });
await deleteAllRiskScores(log, es);
await deleteAllAlerts(supertest, log, es);
await deleteAllRules(supertest, log);
});
afterEach(async () => {
await cleanRiskEngineConfig({ kibanaServer });
await deleteRiskEngineTask({ es, log });
await deleteAllRiskScores(log, es);
await deleteAllAlerts(supertest, log, es);
await deleteAllRules(supertest, log);
});
describe('with some alerts containing hosts', () => {
let documentId: string;
beforeEach(async () => {
documentId = uuidv4();
const baseEvent = buildDocument({ host: { name: 'host-1' } }, documentId);
await indexListOfDocuments(
Array(10)
.fill(baseEvent)
.map((_baseEvent, index) => ({
..._baseEvent,
'host.name': `host-${index}`,
}))
);
await createAndSyncRuleAndAlerts({
query: `id: ${documentId}`,
alerts: 10,
riskScore: 40,
});
});
describe('initializing the risk engine', () => {
beforeEach(async () => {
await riskEngineRoutes.init();
});
it('calculates and persists risk scores for alert documents', async () => {
await waitForRiskScoresToBePresent({ es, log, scoreCount: 10 });
const scores = await readRiskScores(es);
expect(normalizeScores(scores).map(({ id_value: idValue }) => idValue)).to.eql(
Array(10)
.fill(0)
.map((_, index) => `host-${index}`)
);
});
describe('disabling and re-enabling the risk engine', () => {
beforeEach(async () => {
await waitForRiskScoresToBePresent({ es, log, scoreCount: 10 });
await riskEngineRoutes.disable();
await riskEngineRoutes.enable();
});
it('calculates another round of scores', async () => {
await waitForRiskScoresToBePresent({ es, log, scoreCount: 20 });
const scores = await readRiskScores(es);
const expectedHostNames = Array(10)
.fill(0)
.map((_, index) => `host-${index}`);
const actualHostNames = normalizeScores(scores).map(
({ id_value: idValue }) => idValue
);
expect(actualHostNames).to.eql([...expectedHostNames, ...expectedHostNames]);
});
});
describe('disabling the risk engine', () => {
beforeEach(async () => {
await waitForRiskScoresToBePresent({ es, log, scoreCount: 10 });
});
it('removes the risk scoring task', async () => {
const task = await getRiskEngineTask({ es });
expect(task).not.to.be(undefined);
await riskEngineRoutes.disable();
await waitForRiskEngineTaskToBeGone({ es, log });
const disabledTask = await getRiskEngineTask({ es });
expect(disabledTask).to.eql(undefined);
});
});
describe('modifying configuration', () => {
beforeEach(async () => {
await riskEngineRoutes.disable();
});
describe('when task interval is modified', () => {
beforeEach(async () => {
await updateRiskEngineConfigSO({
attributes: {
interval: '1s',
},
kibanaServer,
});
await riskEngineRoutes.enable();
});
it('executes multiple times', async () => {
await waitForRiskScoresToBePresent({ es, log, scoreCount: 30 });
const riskScores = await readRiskScores(es);
expect(riskScores.length).to.be.greaterThan(29);
});
});
});
});
});
describe('with some alerts containing hosts and others containing users', () => {
let hostId: string;
let userId: string;
beforeEach(async () => {
hostId = uuidv4();
const hostEvent = buildDocument({ host: { name: 'host-1' } }, hostId);
await indexListOfDocuments(
Array(10)
.fill(hostEvent)
.map((event, index) => ({
...event,
'host.name': `host-${index}`,
}))
);
userId = uuidv4();
const userEvent = buildDocument({ user: { name: 'user-1' } }, userId);
await indexListOfDocuments(
Array(10)
.fill(userEvent)
.map((event, index) => ({
...event,
'user.name': `user-${index}`,
}))
);
await createAndSyncRuleAndAlerts({
query: `id: ${userId} or id: ${hostId}`,
alerts: 20,
riskScore: 40,
});
await riskEngineRoutes.init();
});
it('calculates and persists risk scores for both types of entities', async () => {
await waitForRiskScoresToBePresent({ es, log, scoreCount: 20 });
const riskScores = await readRiskScores(es);
expect(riskScores.length).to.eql(20);
const scoredIdentifiers = normalizeScores(riskScores).map(
({ id_field: idField }) => idField
);
expect(scoredIdentifiers.includes('host.name')).to.be(true);
expect(scoredIdentifiers.includes('user.name')).to.be(true);
});
});
describe('with alerts in a non-default space', () => {
let namespace: string;
let index: string[];
let documentId: string;
let createAndSyncRuleAndAlertsForOtherSpace: ReturnType<
typeof createAndSyncRuleAndAlertsFactory
>;
beforeEach(async () => {
documentId = uuidv4();
namespace = uuidv4();
index = [`risk-score.risk-score-${namespace}`];
createAndSyncRuleAndAlertsForOtherSpace = createAndSyncRuleAndAlertsFactory({
supertest,
log,
namespace,
});
const riskEngineRoutesForNamespace = riskEngineRouteHelpersFactory(supertest, namespace);
const spaces = getService('spaces');
await spaces.create({
id: namespace,
name: namespace,
disabledFeatures: [],
});
const baseEvent = buildDocument({ host: { name: 'host-1' } }, documentId);
await indexListOfDocuments(
Array(10)
.fill(baseEvent)
.map((_baseEvent, _index) => ({
..._baseEvent,
'host.name': `host-${_index}`,
}))
);
await createAndSyncRuleAndAlertsForOtherSpace({
query: `id: ${documentId}`,
alerts: 10,
riskScore: 40,
});
await riskEngineRoutesForNamespace.init();
});
afterEach(async () => {
await getService('spaces').delete(namespace);
});
it('calculates and persists risk scores for alert documents', async () => {
await waitForRiskScoresToBePresent({
es,
log,
scoreCount: 10,
index,
});
const scores = await readRiskScores(es, index);
expect(normalizeScores(scores).map(({ id_value: idValue }) => idValue)).to.eql(
Array(10)
.fill(0)
.map((_, _index) => `host-${_index}`)
);
});
});
});
});
};

230
x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/utils.ts
View file

@ -12,6 +12,12 @@ import type { ToolingLog } from '@kbn/tooling-log';
import type { EcsRiskScore, RiskScore } from '@kbn/security-solution-plugin/common/risk_engine';
import { riskEngineConfigurationTypeName } from '@kbn/security-solution-plugin/server/lib/risk_engine/saved_object';
import type { KbnClient } from '@kbn/test';
import {
RISK_ENGINE_INIT_URL,
RISK_ENGINE_DISABLE_URL,
RISK_ENGINE_ENABLE_URL,
RISK_ENGINE_STATUS_URL,
} from '@kbn/security-solution-plugin/common/constants';
import {
createRule,
waitForSignalsToBePresent,
@ -19,6 +25,7 @@ import {
getRuleForSignalTesting,
countDownTest,
waitFor,
routeWithNamespace,
} from '../../../utils';
const sanitizeScore = (score: Partial<RiskScore>): Partial<RiskScore> => {
@ -49,7 +56,15 @@ export const buildDocument = (body: object, id?: string) => {
};
export const createAndSyncRuleAndAlertsFactory =
({ supertest, log }: { supertest: SuperTest.SuperTest<SuperTest.Test>; log: ToolingLog }) =>
({
supertest,
log,
namespace,
}: {
supertest: SuperTest.SuperTest<SuperTest.Test>;
log: ToolingLog;
namespace?: string;
}) =>
async ({
alerts = 1,
riskScore = 21,
@ -64,21 +79,26 @@ export const createAndSyncRuleAndAlertsFactory =
riskScoreOverride?: string;
}): Promise<void> => {
const rule = getRuleForSignalTesting(['ecs_compliant']);
const { id } = await createRule(supertest, log, {
...rule,
risk_score: riskScore,
query,
max_signals: maxSignals,
...(riskScoreOverride
? {
risk_score_mapping: [
{ field: riskScoreOverride, operator: 'equals', value: '', risk_score: undefined },
],
}
: {}),
});
await waitForRuleSuccess({ supertest, log, id });
await waitForSignalsToBePresent(supertest, log, alerts, [id]);
const { id } = await createRule(
supertest,
log,
{
...rule,
risk_score: riskScore,
query,
max_signals: maxSignals,
...(riskScoreOverride
? {
risk_score_mapping: [
{ field: riskScoreOverride, operator: 'equals', value: '', risk_score: undefined },
],
}
: {}),
},
namespace
);
await waitForRuleSuccess({ supertest, log, id, namespace });
await waitForSignalsToBePresent(supertest, log, alerts, [id], namespace);
};
/**
@ -99,6 +119,7 @@ export const deleteAllRiskScores = async (
match_all: {},
},
},
ignore_unavailable: true,
refresh: true,
});
return {
@ -110,31 +131,133 @@ export const deleteAllRiskScores = async (
);
};
/**
* Function to read risk scores from ES. By default, it reads from the risk
* score datastream in the default space, but this can be overridden with the
* `index` parameter.
*
* @param {string[]} index - the index or indices to read risk scores from.
* @param {number} size - the size parameter of the query
*/
export const readRiskScores = async (
es: Client,
index: string[] = ['risk-score.risk-score-default']
index: string[] = ['risk-score.risk-score-default'],
size: number = 1000
): Promise<EcsRiskScore[]> => {
const results = await es.search({
index: 'risk-score.risk-score-default',
index,
size,
});
return results.hits.hits.map((hit) => hit._source as EcsRiskScore);
};
export const waitForRiskScoresToBePresent = async (
es: Client,
log: ToolingLog,
index: string[] = ['risk-score.risk-score-default']
): Promise<void> => {
/**
* Function to read risk scores from ES and wait for them to be
* present/readable. By default, it reads from the risk score datastream in the
* default space, but this can be overridden with the `index` parameter.
*
* @param {string[]} index - the index or indices to read risk scores from.
* @param {number} scoreCount - the number of risk scores to wait for. Defaults to 1.
*/
export const waitForRiskScoresToBePresent = async ({
es,
log,
index = ['risk-score.risk-score-default'],
scoreCount = 1,
}: {
es: Client;
log: ToolingLog;
index?: string[];
scoreCount?: number;
}): Promise<void> => {
await waitFor(
async () => {
const riskScores = await readRiskScores(es, index);
return riskScores.length > 0;
const riskScores = await readRiskScores(es, index, scoreCount + 10);
return riskScores.length >= scoreCount;
},
'waitForRiskScoresToBePresent',
log
);
};
export const getRiskEngineTasks = async ({
es,
index = ['.kibana_task_manager*'],
}: {
es: Client;
index?: string[];
}) => {
const result = await es.search({
index,
query: { match: { 'task.taskType': 'risk_engine:risk_scoring' } },
});
return result.hits.hits?.map((hit) => hit._source);
};
export const getRiskEngineTask = async ({
es,
index = ['.kibana_task_manager*'],
}: {
es: Client;
index?: string[];
}) => {
const result = await es.search({
index,
query: { match: { 'task.taskType': 'risk_engine:risk_scoring' } },
});
return result.hits.hits[0]?._source;
};
export const deleteRiskEngineTask = async ({
es,
log,
index = ['.kibana_task_manager*'],
}: {
es: Client;
log: ToolingLog;
index?: string[];
}) => {
await countDownTest(
async () => {
await es.deleteByQuery({
index,
query: {
match: {
'task.taskType': 'risk_engine:risk_scoring',
},
},
conflicts: 'proceed',
});
return {
passed: true,
};
},
'deleteRiskEngineTask',
log
);
};
export const waitForRiskEngineTaskToBeGone = async ({
es,
log,
index = ['.kibana_task_manager*'],
}: {
es: Client;
log: ToolingLog;
index?: string[];
}): Promise<void> => {
await waitFor(
async () => {
const task = await getRiskEngineTask({ es, index });
return task == null;
},
'waitForRiskEngineTaskToBeGone',
log
);
};
export const getRiskEngineConfigSO = async ({ kibanaServer }: { kibanaServer: KbnClient }) => {
const soResponse = await kibanaServer.savedObjects.find({
type: riskEngineConfigurationTypeName,
@ -157,6 +280,28 @@ export const cleanRiskEngineConfig = async ({
}
};
export const updateRiskEngineConfigSO = async ({
attributes,
kibanaServer,
}: {
attributes: object;
kibanaServer: KbnClient;
}) => {
const so = await getRiskEngineConfigSO({ kibanaServer });
if (so) {
await kibanaServer.savedObjects.update({
id: so.id,
type: riskEngineConfigurationTypeName,
attributes: {
...so.attributes,
...attributes,
},
});
} else {
throw Error('No risk engine config found');
}
};
export const legacyTransformIds = [
'ml_hostriskscore_pivot_transform_default',
'ml_hostriskscore_latest_transform_default',
@ -201,7 +346,7 @@ export const clearLegacyTransforms = async ({
}
};
export const createTransforms = async ({ es }: { es: Client }): Promise<void> => {
export const createLegacyTransforms = async ({ es }: { es: Client }): Promise<void> => {
const transforms = legacyTransformIds.map((transform) =>
es.transform.putTransform({
transform_id: transform,
@ -233,3 +378,36 @@ export const createTransforms = async ({ es }: { es: Client }): Promise<void> =>
await Promise.all(transforms);
};
export const riskEngineRouteHelpersFactory = (
supertest: SuperTest.SuperTest<SuperTest.Test>,
namespace?: string
) => ({
init: async () =>
await supertest
.post(routeWithNamespace(RISK_ENGINE_INIT_URL, namespace))
.set('kbn-xsrf', 'true')
.send()
.expect(200),
getStatus: async () =>
await supertest
.get(routeWithNamespace(RISK_ENGINE_STATUS_URL, namespace))
.set('kbn-xsrf', 'true')
.send()
.expect(200),
enable: async () =>
await supertest
.post(routeWithNamespace(RISK_ENGINE_ENABLE_URL, namespace))
.set('kbn-xsrf', 'true')
.send()
.expect(200),
disable: async () =>
await supertest
.post(routeWithNamespace(RISK_ENGINE_DISABLE_URL, namespace))
.set('kbn-xsrf', 'true')
.send()
.expect(200),
});

7
x-pack/test/detection_engine_api_integration/utils/create_rule.ts
View file

@ -14,6 +14,7 @@ import type {
import { DETECTION_ENGINE_RULES_URL } from '@kbn/security-solution-plugin/common/constants';
import { deleteRule } from './delete_rule';
import { routeWithNamespace } from './route_with_namespace';
/**
* Helper to cut down on the noise in some of the tests. If this detects
@ -27,10 +28,12 @@ import { deleteRule } from './delete_rule';
export const createRule = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
rule: RuleCreateProps
rule: RuleCreateProps,
namespace?: string
): Promise<RuleResponse> => {
const route = routeWithNamespace(DETECTION_ENGINE_RULES_URL, namespace);
const response = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.post(route)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.send(rule);

7
x-pack/test/detection_engine_api_integration/utils/get_signals_by_ids.ts
View file

@ -14,6 +14,7 @@ import type { RiskEnrichmentFields } from '@kbn/security-solution-plugin/server/
import { DETECTION_ENGINE_QUERY_SIGNALS_URL } from '@kbn/security-solution-plugin/common/constants';
import { countDownTest } from './count_down_test';
import { getQuerySignalsId } from './get_query_signals_ids';
import { routeWithNamespace } from './route_with_namespace';
/**
* Given an array of rule ids this will return only signals based on that rule id both
@ -25,12 +26,14 @@ export const getSignalsByIds = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
ids: string[],
size?: number
size?: number,
namespace?: string
): Promise<SearchResponse<DetectionAlert & RiskEnrichmentFields>> => {
const signalsOpen = await countDownTest<SearchResponse<DetectionAlert & RiskEnrichmentFields>>(
async () => {
const route = routeWithNamespace(DETECTION_ENGINE_QUERY_SIGNALS_URL, namespace);
const response = await supertest
.post(DETECTION_ENGINE_QUERY_SIGNALS_URL)
.post(route)
.set('kbn-xsrf', 'true')
.send(getQuerySignalsId(ids, size));
if (response.status !== 200) {

1
x-pack/test/detection_engine_api_integration/utils/index.ts
View file

@ -83,6 +83,7 @@ export * from './perform_search_query';
export * from './preview_rule_with_exception_entries';
export * from './preview_rule';
export * from './refresh_index';
export * from './route_with_namespace';
export * from './remove_time_fields_from_telemetry_stats';
export * from './remove_server_generated_properties';
export * from './remove_server_generated_properties_including_rule_id';

14
x-pack/test/detection_engine_api_integration/utils/route_with_namespace.ts Normal file
View file

@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
/**
* Generates a route string with an optional namespace.
* @param route the route string
* @param namespace [optional] the namespace to account for in the route
*/
export const routeWithNamespace = (route: string, namespace?: string) =>
namespace ? `/s/${namespace}${route}` : route;

7
x-pack/test/detection_engine_api_integration/utils/wait_for_rule_status.ts
View file

@ -10,11 +10,13 @@ import type SuperTest from 'supertest';
import { DETECTION_ENGINE_RULES_URL } from '@kbn/security-solution-plugin/common/constants';
import { RuleExecutionStatus } from '@kbn/security-solution-plugin/common/api/detection_engine/rule_monitoring';
import { waitFor } from './wait_for';
import { routeWithNamespace } from './route_with_namespace';
interface WaitForRuleStatusBaseParams {
supertest: SuperTest.SuperTest<SuperTest.Test>;
log: ToolingLog;
afterDate?: Date;
namespace?: string;
}
interface WaitForRuleStatusWithId extends WaitForRuleStatusBaseParams {
@ -38,13 +40,14 @@ export type WaitForRuleStatusParams = WaitForRuleStatusWithId | WaitForRuleStatu
*/
export const waitForRuleStatus = async (
expectedStatus: RuleExecutionStatus,
{ supertest, log, afterDate, ...idOrRuleId }: WaitForRuleStatusParams
{ supertest, log, afterDate, namespace, ...idOrRuleId }: WaitForRuleStatusParams
): Promise<void> => {
await waitFor(
async () => {
const query = 'id' in idOrRuleId ? { id: idOrRuleId.id } : { rule_id: idOrRuleId.ruleId };
const route = routeWithNamespace(DETECTION_ENGINE_RULES_URL, namespace);
const response = await supertest
.get(DETECTION_ENGINE_RULES_URL)
.get(route)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.query(query)

11
x-pack/test/detection_engine_api_integration/utils/wait_for_signals_to_be_present.ts
View file

@ -21,11 +21,18 @@ export const waitForSignalsToBePresent = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
numberOfSignals = 1,
signalIds: string[]
signalIds: string[],
namespace?: string
): Promise<void> => {
await waitFor(
async () => {
const signalsOpen = await getSignalsByIds(supertest, log, signalIds, numberOfSignals);
const signalsOpen = await getSignalsByIds(
supertest,
log,
signalIds,
numberOfSignals,
namespace
);
return signalsOpen.hits.hits.length >= numberOfSignals;
},
'waitForSignalsToBePresent',

9
x-pack/test/security_solution_cypress/cypress/e2e/entity_analytics/entity_analytics_management_page.cy.ts
View file

@ -21,9 +21,9 @@ import {
import { deleteRiskScore, installRiskScoreModule } from '../../tasks/api_calls/risk_scores';
import { RiskScoreEntity } from '../../tasks/risk_scores/common';
import { login, visit, visitWithoutDateRange } from '../../tasks/login';
import { login, visit } from '../../tasks/login';
import { cleanKibana } from '../../tasks/common';
import { ENTITY_ANALYTICS_MANAGEMENT_URL, ALERTS_URL } from '../../urls/navigation';
import { ENTITY_ANALYTICS_MANAGEMENT_URL } from '../../urls/navigation';
import { getNewRule } from '../../objects/rule';
import { createRule } from '../../tasks/api_calls/rules';
import {
@ -44,7 +44,9 @@ import {
describe(
'Entity analytics management page',
{
env: { ftrConfig: { enableExperimental: ['riskScoringRoutesEnabled'] } },
env: {
ftrConfig: { enableExperimental: ['riskScoringRoutesEnabled', 'riskScoringPersistence'] },
},
tags: [tag.ESS, tag.BROKEN_IN_SERVERLESS],
},
() => {
@ -55,7 +57,6 @@ describe(
beforeEach(() => {
login();
visitWithoutDateRange(ALERTS_URL);
createRule(getNewRule({ query: 'user.name:* or host.name:*', risk_score: 70 }));
deleteConfiguration();
visit(ENTITY_ANALYTICS_MANAGEMENT_URL);