github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 01:38:56 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Security Solution][Detection Engine] Running API tests in Serverless using Mocha Tagging (#166755)

Browse source 
# Summary

- Addresses https://github.com/elastic/kibana/issues/161537

## Description 

- This PR follows the second option defined in this
[document](https://docs.google.com/document/d/1mqkpjDdFQRFvx_RPvNmjstVj8SXYMr2mrETMv3esda8/edit#heading=h.rpv1zyeb04ay)
the [Mocha tagging ](https://github.com/mochajs/mocha/wiki/Tagging)
- It introduces a new folder
`x-pack/test/security_solution_api_integration` which will serve as a
centralized location to meet all the requirements related to renaming
tests appropriately and grouping similar tests together. It will
facilitate the management of tests that must be run in Serverless and
ESS environments.

- Within this folder, there is a "config" subdirectory that stores base
configurations specific to both the
[Serverless](https://github.com/elastic/kibana/pull/166755/files#diff-afe1f42d5ac2006de8dc09069448b9e8734a6a950586376cd6e8eeb9110ab5f1R1)
and
[ESS](https://github.com/elastic/kibana/pull/166755/files#diff-4a60bd8c91da08a3f7ec14bf3bfef8449af155611374c32579b0318da03e292cR1)
environments, These configurations build upon the base configuration
provided by test_serverless and api_integrations, incorporating
additional settings such as environment variables and tagging options.

- It demonstrates scenarios involving `@ess`, `@serverless`, and
`@brokenInServerless`.


- The file`
x-pack/test/security_solution_api_integration/test_suites/detections_response/rule_creation/create_rules.ts`
is functional in both **Serverless** and **ESS**. However, some tests
related to roles are currently skipped for Serverless until they are
resolved, and these tests are tagged with `@brokenInServerless`

![image](93ad5698-8776-40c4-875d-3308fedd11cb)


## CI 

- It includes a new entry in the ftr_configs.yml to execute the newly
added tests in the pipeline.


- It involves the addition of mochaOptions in both
serverless/config.base.ts and ess/config.base.ts. In the case of
serverless, it includes **@serverless** while excluding
@brokenInServerless. Similarly, for **ess**, it includes @ess and
excludes **@brokenInEss**.

from `x-pack/test/security_solution_api_integration/config/serverless` 

![image](9413ba0f-0384-4125-a1a9-7108211f4848)


## Update in x-pack/test/detection_engine_api_integration

- The `create_rules.ts` and `create_rule_exceptions` files have been
relocated from
`x-pack/test/detection_engine_api_integration/security_and_spaces/group1`
to their respective domains within the
`x-pack/test/security_solution_api_integration` folder.

- The util files now are copied over from the old folder
`x-pack/test/detection_engine_api_integration` to the new folder and
will be removed once all tests are moved to the new folder to don't
break the existing tests

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Wafaa Nasr 2023-10-11 12:38:14 +02:00 • committed by GitHub
parent ef109cf5c8
commit 650c156b76
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
57 changed files with 4356 additions and 61 deletions
Download patch file Download diff file Expand all files Collapse all files

9
.buildkite/ftr_configs.yml
View file

@ -13,6 +13,9 @@ disabled:
- x-pack/test/functional_with_es_ssl/config.base.ts
- x-pack/test/api_integration/config.ts
- x-pack/test/fleet_api_integration/config.base.ts
- x-pack/test/security_solution_api_integration/config/ess/config.base.ts
- x-pack/test/security_solution_api_integration/config/serverless/config.base.ts
# QA suites that are run out-of-band
- x-pack/test/stack_functional_integration/configs/config.stack_functional_integration_base.js
@ -449,3 +452,9 @@ enabled:
- x-pack/performance/journeys/apm_service_inventory.ts
- x-pack/test/custom_branding/config.ts
- x-pack/test/profiling_api_integration/cloud/config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/serverless.config.ts
- x-pack/test/security_solution_api_integration/test_suites/detections_response/ess.config.ts

2
.eslintrc.js
View file

@ -606,6 +606,8 @@ module.exports = {
'x-pack/test_serverless/**/config*.ts',
'x-pack/test_serverless/*/test_suites/**/*',
'x-pack/test/profiling_api_integration/**/*.ts',
'x-pack/test/security_solution_api_integration/*/test_suites/**/*',
'x-pack/test/security_solution_api_integration/**/config*.ts',
],
rules: {
'import/no-default-export': 'off',

4
.github/CODEOWNERS vendored
View file

@ -1302,6 +1302,8 @@ x-pack/plugins/cloud_integrations/cloud_full_story/server/config.ts @elastic/kib
/x-pack/test/security_solution_cypress/cypress/e2e/entity_analytics @elastic/security-detection-engine
/x-pack/test/security_solution_cypress/cypress/e2e/exceptions @elastic/security-detection-engine
/x-pack/test/security_solution_cypress/cypress/e2e/overview @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/exceptions @elastic/security-detection-engine
x-pack/test/security_solution_api_integration/test_suites/detections_response/rule_creation @elastic/security-detection-engine
## Security Threat Intelligence - Under Security Platform
/x-pack/plugins/security_solution/public/common/components/threat_match @elastic/security-detection-engine
@ -1323,6 +1325,8 @@ x-pack/plugins/cloud_integrations/cloud_full_story/server/config.ts @elastic/kib
/x-pack/plugins/security_solution/server/routes @elastic/security-detections-response @elastic/security-threat-hunting
/x-pack/plugins/security_solution/server/utils @elastic/security-detections-response @elastic/security-threat-hunting
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils @elastic/security-detections-response
## Security Solution sub teams - security-defend-workflows
/x-pack/plugins/security_solution/public/management/ @elastic/security-defend-workflows

2
x-pack/test/detection_engine_api_integration/security_and_spaces/group1/index.ts
View file

@ -19,11 +19,9 @@ export default ({ loadTestFile }: FtrProviderContext): void => {
loadTestFile(require.resolve('./update_actions'));
loadTestFile(require.resolve('./check_privileges'));
loadTestFile(require.resolve('./create_index'));
loadTestFile(require.resolve('./create_rules'));
loadTestFile(require.resolve('./preview_rules'));
loadTestFile(require.resolve('./create_rules_bulk'));
loadTestFile(require.resolve('./create_new_terms'));
loadTestFile(require.resolve('./create_rule_exceptions'));
loadTestFile(require.resolve('./delete_rules'));
loadTestFile(require.resolve('./delete_rules_bulk'));
loadTestFile(require.resolve('./export_rules'));

46
x-pack/test/security_solution_api_integration/README.md Normal file
View file

@ -0,0 +1,46 @@
# security_solution_api_integration
This directory serves as a centralized location to place the security solution tests that run in Serverless and ESS environments.
## Subdirectories
1. `config` stores base configurations specific to both the Serverless and ESS environments, These configurations build upon the base configuration provided by `xpack/test_serverless` and `x-pack-api_integrations`, incorporating additional settings such as environment variables and tagging options.
2. `test_suites` directory now houses all the tests along with their utility functions. As an initial step,
we have introduced the `detection_response` directory to consolidate all the integration tests related to detection and response APIs.
## Overview
- In this directory, Mocha tagging is utilized to assign tags to specific test suites and individual test cases. This tagging system enables the ability to selectively apply tags to test suites and test cases, facilitating the exclusion of specific test cases within a test suite as needed.
- There are three primary tags that have been defined: @ess, @serverless, and @brokenInServerless
- Test suites and cases are prefixed with specific tags to determine their execution in particular environments or to exclude them from specific environments.
ex:
```
describe('@serverless @ess create_rules', () => { ==> tests in this suite will run in both Ess and Serverless
describe('creating rules', () => {});
describe('@brokenInServerless missing timestamps', () => {}); ==> tests in this suite will be excluded in Serverless
```
## Adding new security area's tests
1. Within the `test_suites` directory, create a new area folder.
2. Introduce `ess.config` and `serverless.config` files to reference the new test files and incorporate any additional custom properties defined in the `CreateTestConfigOptions` interface.
3. In these new configuration files, include references to the base configurations located under the config directory to inherit CI configurations, environment variables, and other settings.
4. Append a new entry in the `ftr_configs.yml` file to enable the execution of the newly added tests within the CI pipeline.
## Testing locally
In the `package.json` file, you'll find commands to configure the server for each environment and to run tests against that specific environment. These commands adhere to the Mocha tagging system, allowing for the inclusion and exclusion of tags, mirroring the setup of the CI pipeline.

13
x-pack/test/security_solution_api_integration/config/ess/config.base.trial.ts Normal file
View file

@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { createTestConfig } from './config.base';
export default createTestConfig({
license: 'trial',
ssl: true,
});

113
x-pack/test/security_solution_api_integration/config/ess/config.base.ts Normal file
View file

@ -0,0 +1,113 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { CA_CERT_PATH } from '@kbn/dev-utils';
import { FtrConfigProviderContext, kbnTestConfig, kibanaTestUser } from '@kbn/test';
import { services } from '../../../api_integration/services';
interface CreateTestConfigOptions {
license: string;
ssl?: boolean;
}
// test.not-enabled is specifically not enabled
const enabledActionTypes = [
'.email',
'.index',
'.pagerduty',
'.swimlane',
'.server-log',
'.servicenow',
'.slack',
'.webhook',
'test.authorization',
'test.failing',
'test.index-record',
'test.noop',
'test.rate-limit',
];
export function createTestConfig(options: CreateTestConfigOptions, testFiles?: string[]) {
const { license = 'trial', ssl = false } = options;
return async ({ readConfigFile }: FtrConfigProviderContext) => {
const xPackApiIntegrationTestsConfig = await readConfigFile(
require.resolve('../../../api_integration/config.ts')
);
const servers = {
...xPackApiIntegrationTestsConfig.get('servers'),
elasticsearch: {
...xPackApiIntegrationTestsConfig.get('servers.elasticsearch'),
protocol: ssl ? 'https' : 'http',
},
};
return {
testFiles,
servers,
services,
junit: {
reportName: 'X-Pack Detection Engine API Integration Tests',
},
esTestCluster: {
...xPackApiIntegrationTestsConfig.get('esTestCluster'),
license,
ssl,
serverArgs: [`xpack.license.self_generated.type=${license}`],
},
kbnTestServer: {
...xPackApiIntegrationTestsConfig.get('kbnTestServer'),
env: {
ELASTICSEARCH_USERNAME: kbnTestConfig.getUrlParts(kibanaTestUser).username,
},
serverArgs: [
...xPackApiIntegrationTestsConfig.get('kbnTestServer.serverArgs'),
`--xpack.actions.allowedHosts=${JSON.stringify(['localhost', 'some.non.existent.com'])}`,
`--xpack.actions.enabledActionTypes=${JSON.stringify(enabledActionTypes)}`,
'--xpack.eventLog.logEntries=true',
`--xpack.securitySolution.alertIgnoreFields=${JSON.stringify([
'testing_ignored.constant',
'/testing_regex*/',
])}`, // See tests within the file "ignore_fields.ts" which use these values in "alertIgnoreFields"
'--xpack.ruleRegistry.write.enabled=true',
'--xpack.ruleRegistry.write.cache.enabled=false',
'--xpack.ruleRegistry.unsafe.indexUpgrade.enabled=true',
'--xpack.ruleRegistry.unsafe.legacyMultiTenancy.enabled=true',
`--xpack.securitySolution.enableExperimental=${JSON.stringify([
'previewTelemetryUrlEnabled',
'riskScoringPersistence',
'riskScoringRoutesEnabled',
])}`,
'--xpack.task_manager.poll_interval=1000',
`--xpack.actions.preconfigured=${JSON.stringify({
'my-test-email': {
actionTypeId: '.email',
name: 'TestEmail#xyz',
config: {
from: 'me@test.com',
service: '__json',
},
secrets: {
user: 'user',
password: 'password',
},
},
})}`,
...(ssl
? [
`--elasticsearch.hosts=${servers.elasticsearch.protocol}://${servers.elasticsearch.hostname}:${servers.elasticsearch.port}`,
`--elasticsearch.ssl.certificateAuthorities=${CA_CERT_PATH}`,
]
: []),
],
},
mochaOpts: {
grep: '/^(?!.*@brokenInEss).*@ess.*/',
},
};
};
}

35
x-pack/test/security_solution_api_integration/config/serverless/config.base.ts Normal file
View file

@ -0,0 +1,35 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { FtrConfigProviderContext } from '@kbn/test';
export interface CreateTestConfigOptions {
testFiles: string[];
junit: { reportName: string };
}
export function createTestConfig(options: CreateTestConfigOptions) {
return async ({ readConfigFile }: FtrConfigProviderContext) => {
const svlSharedConfig = await readConfigFile(
require.resolve('../../../../test_serverless/shared/config.base.ts')
);
return {
...svlSharedConfig.getAll(),
kbnTestServer: {
...svlSharedConfig.get('kbnTestServer'),
serverArgs: [...svlSharedConfig.get('kbnTestServer.serverArgs'), '--serverless=security'],
},
testFiles: options.testFiles,
junit: options.junit,
mochaOpts: {
...svlSharedConfig.get('mochaOpts'),
grep: '/^(?!.*@brokenInServerless).*@serverless.*/',
},
};
};
}

BIN
x-pack/test/security_solution_api_integration/es_archive/serverless/auditbeat/hosts/data.json.gz Normal file
View file

Binary file not shown.

2603
x-pack/test/security_solution_api_integration/es_archive/serverless/auditbeat/hosts/mappings.json Normal file
View file

File diff suppressed because it is too large Load diff

11
x-pack/test/security_solution_api_integration/es_archive_path_builder/constants.ts Normal file
View file

@ -0,0 +1,11 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export const SERVERLESS_ES_ARCHIVE_PATH =
'x-pack/test/security_solution_api_integration/es_archive/serverless';
export const ESS_ES_ARCHIVE_PATH = 'x-pack/test/functional/es_archives';

27
x-pack/test/security_solution_api_integration/es_archive_path_builder/index.ts Normal file
View file

@ -0,0 +1,27 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { SERVERLESS_ES_ARCHIVE_PATH, ESS_ES_ARCHIVE_PATH } from './constants';
export class EsArchivePathBuilder {
constructor(private isServerless: boolean) {
this.isServerless = isServerless;
}
/**
* @param resourceUri represents the data type, e.g., auditbeat, and its associated index for hosts.
* @returns the complete path based on the environment from which we intend to load the data.
*/
getPath(resourceUri: string): string {
const archivePath = this.getEsArchivePathBasedOnEnv();
return `${archivePath}/${resourceUri}`;
}
private getEsArchivePathBasedOnEnv(): string {
return this.isServerless ? SERVERLESS_ES_ARCHIVE_PATH : ESS_ES_ARCHIVE_PATH;
}
}

10
x-pack/test/security_solution_api_integration/ftr_provider_context.d.ts vendored Normal file
View file

@ -0,0 +1,10 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { FtrProviderContext } from '../../test_serverless/api_integration/ftr_provider_context';
export type { FtrProviderContext };

13
x-pack/test/security_solution_api_integration/package.json Normal file
View file

@ -0,0 +1,13 @@
{
"author": "Elastic",
"name": "@kbn/security_solution_api_integration",
"version": "1.0.0",
"private": true,
"license": "Elastic License 2.0",
"scripts": {
"detectionResponse:server:serverless": "node ../../../scripts/functional_tests_server.js --config ./test_suites/detections_response/serverless.config.ts",
"detectionResponse:runner:serverless": "node ../../../scripts/functional_test_runner --config=test_suites/detections_response/serverless.config.ts --grep @serverless --grep @brokenInServerless --invert",
"detectionResponse:server:ess": "node ../../../scripts/functional_tests_server.js --config ./test_suites/detections_response/ess.config.ts",
"detectionResponse:runner:ess": "node ../../../scripts/functional_test_runner --config=test_suites/detections_response/ess.config.ts --grep @ess"
}
}

22
x-pack/test/security_solution_api_integration/test_suites/detections_response/ess.config.ts Normal file
View file

@ -0,0 +1,22 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { FtrConfigProviderContext } from '@kbn/test';
export default async function ({ readConfigFile }: FtrConfigProviderContext) {
const functionalConfig = await readConfigFile(
require.resolve('../../config/ess/config.base.trial.ts')
);
return {
...functionalConfig.getAll(),
testFiles: [require.resolve('.')],
junit: {
reportName: 'Detection Engine ESS API Integration Tests',
},
};
}

13
x-pack/test/security_solution_api_integration/test_suites/detections_response/exceptions/index.ts Normal file
View file

@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { FtrProviderContext } from '../../../ftr_provider_context';
export default function ({ loadTestFile }: FtrProviderContext) {
describe('Exceptions API', function () {
loadTestFile(require.resolve('./rule_exception/create_rule_exceptions'));
});
}

26
x-pack/test/detection_engine_api_integration/security_and_spaces/group1/create_rule_exceptions.ts → x-pack/test/security_solution_api_integration/test_suites/detections_response/exceptions/rule_exception/create_rule_exceptions.ts
View file

@ -15,12 +15,13 @@ import {
ExceptionListTypeEnum,
} from '@kbn/securitysolution-io-ts-list-types';
import { getCreateExceptionListMinimalSchemaMock } from '@kbn/lists-plugin/common/schemas/request/create_exception_list_schema.mock';
import { FtrProviderContext } from '../../common/ftr_provider_context';
import { FtrProviderContext } from '../../../../ftr_provider_context';
import {
getRule,
createRule,
getSimpleRule,
createSignalsIndex,
createAlertsIndex,
deleteAllRules,
createExceptionList,
deleteAllAlerts,
@ -28,7 +29,7 @@ import {
import {
deleteAllExceptions,
removeExceptionListItemServerGeneratedProperties,
} from '../../../lists_api_integration/utils';
} from '../../../../../lists_api_integration/utils';
const getRuleExceptionItemMock = (): CreateRuleExceptionListItemSchema => ({
description: 'Exception item for rule default exception list',
@ -44,15 +45,16 @@ const getRuleExceptionItemMock = (): CreateRuleExceptionListItemSchema => ({
type: 'simple',
});
// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext) => {
const supertest = getService('supertest');
const log = getService('log');
const es = getService('es');
const config = getService('config');
const ELASTICSEARCH_USERNAME = config.get('servers.kibana.username');
describe('create_rule_exception_route', () => {
describe('@serverless @ess create_rule_exception_route', () => {
before(async () => {
await createSignalsIndex(supertest, log);
await createAlertsIndex(supertest, log);
});
after(async () => {
@ -84,7 +86,7 @@ export default ({ getService }: FtrProviderContext) => {
expect(itemsWithoutServerGeneratedValues).to.eql([
{
comments: [],
created_by: 'elastic',
created_by: ELASTICSEARCH_USERNAME,
description: 'Exception item for rule default exception list',
entries: [
{
@ -100,7 +102,7 @@ export default ({ getService }: FtrProviderContext) => {
os_types: [],
tags: [],
type: 'simple',
updated_by: 'elastic',
updated_by: ELASTICSEARCH_USERNAME,
},
]);
expect(udpatedRule.exceptions_list.some((list) => list.type === 'rule_default')).to.eql(true);
@ -148,7 +150,7 @@ export default ({ getService }: FtrProviderContext) => {
expect(itemsWithoutServerGeneratedValues).to.eql([
{
comments: [],
created_by: 'elastic',
created_by: ELASTICSEARCH_USERNAME,
description: 'Exception item for rule default exception list',
entries: [
{
@ -164,7 +166,7 @@ export default ({ getService }: FtrProviderContext) => {
os_types: [],
tags: [],
type: 'simple',
updated_by: 'elastic',
updated_by: ELASTICSEARCH_USERNAME,
},
]);
});
@ -210,7 +212,7 @@ export default ({ getService }: FtrProviderContext) => {
);
expect(itemsWithoutServerGeneratedValues[0]).to.eql({
comments: [],
created_by: 'elastic',
created_by: ELASTICSEARCH_USERNAME,
description: 'Exception item for rule default exception list',
entries: [
{
@ -226,7 +228,7 @@ export default ({ getService }: FtrProviderContext) => {
os_types: [],
tags: [],
type: 'simple',
updated_by: 'elastic',
updated_by: ELASTICSEARCH_USERNAME,
});
});

15
x-pack/test/security_solution_api_integration/test_suites/detections_response/index.ts Normal file
View file

@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { FtrProviderContext } from '../../ftr_provider_context';
export default function ({ loadTestFile }: FtrProviderContext) {
describe('Detections response API', function () {
loadTestFile(require.resolve('./exceptions'));
loadTestFile(require.resolve('./rule_creation'));
});
}

101
x-pack/test/detection_engine_api_integration/security_and_spaces/group1/create_rules.ts → x-pack/test/security_solution_api_integration/test_suites/detections_response/rule_creation/create_rules.ts
View file

@ -17,9 +17,9 @@ import { RuleCreateProps } from '@kbn/security-solution-plugin/common/api/detect
import { ExceptionListTypeEnum } from '@kbn/securitysolution-io-ts-list-types';
import { ROLES } from '@kbn/security-solution-plugin/common/test';
import { FtrProviderContext } from '../../common/ftr_provider_context';
import { FtrProviderContext } from '../../../ftr_provider_context';
import {
createSignalsIndex,
createAlertsIndex,
deleteAllRules,
getSimpleRule,
getSimpleRuleOutput,
@ -30,43 +30,50 @@ import {
getSimpleMlRule,
getSimpleMlRuleOutput,
waitForRuleSuccess,
getRuleForSignalTesting,
getRuleForSignalTestingWithTimestampOverride,
getRuleForAlertTesting,
getRuleForAlertTestingWithTimestampOverride,
waitForAlertToComplete,
waitForSignalsToBePresent,
getThresholdRuleForSignalTesting,
waitForAlertsToBePresent,
getThresholdRuleForAlertTesting,
waitForRulePartialFailure,
createRule,
deleteAllAlerts,
} from '../../utils';
import { createUserAndRole, deleteUserAndRole } from '../../../common/services/security_solution';
import {
removeUUIDFromActions,
getActionsWithFrequencies,
getActionsWithoutFrequencies,
getSomeActionsWithFrequencies,
} from '../../utils/get_rule_actions';
import { removeUUIDFromActions } from '../../utils/remove_uuid_from_actions';
updateUsername,
} from '../utils';
import {
createUserAndRole,
deleteUserAndRole,
} from '../../../../common/services/security_solution';
import { EsArchivePathBuilder } from '../../../es_archive_path_builder';
// eslint-disable-next-line import/no-default-export
export default ({ getService }: FtrProviderContext) => {
const esArchiver = getService('esArchiver');
const supertest = getService('supertest');
const supertestWithoutAuth = getService('supertestWithoutAuth');
const log = getService('log');
const es = getService('es');
const config = getService('config');
const ELASTICSEARCH_USERNAME = config.get('servers.kibana.username');
const isServerless = config.get('serverless');
const dataPathBuilder = new EsArchivePathBuilder(isServerless);
const path = dataPathBuilder.getPath('auditbeat/hosts');
describe('create_rules', () => {
describe('@serverless @ess create_rules', () => {
describe('creating rules', () => {
before(async () => {
await esArchiver.load('x-pack/test/functional/es_archives/auditbeat/hosts');
await esArchiver.load(path);
});
after(async () => {
await esArchiver.unload('x-pack/test/functional/es_archives/auditbeat/hosts');
await esArchiver.unload(path);
});
beforeEach(async () => {
await createSignalsIndex(supertest, log);
await createAlertsIndex(supertest, log);
});
afterEach(async () => {
@ -103,7 +110,8 @@ export default ({ getService }: FtrProviderContext) => {
.expect(200);
const bodyToCompare = removeServerGeneratedProperties(body);
expect(bodyToCompare).to.eql(getSimpleRuleOutput());
const expectedRule = updateUsername(getSimpleRuleOutput(), ELASTICSEARCH_USERNAME);
expect(bodyToCompare).to.eql(expectedRule);
});
/*
@ -127,7 +135,7 @@ export default ({ getService }: FtrProviderContext) => {
*/
it('should create a single rule with a rule_id and validate it ran successfully', async () => {
const rule = {
...getRuleForSignalTesting(['auditbeat-*']),
...getRuleForAlertTesting(['auditbeat-*']),
query: 'process.executable: "/usr/bin/sudo"',
};
const { body } = await supertest
@ -141,7 +149,7 @@ export default ({ getService }: FtrProviderContext) => {
});
it('should create a single rule with a rule_id and an index pattern that does not match anything available and partial failure for the rule', async () => {
const simpleRule = getRuleForSignalTesting(['does-not-exist-*']);
const simpleRule = getRuleForAlertTesting(['does-not-exist-*']);
const { body } = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
@ -171,7 +179,7 @@ export default ({ getService }: FtrProviderContext) => {
it('should create a single rule with a rule_id and an index pattern that does not match anything and an index pattern that does and the rule should be successful', async () => {
const rule = {
...getRuleForSignalTesting(['does-not-exist-*', 'auditbeat-*']),
...getRuleForAlertTesting(['does-not-exist-*', 'auditbeat-*']),
query: 'process.executable: "/usr/bin/sudo"',
};
const { body } = await supertest
@ -198,7 +206,7 @@ export default ({ getService }: FtrProviderContext) => {
const expected = {
actions: [],
author: [],
created_by: 'elastic',
created_by: ELASTICSEARCH_USERNAME,
description: 'Simple Rule Query',
enabled: true,
false_positives: [],
@ -220,7 +228,7 @@ export default ({ getService }: FtrProviderContext) => {
setup: '',
severity: 'high',
severity_mapping: [],
updated_by: 'elastic',
updated_by: ELASTICSEARCH_USERNAME,
tags: [],
to: 'now',
type: 'query',
@ -249,7 +257,11 @@ export default ({ getService }: FtrProviderContext) => {
.expect(200);
const bodyToCompare = removeServerGeneratedPropertiesIncludingRuleId(body);
expect(bodyToCompare).to.eql(getSimpleRuleOutputWithoutRuleId());
const expectedRule = updateUsername(
getSimpleRuleOutputWithoutRuleId(),
ELASTICSEARCH_USERNAME
);
expect(bodyToCompare).to.eql(expectedRule);
});
it('creates a single Machine Learning rule from a legacy ML Rule format', async () => {
@ -265,7 +277,8 @@ export default ({ getService }: FtrProviderContext) => {
.expect(200);
const bodyToCompare = removeServerGeneratedProperties(body);
expect(bodyToCompare).to.eql(getSimpleMlRuleOutput());
const expectedRule = updateUsername(getSimpleMlRuleOutput(), ELASTICSEARCH_USERNAME);
expect(bodyToCompare).to.eql(expectedRule);
});
it('should create a single Machine Learning rule', async () => {
@ -277,7 +290,8 @@ export default ({ getService }: FtrProviderContext) => {
.expect(200);
const bodyToCompare = removeServerGeneratedProperties(body);
expect(bodyToCompare).to.eql(getSimpleMlRuleOutput());
const expectedRule = updateUsername(getSimpleMlRuleOutput(), ELASTICSEARCH_USERNAME);
expect(bodyToCompare).to.eql(expectedRule);
});
it('should cause a 409 conflict if we attempt to create the same rule_id twice', async () => {
@ -418,7 +432,7 @@ export default ({ getService }: FtrProviderContext) => {
});
});
describe('t1_analyst', () => {
describe('@brokenInServerless t1_analyst', () => {
const role = ROLES.t1_analyst;
beforeEach(async () => {
@ -442,7 +456,7 @@ export default ({ getService }: FtrProviderContext) => {
describe('threshold validation', () => {
it('should result in 400 error if no threshold-specific fields are provided', async () => {
const { threshold, ...rule } = getThresholdRuleForSignalTesting(['*']);
const { threshold, ...rule } = getThresholdRuleForAlertTesting(['*']);
const { body } = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
@ -458,7 +472,7 @@ export default ({ getService }: FtrProviderContext) => {
});
it('should result in 400 error if more than 3 threshold fields', async () => {
const rule = getThresholdRuleForSignalTesting(['*']);
const rule = getThresholdRuleForAlertTesting(['*']);
const { body } = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
@ -479,7 +493,7 @@ export default ({ getService }: FtrProviderContext) => {
});
it('should result in 400 error if threshold value is less than 1', async () => {
const rule = getThresholdRuleForSignalTesting(['*']);
const rule = getThresholdRuleForAlertTesting(['*']);
const { body } = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
@ -501,7 +515,7 @@ export default ({ getService }: FtrProviderContext) => {
});
it('should result in 400 error if cardinality is also an agg field', async () => {
const rule = getThresholdRuleForSignalTesting(['*']);
const rule = getThresholdRuleForAlertTesting(['*']);
const { body } = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
@ -528,9 +542,9 @@ export default ({ getService }: FtrProviderContext) => {
});
});
describe('missing timestamps', () => {
describe('@brokenInServerless missing timestamps', () => {
beforeEach(async () => {
await createSignalsIndex(supertest, log);
await createAlertsIndex(supertest, log);
// to edit these files run the following script
// cd $HOME/kibana/x-pack && nvm use && node ../scripts/es_archiver edit security_solution/timestamp_override
await esArchiver.load(
@ -549,7 +563,7 @@ export default ({ getService }: FtrProviderContext) => {
// defaults to event.ingested timestamp override.
// event.ingested is one of the timestamp fields set on the es archive data
// inside of x-pack/test/functional/es_archives/security_solution/timestamp_override/data.json.gz
const simpleRule = getRuleForSignalTestingWithTimestampOverride(['myfakeindex-1']);
const simpleRule = getRuleForAlertTestingWithTimestampOverride(['myfakeindex-1']);
const { body } = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
@ -583,7 +597,7 @@ export default ({ getService }: FtrProviderContext) => {
// defaults to event.ingested timestamp override.
// event.ingested is one of the timestamp fields set on the es archive data
// inside of x-pack/test/functional/es_archives/security_solution/timestamp_override/data.json.gz
const simpleRule = getRuleForSignalTestingWithTimestampOverride(['myfa*']);
const simpleRule = getRuleForAlertTestingWithTimestampOverride(['myfa*']);
const { body } = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
@ -597,7 +611,7 @@ export default ({ getService }: FtrProviderContext) => {
log,
id: bodyId,
});
await waitForSignalsToBePresent(supertest, log, 2, [bodyId]);
await waitForAlertsToBePresent(supertest, log, 2, [bodyId]);
const { body: rule } = await supertest
.get(DETECTION_ENGINE_RULES_URL)
@ -611,7 +625,7 @@ export default ({ getService }: FtrProviderContext) => {
});
});
describe('per-action frequencies', () => {
describe('@brokenInServerless per-action frequencies', () => {
const createSingleRule = async (rule: RuleCreateProps) => {
const createdRule = await createRule(supertest, log, rule);
createdRule.actions = removeUUIDFromActions(createdRule.actions);
@ -629,8 +643,7 @@ export default ({ getService }: FtrProviderContext) => {
simpleRule.actions = actionsWithoutFrequencies;
const createdRule = await createSingleRule(simpleRule);
const expectedRule = getSimpleRuleOutput();
const expectedRule = updateUsername(getSimpleRuleOutput(), ELASTICSEARCH_USERNAME);
expectedRule.actions = actionsWithoutFrequencies.map((action) => ({
...action,
frequency: NOTIFICATION_DEFAULT_FREQUENCY,
@ -652,8 +665,7 @@ export default ({ getService }: FtrProviderContext) => {
simpleRule.actions = actionsWithoutFrequencies;
const createdRule = await createSingleRule(simpleRule);
const expectedRule = getSimpleRuleOutput();
const expectedRule = updateUsername(getSimpleRuleOutput(), ELASTICSEARCH_USERNAME);
expectedRule.actions = actionsWithoutFrequencies.map((action) => ({
...action,
frequency: { summary: true, throttle, notifyWhen: 'onThrottleInterval' },
@ -683,8 +695,7 @@ export default ({ getService }: FtrProviderContext) => {
simpleRule.actions = actionsWithFrequencies;
const createdRule = await createSingleRule(simpleRule);
const expectedRule = getSimpleRuleOutput();
const expectedRule = updateUsername(getSimpleRuleOutput(), ELASTICSEARCH_USERNAME);
expectedRule.actions = actionsWithFrequencies;
const rule = removeServerGeneratedProperties(createdRule);
@ -704,8 +715,7 @@ export default ({ getService }: FtrProviderContext) => {
simpleRule.actions = someActionsWithFrequencies;
const createdRule = await createSingleRule(simpleRule);
const expectedRule = getSimpleRuleOutput();
const expectedRule = updateUsername(getSimpleRuleOutput(), ELASTICSEARCH_USERNAME);
expectedRule.actions = someActionsWithFrequencies.map((action) => ({
...action,
frequency: action.frequency ?? NOTIFICATION_DEFAULT_FREQUENCY,
@ -727,8 +737,7 @@ export default ({ getService }: FtrProviderContext) => {
simpleRule.actions = someActionsWithFrequencies;
const createdRule = await createSingleRule(simpleRule);
const expectedRule = getSimpleRuleOutput();
const expectedRule = updateUsername(getSimpleRuleOutput(), ELASTICSEARCH_USERNAME);
expectedRule.actions = someActionsWithFrequencies.map((action) => ({
...action,
frequency: action.frequency ?? {

13
x-pack/test/security_solution_api_integration/test_suites/detections_response/rule_creation/index.ts Normal file
View file

@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { FtrProviderContext } from '../../../ftr_provider_context';
export default function ({ loadTestFile }: FtrProviderContext) {
describe('Rule creation API', function () {
loadTestFile(require.resolve('./create_rules'));
});
}

15
x-pack/test/security_solution_api_integration/test_suites/detections_response/serverless.config.ts Normal file
View file

@ -0,0 +1,15 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { createTestConfig } from '../../config/serverless/config.base';
export default createTestConfig({
testFiles: [require.resolve('.')],
junit: {
reportName: 'Detection Engine Serverless API Integration Tests',
},
});

14
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/action/get_slack_action.ts Normal file
View file

@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export const getSlackAction = () => ({
actionTypeId: '.slack',
secrets: {
webhookUrl: 'http://localhost:123',
},
name: 'Slack connector',
});

19
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/action/get_web_hook_action.ts Normal file
View file

@ -0,0 +1,19 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export const getWebHookAction = () => ({
actionTypeId: '.webhook',
config: {
method: 'post',
url: 'http://localhost',
},
secrets: {
user: 'example',
password: 'example',
},
name: 'Some connector',
});

14
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/action/remove_uuid_from_actions.ts Normal file
View file

@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { RuleActionArray } from '@kbn/securitysolution-io-ts-alerting-types';
export const removeUUIDFromActions = (actions: RuleActionArray): RuleActionArray => {
return actions.map(({ uuid, ...restOfAction }) => ({
...restOfAction,
}));
};

37
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/alert/create_alerts_index.ts Normal file
View file

@ -0,0 +1,37 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type SuperTest from 'supertest';
import { ToolingLog } from '@kbn/tooling-log';
import { DETECTION_ENGINE_INDEX_URL } from '@kbn/security-solution-plugin/common/constants';
import { countDownTest } from '../count_down_test';
/**
* Creates the alerts index for use inside of beforeEach blocks of tests
* This will retry 50 times before giving up and hopefully still not interfere with other tests
* @param supertest The supertest client library
*/
export const createAlertsIndex = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog
): Promise<void> => {
await countDownTest(
async () => {
await supertest
.post(DETECTION_ENGINE_INDEX_URL)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.send();
return {
passed: true,
};
},
'createAlertsIndex',
log
);
};

49
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/alert/delete_all_alerts.ts Normal file
View file

@ -0,0 +1,49 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type SuperTest from 'supertest';
import type { ToolingLog } from '@kbn/tooling-log';
import type { Client } from '@elastic/elasticsearch';
import { DETECTION_ENGINE_INDEX_URL } from '@kbn/security-solution-plugin/common/constants';
import { countDownTest } from '../count_down_test';
/**
* Deletes all alerts from a given index or indices, defaults to `.alerts-security.alerts-*`
* For use inside of afterEach blocks of tests
*/
export const deleteAllAlerts = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
es: Client,
index: Array<'.alerts-security.alerts-*' | '.preview.alerts-security.alerts-*'> = [
'.alerts-security.alerts-*',
]
): Promise<void> => {
await countDownTest(
async () => {
await supertest
.delete(DETECTION_ENGINE_INDEX_URL)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.send();
await es.deleteByQuery({
index,
body: {
query: {
match_all: {},
},
},
refresh: true,
});
return {
passed: true,
};
},
'deleteAllAlerts',
log
);
};

56
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/alert/get_alerts_by_ids.ts Normal file
View file

@ -0,0 +1,56 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { SearchResponse } from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import type { DetectionAlert } from '@kbn/security-solution-plugin/common/api/detection_engine';
import type { RiskEnrichmentFields } from '@kbn/security-solution-plugin/server/lib/detection_engine/rule_types/utils/enrichments/types';
import { DETECTION_ENGINE_QUERY_SIGNALS_URL as DETECTION_ENGINE_QUERY_ALERTS_URL } from '@kbn/security-solution-plugin/common/constants';
import { countDownTest } from '../count_down_test';
import { getQueryAlertsId } from './get_query_alerts_ids';
/**
* Given an array of rule ids this will return only alerts based on that rule id both
* open and closed
* @param supertest agent
* @param ids Array of the rule ids
*/
export const getAlertsByIds = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
ids: string[],
size?: number
): Promise<SearchResponse<DetectionAlert & RiskEnrichmentFields>> => {
const alertsOpen = await countDownTest<SearchResponse<DetectionAlert & RiskEnrichmentFields>>(
async () => {
const response = await supertest
.post(DETECTION_ENGINE_QUERY_ALERTS_URL)
.set('kbn-xsrf', 'true')
.send(getQueryAlertsId(ids, size));
if (response.status !== 200) {
return {
passed: false,
errorMessage: `Status is not 200 as expected, it is: ${response.status}`,
};
} else {
return {
passed: true,
returnValue: response.body,
};
}
},
'getAlertsByIds',
log
);
if (alertsOpen == null) {
throw new Error('Alerts not defined after countdown, cannot continue');
} else {
return alertsOpen;
}
};

23
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/alert/get_query_alerts_ids.ts Normal file
View file

@ -0,0 +1,23 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { ALERT_RULE_UUID } from '@kbn/rule-data-utils';
/**
* Given an array of ids for a test this will get the alerts
* created from that rule's regular id.
* @param ids The rule_id to search for alerts
*/
export const getQueryAlertsId = (ids: string[], size = 10) => ({
size,
sort: ['@timestamp'],
query: {
terms: {
[ALERT_RULE_UUID]: ids,
},
},
});

33
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/alert/wait_for_alert_to_complete.ts Normal file
View file

@ -0,0 +1,33 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import { waitFor } from '../wait_for';
export const waitForAlertToComplete = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
id: string
): Promise<void> => {
await waitFor(
async () => {
const response = await supertest.get(`/api/alerts/alert/${id}/state`).set('kbn-xsrf', 'true');
if (response.status !== 200) {
log.debug(
`Did not get an expected 200 "ok" when waiting for an alert to complete (waitForAlertToComplete). CI issues could happen. Suspect this line if you are seeing CI issues. body: ${JSON.stringify(
response.body
)}, status: ${JSON.stringify(response.status)}`
);
}
return response.body.previousStartedAt != null;
},
'waitForAlertToComplete',
log
);
};

34
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/alert/wait_for_alerts_to_be_present.ts Normal file
View file

@ -0,0 +1,34 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import { getAlertsByIds } from './get_alerts_by_ids';
import { waitFor } from '../wait_for';
/**
* Waits for the signal hits to be greater than the supplied number
* before continuing with a default of at least one signal
* @param supertest Deps
* @param numberOfAlerts The number of alerts to wait for, default is 1
*/
export const waitForAlertsToBePresent = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
numberOfAlerts = 1,
alertIds: string[]
): Promise<void> => {
await waitFor(
async () => {
const alertsOpen = await getAlertsByIds(supertest, log, alertIds, numberOfAlerts);
return alertsOpen.hits.hits.length >= numberOfAlerts;
},
'waitForAlertsToBePresent',
log
);
};

78
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/count_down_test.ts Normal file
View file

@ -0,0 +1,78 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
/**
* Does a plain countdown and checks against a boolean to determine if to wait and try again.
* This is useful for over the wire things that can cause issues such as conflict or timeouts
* for testing resiliency.
* @param functionToTest The function to test against
* @param name The name of the function to print if we encounter errors
* @param log The tooling logger
* @param retryCount The number of times to retry before giving up (has default)
* @param timeoutWait Time to wait before trying again (has default)
*/
export const countDownTest = async <T>(
functionToTest: () => Promise<{
passed: boolean;
returnValue?: T | undefined;
errorMessage?: string;
}>,
name: string,
log: ToolingLog,
retryCount: number = 50,
timeoutWait = 250,
ignoreThrow: boolean = false
): Promise<T | undefined> => {
if (retryCount > 0) {
try {
const testReturn = await functionToTest();
if (!testReturn.passed) {
const error = testReturn.errorMessage != null ? ` error: ${testReturn.errorMessage},` : '';
log.error(`Failure trying to ${name},${error} retries left are: ${retryCount - 1}`);
// retry, counting down, and delay a bit before
await new Promise((resolve) => setTimeout(resolve, timeoutWait));
const returnValue = await countDownTest(
functionToTest,
name,
log,
retryCount - 1,
timeoutWait,
ignoreThrow
);
return returnValue;
} else {
return testReturn.returnValue;
}
} catch (err) {
if (ignoreThrow) {
throw err;
} else {
log.error(
`Failure trying to ${name}, with exception message of: ${
err.message
}, retries left are: ${retryCount - 1}`
);
// retry, counting down, and delay a bit before
await new Promise((resolve) => setTimeout(resolve, timeoutWait));
const returnValue = await countDownTest(
functionToTest,
name,
log,
retryCount - 1,
timeoutWait,
ignoreThrow
);
return returnValue;
}
}
} else {
log.error(`Could not ${name}, no retries are left`);
return undefined;
}
};

68
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/exception_list_and_item/exception_list/create_exception_list.ts Normal file
View file

@ -0,0 +1,68 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import type {
CreateExceptionListSchema,
ExceptionListSchema,
} from '@kbn/securitysolution-io-ts-list-types';
import { EXCEPTION_LIST_URL } from '@kbn/securitysolution-list-constants';
import { deleteExceptionList } from './delete_exception_list';
/**
* Helper to cut down on the noise in some of the tests. This checks for
* an expected 200 still and does not try to any retries. Creates exception lists
* @param supertest The supertest deps
* @param exceptionList The exception list to create
* @param log The tooling logger
*/
export const createExceptionList = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
exceptionList: CreateExceptionListSchema
): Promise<ExceptionListSchema> => {
const response = await supertest
.post(EXCEPTION_LIST_URL)
.set('kbn-xsrf', 'true')
.send(exceptionList);
if (response.status === 409) {
if (exceptionList.list_id != null) {
log.error(
`When creating an exception list found an unexpected conflict (409) creating an exception list (createExceptionList), will attempt a cleanup and one time re-try. This usually indicates a bad cleanup or race condition within the tests: ${JSON.stringify(
response.body
)}, status: ${JSON.stringify(response.status)}`
);
await deleteExceptionList(supertest, log, exceptionList.list_id);
const secondResponseTry = await supertest
.post(EXCEPTION_LIST_URL)
.set('kbn-xsrf', 'true')
.send(exceptionList);
if (secondResponseTry.status !== 200) {
throw new Error(
`Unexpected non 200 ok when attempting to create an exception list (second try): ${JSON.stringify(
response.body
)}`
);
} else {
return secondResponseTry.body;
}
} else {
throw new Error('When creating an exception list found an unexpected conflict (404)');
}
} else if (response.status !== 200) {
throw new Error(
`Unexpected non 200 ok when attempting to create an exception list: ${JSON.stringify(
response.status
)}`
);
} else {
return response.body;
}
};

37
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/exception_list_and_item/exception_list/delete_exception_list.ts Normal file
View file

@ -0,0 +1,37 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import { EXCEPTION_LIST_URL } from '@kbn/securitysolution-list-constants';
import type { RuleResponse } from '@kbn/security-solution-plugin/common/api/detection_engine';
/**
* Helper to cut down on the noise in some of the tests. Does a delete of an exception list.
* It does not check for a 200 "ok" on this.
* @param supertest The supertest deps
* @param listId The exception list to delete
* @param log The tooling logger
*/
export const deleteExceptionList = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
listId: string
): Promise<RuleResponse> => {
const response = await supertest
.delete(`${EXCEPTION_LIST_URL}?list_id=${listId}`)
.set('kbn-xsrf', 'true');
if (response.status !== 200) {
log.error(
`Did not get an expected 200 "ok" when deleting an exception list (deleteExceptionList). CI issues could happen. Suspect this line if you are seeing CI issues. body: ${JSON.stringify(
response.body
)}, status: ${JSON.stringify(response.status)}`
);
}
return response.body;
};

41
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/index.ts Normal file
View file

@ -0,0 +1,41 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export * from './rule/get_rule';
export * from './rule/get_simple_rule';
export * from './rule/create_rule';
export * from './rule/delete_all_rules';
export * from './rule/delete_rule';
export * from './rule/get_simple_rule_output';
export * from './rule/get_simple_rule_output_without_rule_id';
export * from './rule/get_simple_rule_without_rule_id';
export * from './rule/get_simple_rule_without_rule_id';
export * from './rule/remove_server_generated_properties';
export * from './rule/remove_server_generated_properties_including_rule_id';
export * from './rule/get_simple_ml_rule';
export * from './rule/get_simple_ml_rule_output';
export * from './rule/wait_for_rule_status';
export * from './rule/get_rule_for_alert_testing_with_timestamp_override';
export * from './rule/get_rule_for_alert_testing';
export * from './rule/get_threshold_rule_for_alert_testing';
export * from './rule/get_rule_actions';
export * from './exception_list_and_item/exception_list/create_exception_list';
export * from './exception_list_and_item/exception_list/delete_exception_list';
// TODO rename signal to alert
export * from './alert/create_alerts_index';
export * from './alert/delete_all_alerts';
export * from './alert/wait_for_alert_to_complete';
export * from './alert/wait_for_alerts_to_be_present';
export * from './alert/wait_for_alert_to_complete';
export * from './action/get_slack_action';
export * from './action/get_web_hook_action';
export * from './action/remove_uuid_from_actions';
export * from './count_down_test';
export * from './update_username';

14
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/route_with_namespace.ts Normal file
View file

@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
/**
* Generates a route string with an optional namespace.
* @param route the route string
* @param namespace [optional] the namespace to account for in the route
*/
export const routeWithNamespace = (route: string, namespace?: string) =>
namespace ? `/s/${namespace}${route}` : route;

74
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/create_rule.ts Normal file
View file

@ -0,0 +1,74 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import type {
RuleCreateProps,
RuleResponse,
} from '@kbn/security-solution-plugin/common/api/detection_engine';
import { DETECTION_ENGINE_RULES_URL } from '@kbn/security-solution-plugin/common/constants';
import { deleteRule } from './delete_rule';
import { routeWithNamespace } from '../route_with_namespace';
/**
* Helper to cut down on the noise in some of the tests. If this detects
* a conflict it will try to manually remove the rule before re-adding the rule one time and log
* and error about the race condition.
* rule a second attempt. It only re-tries adding the rule if it encounters a conflict once.
* @param supertest The supertest deps
* @param log The tooling logger
* @param rule The rule to create
*/
export const createRule = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
rule: RuleCreateProps,
namespace?: string
): Promise<RuleResponse> => {
const route = routeWithNamespace(DETECTION_ENGINE_RULES_URL, namespace);
const response = await supertest
.post(route)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.send(rule);
if (response.status === 409) {
if (rule.rule_id != null) {
log.debug(
`Did not get an expected 200 "ok" when creating a rule (createRule). CI issues could happen. Suspect this line if you are seeing CI issues. body: ${JSON.stringify(
response.body
)}, status: ${JSON.stringify(response.status)}`
);
await deleteRule(supertest, rule.rule_id);
const secondResponseTry = await supertest
.post(DETECTION_ENGINE_RULES_URL)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.send(rule);
if (secondResponseTry.status !== 200) {
throw new Error(
`Unexpected non 200 ok when attempting to create a rule (second try): ${JSON.stringify(
response.body
)}`
);
} else {
return secondResponseTry.body;
}
} else {
throw new Error('When creating a rule found an unexpected conflict (404)');
}
} else if (response.status !== 200) {
throw new Error(
`Unexpected non 200 ok when attempting to create a rule: ${JSON.stringify(
response.status
)},${JSON.stringify(response, null, 4)}`
);
} else {
return response.body;
}
};

47
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/delete_all_rules.ts Normal file
View file

@ -0,0 +1,47 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import {
DETECTION_ENGINE_RULES_BULK_ACTION,
DETECTION_ENGINE_RULES_URL,
} from '@kbn/security-solution-plugin/common/constants';
import { countDownTest } from '../count_down_test';
/**
* Removes all rules by looping over any found and removing them from REST.
* @param supertest The supertest agent.
*/
export const deleteAllRules = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog
): Promise<void> => {
await countDownTest(
async () => {
await supertest
.post(DETECTION_ENGINE_RULES_BULK_ACTION)
.send({ action: 'delete', query: '' })
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31');
const { body: finalCheck } = await supertest
.get(`${DETECTION_ENGINE_RULES_URL}/_find`)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.send();
return {
passed: finalCheck.data.length === 0,
};
},
'deleteAllRules',
log,
50,
1000
);
};

30
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/delete_rule.ts Normal file
View file

@ -0,0 +1,30 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type SuperTest from 'supertest';
import type { RuleResponse } from '@kbn/security-solution-plugin/common/api/detection_engine';
import { DETECTION_ENGINE_RULES_URL } from '@kbn/security-solution-plugin/common/constants';
/**
* Helper to cut down on the noise in some of the tests. Does a delete of a rule.
* It does not check for a 200 "ok" on this.
* @param supertest The supertest deps
* @param ruleId The rule id to delete
*/
export const deleteRule = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
ruleId: string
): Promise<RuleResponse> => {
const response = await supertest
.delete(`${DETECTION_ENGINE_RULES_URL}?rule_id=${ruleId}`)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.expect(200);
return response.body;
};

38
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_rule.ts Normal file
View file

@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import type { RuleResponse } from '@kbn/security-solution-plugin/common/api/detection_engine';
import { DETECTION_ENGINE_RULES_URL } from '@kbn/security-solution-plugin/common/constants';
/**
* Helper to cut down on the noise in some of the tests. This gets
* a particular rule.
* @param supertest The supertest deps
* @param rule The rule to create
*/
export const getRule = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
log: ToolingLog,
ruleId: string
): Promise<RuleResponse> => {
const response = await supertest
.get(`${DETECTION_ENGINE_RULES_URL}?rule_id=${ruleId}`)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31');
if (response.status !== 200) {
log.error(
`Did not get an expected 200 "ok" when getting a rule (getRule). CI issues could happen. Suspect this line if you are seeing CI issues. body: ${JSON.stringify(
response.body
)}, status: ${JSON.stringify(response.status)}`
);
}
return response.body;
};

96
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_rule_actions.ts Normal file
View file

@ -0,0 +1,96 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type SuperTest from 'supertest';
import { RuleActionArray } from '@kbn/securitysolution-io-ts-alerting-types';
import { getSlackAction } from '..';
import { getWebHookAction } from '..';
const createConnector = async (
supertest: SuperTest.SuperTest<SuperTest.Test>,
payload: Record<string, unknown>
) =>
(await supertest.post('/api/actions/action').set('kbn-xsrf', 'true').send(payload).expect(200))
.body;
const createWebHookConnector = (supertest: SuperTest.SuperTest<SuperTest.Test>) =>
createConnector(supertest, getWebHookAction());
const createSlackConnector = (supertest: SuperTest.SuperTest<SuperTest.Test>) =>
createConnector(supertest, getSlackAction());
export const getActionsWithoutFrequencies = async (
supertest: SuperTest.SuperTest<SuperTest.Test>
): Promise<RuleActionArray> => {
const webHookAction = await createWebHookConnector(supertest);
const slackConnector = await createSlackConnector(supertest);
return [
{
group: 'default',
id: webHookAction.id,
action_type_id: '.webhook',
params: { message: 'Email message' },
},
{
group: 'default',
id: slackConnector.id,
action_type_id: '.slack',
params: { message: 'Slack message' },
},
];
};
export const getActionsWithFrequencies = async (
supertest: SuperTest.SuperTest<SuperTest.Test>
): Promise<RuleActionArray> => {
const webHookAction = await createWebHookConnector(supertest);
const slackConnector = await createSlackConnector(supertest);
return [
{
group: 'default',
id: webHookAction.id,
action_type_id: '.webhook',
params: { message: 'Email message' },
frequency: { summary: true, throttle: null, notifyWhen: 'onActiveAlert' },
},
{
group: 'default',
id: slackConnector.id,
action_type_id: '.slack',
params: { message: 'Slack message' },
frequency: { summary: false, throttle: '3d', notifyWhen: 'onThrottleInterval' },
},
];
};
export const getSomeActionsWithFrequencies = async (
supertest: SuperTest.SuperTest<SuperTest.Test>
): Promise<RuleActionArray> => {
const webHookAction = await createWebHookConnector(supertest);
const slackConnector = await createSlackConnector(supertest);
return [
{
group: 'default',
id: webHookAction.id,
action_type_id: '.webhook',
params: { message: 'Email message' },
frequency: { summary: true, throttle: null, notifyWhen: 'onActiveAlert' },
},
{
group: 'default',
id: slackConnector.id,
action_type_id: '.slack',
params: { message: 'Slack message' },
frequency: { summary: false, throttle: '3d', notifyWhen: 'onThrottleInterval' },
},
{
group: 'default',
id: slackConnector.id,
action_type_id: '.slack',
params: { message: 'Slack message' },
},
];
};

32
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_rule_for_alert_testing.ts Normal file
View file

@ -0,0 +1,32 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { QueryRuleCreateProps } from '@kbn/security-solution-plugin/common/api/detection_engine';
/**
* This is a typical signal testing rule that is easy for most basic testing of output of alerts.
* It starts out in an enabled true state. The 'from' is set very far back to test the basics of signal
* creation and testing by getting all the alerts at once.
* @param ruleId The optional ruleId which is rule-1 by default.
* @param enabled Enables the rule on creation or not. Defaulted to true.
*/
export const getRuleForAlertTesting = (
index: string[],
ruleId = 'rule-1',
enabled = true
): QueryRuleCreateProps => ({
name: 'Signal Testing Query',
description: 'Tests a simple query',
enabled,
risk_score: 1,
rule_id: ruleId,
severity: 'high',
index,
type: 'query',
query: '*:*',
from: '1900-01-01T00:00:00.000Z',
});

27
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_rule_for_alert_testing_with_timestamp_override.ts Normal file
View file

@ -0,0 +1,27 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { QueryRuleCreateProps } from '@kbn/security-solution-plugin/common/api/detection_engine';
export const getRuleForAlertTestingWithTimestampOverride = (
index: string[],
ruleId = 'rule-1',
enabled = true,
timestampOverride = 'event.ingested'
): QueryRuleCreateProps => ({
name: 'Signal Testing Query',
description: 'Tests a simple query',
enabled,
risk_score: 1,
rule_id: ruleId,
severity: 'high',
index,
type: 'query',
query: '*:*',
timestamp_override: timestampOverride,
from: '1900-01-01T00:00:00.000Z',
});

25
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_simple_ml_rule.ts Normal file
View file

@ -0,0 +1,25 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { RuleCreateProps } from '@kbn/security-solution-plugin/common/api/detection_engine';
/**
* This is a representative ML rule payload as expected by the server
* @param ruleId The rule id
* @param enabled Set to tru to enable it, by default it is off
*/
export const getSimpleMlRule = (ruleId = 'rule-1', enabled = false): RuleCreateProps => ({
name: 'Simple ML Rule',
description: 'Simple Machine Learning Rule',
enabled,
anomaly_threshold: 44,
risk_score: 1,
rule_id: ruleId,
severity: 'high',
machine_learning_job_id: ['some_job_id'],
type: 'machine_learning',
});

25
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_simple_ml_rule_output.ts Normal file
View file

@ -0,0 +1,25 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { MachineLearningRule } from '@kbn/security-solution-plugin/common/api/detection_engine';
import { getMockSharedResponseSchema } from './get_simple_rule_output';
import { removeServerGeneratedProperties } from './remove_server_generated_properties';
const getBaseMlRuleOutput = (ruleId = 'rule-1'): MachineLearningRule => {
return {
...getMockSharedResponseSchema(ruleId),
name: 'Simple ML Rule',
description: 'Simple Machine Learning Rule',
anomaly_threshold: 44,
machine_learning_job_id: ['some_job_id'],
type: 'machine_learning',
};
};
export const getSimpleMlRuleOutput = (ruleId = 'rule-1') => {
return removeServerGeneratedProperties(getBaseMlRuleOutput(ruleId));
};

25
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_simple_rule.ts Normal file
View file

@ -0,0 +1,25 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { QueryRuleCreateProps } from '@kbn/security-solution-plugin/common/api/detection_engine';
/**
* This is a typical simple rule for testing that is easy for most basic testing
* @param ruleId
* @param enabled Enables the rule on creation or not. Defaulted to true.
*/
export const getSimpleRule = (ruleId = 'rule-1', enabled = false): QueryRuleCreateProps => ({
name: 'Simple Rule Query',
description: 'Simple Rule Query',
enabled,
risk_score: 1,
rule_id: ruleId,
severity: 'high',
index: ['auditbeat-*'],
type: 'query',
query: 'user.name: root or user.name: admin',
});

85
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_simple_rule_output.ts Normal file
View file

@ -0,0 +1,85 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type {
RuleResponse,
SharedResponseProps,
} from '@kbn/security-solution-plugin/common/api/detection_engine';
import { removeServerGeneratedProperties } from './remove_server_generated_properties';
export const getMockSharedResponseSchema = (
ruleId = 'rule-1',
enabled = false
): SharedResponseProps => ({
actions: [],
author: [],
created_by: 'elastic',
description: 'Simple Rule Query',
enabled,
false_positives: [],
from: 'now-6m',
immutable: false,
interval: '5m',
rule_id: ruleId,
output_index: '',
max_signals: 100,
related_integrations: [],
required_fields: [],
risk_score: 1,
risk_score_mapping: [],
name: 'Simple Rule Query',
references: [],
setup: '',
severity: 'high' as const,
severity_mapping: [],
updated_by: 'elastic',
tags: [],
to: 'now',
threat: [],
throttle: undefined,
exceptions_list: [],
version: 1,
revision: 0,
id: 'id',
updated_at: '2020-07-08T16:36:32.377Z',
created_at: '2020-07-08T16:36:32.377Z',
building_block_type: undefined,
note: undefined,
license: undefined,
outcome: undefined,
alias_target_id: undefined,
alias_purpose: undefined,
timeline_id: undefined,
timeline_title: undefined,
meta: undefined,
rule_name_override: undefined,
timestamp_override: undefined,
timestamp_override_fallback_disabled: undefined,
namespace: undefined,
investigation_fields: undefined,
});
const getQueryRuleOutput = (ruleId = 'rule-1', enabled = false): RuleResponse => ({
...getMockSharedResponseSchema(ruleId, enabled),
index: ['auditbeat-*'],
language: 'kuery',
query: 'user.name: root or user.name: admin',
type: 'query',
data_view_id: undefined,
filters: undefined,
saved_id: undefined,
response_actions: undefined,
alert_suppression: undefined,
});
/**
* This is the typical output of a simple rule that Kibana will output with all the defaults
* except for the server generated properties. Useful for testing end to end tests.
*/
export const getSimpleRuleOutput = (ruleId = 'rule-1', enabled = false) => {
return removeServerGeneratedProperties(getQueryRuleOutput(ruleId, enabled));
};

21
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_simple_rule_output_without_rule_id.ts Normal file
View file

@ -0,0 +1,21 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { getSimpleRuleOutput } from './get_simple_rule_output';
import { RuleWithoutServerGeneratedProperties } from './remove_server_generated_properties';
/**
* This is the typical output of a simple rule that Kibana will output with all the defaults except
* for all the server generated properties such as created_by. Useful for testing end to end tests.
*/
export const getSimpleRuleOutputWithoutRuleId = (
ruleId = 'rule-1'
): Omit<RuleWithoutServerGeneratedProperties, 'rule_id'> => {
const rule = getSimpleRuleOutput(ruleId);
const { rule_id: rId, ...ruleWithoutRuleId } = rule;
return ruleWithoutRuleId;
};

19
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_simple_rule_without_rule_id.ts Normal file
View file

@ -0,0 +1,19 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { RuleCreateProps } from '@kbn/security-solution-plugin/common/api/detection_engine';
import { getSimpleRule } from './get_simple_rule';
/**
* This is a typical simple rule for testing that is easy for most basic testing
*/
export const getSimpleRuleWithoutRuleId = (): RuleCreateProps => {
const simpleRule = getSimpleRule();
// eslint-disable-next-line @typescript-eslint/naming-convention
const { rule_id, ...ruleWithoutId } = simpleRule;
return ruleWithoutId;
};

31
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/get_threshold_rule_for_alert_testing.ts Normal file
View file

@ -0,0 +1,31 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ThresholdRuleCreateProps } from '@kbn/security-solution-plugin/common/api/detection_engine';
import { getRuleForAlertTesting } from './get_rule_for_alert_testing';
/**
* This is a typical signal testing rule that is easy for most basic testing of output of Threshold alerts.
* It starts out in an enabled true state. The 'from' is set very far back to test the basics of signal
* creation for Threshold and testing by getting all the alerts at once.
* @param ruleId The optional ruleId which is threshold-rule by default.
* @param enabled Enables the rule on creation or not. Defaulted to true.
*/
export const getThresholdRuleForAlertTesting = (
index: string[],
ruleId = 'threshold-rule',
enabled = true
): ThresholdRuleCreateProps => ({
...getRuleForAlertTesting(index, ruleId, enabled),
type: 'threshold',
language: 'kuery',
query: '*:*',
threshold: {
field: 'process.name',
value: 21,
},
});

29
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/remove_server_generated_properties.ts Normal file
View file

@ -0,0 +1,29 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { RuleResponse } from '@kbn/security-solution-plugin/common/api/detection_engine';
import { omit, pickBy } from 'lodash';
const serverGeneratedProperties = ['id', 'created_at', 'updated_at', 'execution_summary'] as const;
type ServerGeneratedProperties = typeof serverGeneratedProperties[number];
export type RuleWithoutServerGeneratedProperties = Omit<RuleResponse, ServerGeneratedProperties>;
/**
* This will remove server generated properties such as date times, etc...
* @param rule Rule to pass in to remove typical server generated properties
*/
export const removeServerGeneratedProperties = (
rule: RuleResponse
): RuleWithoutServerGeneratedProperties => {
const removedProperties = omit(rule, serverGeneratedProperties);
// We're only removing undefined values, so this cast correctly narrows the type
return pickBy(
removedProperties,
(value) => value !== undefined
) as RuleWithoutServerGeneratedProperties;
};

23
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/remove_server_generated_properties_including_rule_id.ts Normal file
View file

@ -0,0 +1,23 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { RuleResponse } from '@kbn/security-solution-plugin/common/api/detection_engine';
import { removeServerGeneratedProperties } from './remove_server_generated_properties';
/**
* This will remove server generated properties such as date times, etc... including the rule_id
* @param rule Rule to pass in to remove typical server generated properties
*/
export const removeServerGeneratedPropertiesIncludingRuleId = (
rule: RuleResponse
): Partial<RuleResponse> => {
const ruleWithRemovedProperties = removeServerGeneratedProperties(rule);
// eslint-disable-next-line @typescript-eslint/naming-convention
const { rule_id, ...additionalRuledIdRemoved } = ruleWithRemovedProperties;
return additionalRuledIdRemoved;
};

82
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/rule/wait_for_rule_status.ts Normal file
View file

@ -0,0 +1,82 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
import type SuperTest from 'supertest';
import { DETECTION_ENGINE_RULES_URL } from '@kbn/security-solution-plugin/common/constants';
import {
RuleExecutionStatus,
RuleExecutionStatusEnum,
} from '@kbn/security-solution-plugin/common/api/detection_engine/rule_monitoring';
import { waitFor } from '../wait_for';
import { routeWithNamespace } from '../route_with_namespace';
interface WaitForRuleStatusBaseParams {
supertest: SuperTest.SuperTest<SuperTest.Test>;
log: ToolingLog;
afterDate?: Date;
namespace?: string;
}
interface WaitForRuleStatusWithId extends WaitForRuleStatusBaseParams {
id: string;
ruleId?: never;
}
interface WaitForRuleStatusWithRuleId extends WaitForRuleStatusBaseParams {
ruleId: string;
id?: never;
}
export type WaitForRuleStatusParams = WaitForRuleStatusWithId | WaitForRuleStatusWithRuleId;
/**
* Waits for rule to settle in a provided status.
* Depending on wether `id` or `ruleId` provided it may impact the behavior.
* - `id` leads to fetching a rule via ES Get API (rulesClient.resolve -> SOClient.resolve -> ES Get API)
* - `ruleId` leads to fetching a rule via ES Search API (rulesClient.find -> SOClient.find -> ES Search API)
* ES Search API may return outdated data while ES Get API always returns fresh data
*/
export const waitForRuleStatus = async (
expectedStatus: RuleExecutionStatus,
{ supertest, log, afterDate, namespace, ...idOrRuleId }: WaitForRuleStatusParams
): Promise<void> => {
await waitFor(
async () => {
const query = 'id' in idOrRuleId ? { id: idOrRuleId.id } : { rule_id: idOrRuleId.ruleId };
const route = routeWithNamespace(DETECTION_ENGINE_RULES_URL, namespace);
const response = await supertest
.get(route)
.set('kbn-xsrf', 'true')
.set('elastic-api-version', '2023-10-31')
.query(query)
.expect(200);
// TODO: https://github.com/elastic/kibana/pull/121644 clean up, make type-safe
const rule = response.body;
const ruleStatus = rule?.execution_summary?.last_execution.status;
const ruleStatusDate = rule?.execution_summary?.last_execution.date;
return (
rule != null &&
ruleStatus === expectedStatus &&
(afterDate ? new Date(ruleStatusDate) > afterDate : true)
);
},
'waitForRuleStatus',
log
);
};
export const waitForRuleSuccess = (params: WaitForRuleStatusParams): Promise<void> =>
waitForRuleStatus(RuleExecutionStatusEnum.succeeded, params);
export const waitForRulePartialFailure = (params: WaitForRuleStatusParams): Promise<void> =>
waitForRuleStatus(RuleExecutionStatusEnum['partial failure'], params);
export const waitForRuleFailure = (params: WaitForRuleStatusParams): Promise<void> =>
waitForRuleStatus(RuleExecutionStatusEnum.failed, params);

16
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/update_username.ts Normal file
View file

@ -0,0 +1,16 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export const updateUsername = <T extends Record<string, any>>(
entity: T,
username: string
): T & { created_by: string; updated_by: string } => {
return {
...entity,
created_by: username,
updated_by: username,
};
};

35
x-pack/test/security_solution_api_integration/test_suites/detections_response/utils/wait_for.ts Normal file
View file

@ -0,0 +1,35 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import type { ToolingLog } from '@kbn/tooling-log';
// Similar to ReactJs's waitFor from here: https://testing-library.com/docs/dom-testing-library/api-async#waitfor
export const waitFor = async (
functionToTest: () => Promise<boolean>,
functionName: string,
log: ToolingLog,
maxTimeout: number = 400000,
timeoutWait: number = 250
): Promise<void> => {
let found = false;
let numberOfTries = 0;
const maxTries = Math.floor(maxTimeout / timeoutWait);
while (!found && numberOfTries < maxTries) {
if (await functionToTest()) {
found = true;
} else {
log.debug(`Try number ${numberOfTries} out of ${maxTries} for function ${functionName}`);
numberOfTries++;
}
await new Promise((resolveTimeout) => setTimeout(resolveTimeout, timeoutWait));
}
if (!found) {
throw new Error(`timed out waiting for function condition to be true within ${functionName}`);
}
};

14
x-pack/test/security_solution_api_integration/test_suites/index.ts Normal file
View file

@ -0,0 +1,14 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
import { FtrProviderContext } from '../ftr_provider_context';
export default function ({ loadTestFile }: FtrProviderContext) {
describe('Security solution API', function () {
loadTestFile(require.resolve('./detections_response'));
});
}

31
x-pack/test/security_solution_api_integration/tsconfig.json Normal file
View file

@ -0,0 +1,31 @@
{
"extends": "../../../tsconfig.base.json",
"compilerOptions": {
"outDir": "target/types",
"types": ["node", "jest","@kbn/ambient-ftr-types"]
},
"include": [
"**/*",
"../../../typings/**/*",
"../../../packages/kbn-test/types/ftr_globals/**/*",
],
"exclude": [
"target/**/*"
],
"kbn_references": [
{ "path": "../../test_serverless/tsconfig.json" },
{ "path": "../../test_serverless/api_integration/**/*" },
{ "path": "../../test_serverless/shared/**/*" },
{ "path": "../../api_integration/services/**/*" },
"@kbn/dev-utils",
"@kbn/test",
"@kbn/expect",
"@kbn/security-solution-plugin",
"@kbn/securitysolution-io-ts-list-types",
"@kbn/lists-plugin",
"@kbn/securitysolution-io-ts-alerting-types",
"@kbn/tooling-log",
"@kbn/rule-data-utils",
"@kbn/securitysolution-list-constants"
]
}

2
x-pack/test/tsconfig.json
View file

@ -18,7 +18,7 @@
"../../typings/**/*",
"../../packages/kbn-test/types/ftr_globals/**/*"
],
"exclude": ["security_solution_cypress/cypress/**/*", "target/**/*", "*/plugins/**/*", "*/packages/**/*", "*/*/packages/**/*" ],
"exclude": ["security_solution_cypress/cypress/**/*", "target/**/*", "*/plugins/**/*", "*/packages/**/*", "*/*/packages/**/*","security_solution_api_integration/**/*" ],
"kbn_references": [
{ "path": "../../test/tsconfig.json" },
"@kbn/core",