mirror of
https://github.com/elastic/kibana.git
synced 2025-06-27 18:51:07 -04:00
[Automatic Migrations] Siem migrations new endpoint (#219597)
## Summary Resolves https://github.com/elastic/security-team/issues/12483 This PR changes REST API Endpoints scheme to align with https://github.com/elastic/security-team/issues/12483. Below is the summary of changes done. ### API Scheme changes The REST API scheme has been changed to reflect https://github.com/elastic/security-team/issues/12483. This is pretty much self explanatory as defined in below openapi schema yaml : - [x-pack/solutions/security/plugins/security_solution/common/siem_migrations/model/api/rules/rule_migration.schema.yaml](https://github.com/elastic/kibana/pull/219597/files#diff-3025af9eca156f3308474e2b42808da1531423457b7791daf6660db95a53b978) ### Introduction of Delete Migration API This PR also adds `DELETE` method on route `/rules/siem_migrations/{migration_id}` for deleting a migrations. Deleting a migration does below operations: - Stops a migration if it is running - Deletes the rules, resources related to migration and migration document itself. ### File Reorganizations Directly structure has been changed a little bit to reflect the endpoint. There is a sub-directory called `rules` which deals with only `rules` of the migration and the root directly only contains the endpoints related to the migration. #### Before ``` //siem_migrations/rules/api ├── create.ts ├── get.ts ├── start.ts ├── update.ts ├── ├── ├── ``` #### After ``` //siem_migrations/rules/api ├── create.ts ├── delete.ts ├── get.ts ├── rules │ ├── add.ts │ ├── get.ts │ └── update.ts ├── ├── ├── ``` ## Migration Strategy ### TL,DR; ```mermaid flowchart TD StartM[Start Migration] --> isMigExists{Does Migration Index Exists} isMigExists -->|Yes|FetchMDoc[Fetch Migration Docs] isMigExists -->|No|CreateMIndex[Create MigrationIndex] CreateMIndex --> FetchMDoc FetchMDoc --> FetchMRules[Fetch Migration Stats Rules index] FetchMRules --> FilterMigration{Filter Migration Docs not in Migration Index} FilterMigration --> |is Empty|END[END] FilterMigration --> |is Not Empty| CreateMDocs[Create Migration Docs] CreateMDocs --> END ``` At the time of merging this PR, the Migration indices can be in 3 states: ### There are migrations created after https://github.com/elastic/kibana/pull/216164 and this means that there are `some` migrations existing in `.kibana-siem-migrations-migrations-<space_id>` and migrations created before above mentioned PR will only exist in `.kibana-siem-migrations-rules-<space_id>`. In this case `migrateRuleMigrationIndex` will create migration in below steps: 1. Look for **all** migration Documents in `.kibana-siem-migrations-migrations-<space_id>` 2. Get **all** Migrations stats from `.kibana-siem-migrations-rules-<space_id>` which includes below properties - migration_id : will help in reconciling the migration id in .kibana-siem-migrations-migrations-<space_id>` index - created_at : Date on which migration_id was created. - created_by: User who created the migrations. 3. A new document with above migration will be added to `.kibana-siem-migrations-migrations-<space_id>`. 4. Now both `.kibana-siem-migrations-migrations-<space_id>` and `.kibana-siem-migrations-rules-<space_id>` will be in sync. ### Alternatively, there are no migration created after https://github.com/elastic/kibana/pull/216164. In that case, there is a possibility that `.kibana-siem-migrations-migrations-<space_id>`, will not even exist. In this case `migrateRuleMigrationIndex` will create migration in below steps: 1. Create the `.kibana-siem-migrations-migrations-<space_id>` index. 2. Do steps mentioned in above scenario. ### Once the migrations has been run successfully, both `.kibana-siem-migrations-migrations-<space_id>` index and `.kibana-siem-migrations-rules-<space_id>` will be in sync. 1. In this case, migration will not run, since it tries to filter the migrations by `id` which exist in `kibana-siem-migrations-rules-<space_id>` but do not exist in `kibana-siem-migrations-migrations-<space_id>` ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Jatin Kathuria
GitHub
parent
a09b95d753
commit
bf642b0039
119 changed files with 4580 additions and 1604 deletions
|
|
@ -30,8 +30,8 @@ import { CreateAssetCriticalityRecordRequestBodyInput } from '@kbn/security-solu
|
|||
import { CreatePrivMonUserRequestBodyInput } from '@kbn/security-solution-plugin/common/api/entity_analytics/privilege_monitoring/users/create.gen';
|
||||
import { CreateRuleRequestBodyInput } from '@kbn/security-solution-plugin/common/api/detection_engine/rule_management/crud/create_rule/create_rule_route.gen';
|
||||
import {
|
||||
CreateRuleMigrationRequestParamsInput,
|
||||
CreateRuleMigrationRequestBodyInput,
|
||||
CreateRuleMigrationRulesRequestParamsInput,
|
||||
CreateRuleMigrationRulesRequestBodyInput,
|
||||
} from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { CreateTimelinesRequestBodyInput } from '@kbn/security-solution-plugin/common/api/timeline/create_timelines/create_timelines_route.gen';
|
||||
import {
|
||||
|
|
@ -46,6 +46,7 @@ import {
|
|||
import { DeleteNoteRequestBodyInput } from '@kbn/security-solution-plugin/common/api/timeline/delete_note/delete_note_route.gen';
|
||||
import { DeletePrivMonUserRequestParamsInput } from '@kbn/security-solution-plugin/common/api/entity_analytics/privilege_monitoring/users/delete.gen';
|
||||
import { DeleteRuleRequestQueryInput } from '@kbn/security-solution-plugin/common/api/detection_engine/rule_management/crud/delete_rule/delete_rule_route.gen';
|
||||
import { DeleteRuleMigrationRequestParamsInput } from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { DeleteTimelinesRequestBodyInput } from '@kbn/security-solution-plugin/common/api/timeline/delete_timelines/delete_timelines_route.gen';
|
||||
import { DeprecatedTriggerRiskScoreCalculationRequestBodyInput } from '@kbn/security-solution-plugin/common/api/entity_analytics/risk_engine/entity_calculation_route.gen';
|
||||
import { EndpointExecuteActionRequestBodyInput } from '@kbn/security-solution-plugin/common/api/endpoint/actions/response_actions/execute/execute.gen';
|
||||
|
|
@ -93,16 +94,17 @@ import {
|
|||
GetRuleExecutionResultsRequestQueryInput,
|
||||
GetRuleExecutionResultsRequestParamsInput,
|
||||
} from '@kbn/security-solution-plugin/common/api/detection_engine/rule_monitoring/rule_execution_logs/get_rule_execution_results/get_rule_execution_results_route.gen';
|
||||
import {
|
||||
GetRuleMigrationRequestQueryInput,
|
||||
GetRuleMigrationRequestParamsInput,
|
||||
} from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { GetRuleMigrationRequestParamsInput } from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { GetRuleMigrationPrebuiltRulesRequestParamsInput } from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import {
|
||||
GetRuleMigrationResourcesRequestQueryInput,
|
||||
GetRuleMigrationResourcesRequestParamsInput,
|
||||
} from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { GetRuleMigrationResourcesMissingRequestParamsInput } from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import {
|
||||
GetRuleMigrationRulesRequestQueryInput,
|
||||
GetRuleMigrationRulesRequestParamsInput,
|
||||
} from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { GetRuleMigrationStatsRequestParamsInput } from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { GetRuleMigrationTranslationStatsRequestParamsInput } from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import { GetTimelineRequestQueryInput } from '@kbn/security-solution-plugin/common/api/timeline/get_timeline/get_timeline_route.gen';
|
||||
|
|
@ -159,9 +161,10 @@ import {
|
|||
UpdatePrivMonUserRequestBodyInput,
|
||||
} from '@kbn/security-solution-plugin/common/api/entity_analytics/privilege_monitoring/users/update.gen';
|
||||
import { UpdateRuleRequestBodyInput } from '@kbn/security-solution-plugin/common/api/detection_engine/rule_management/crud/update_rule/update_rule_route.gen';
|
||||
import { UpdateRuleMigrationRequestParamsInput } from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import {
|
||||
UpdateRuleMigrationRequestParamsInput,
|
||||
UpdateRuleMigrationRequestBodyInput,
|
||||
UpdateRuleMigrationRulesRequestParamsInput,
|
||||
UpdateRuleMigrationRulesRequestBodyInput,
|
||||
} from '@kbn/security-solution-plugin/common/siem_migrations/model/api/rules/rule_migration.gen';
|
||||
import {
|
||||
UpdateWorkflowInsightRequestParamsInput,
|
||||
|
|
@ -414,13 +417,26 @@ For detailed information on Kibana actions and alerting, and additional API call
|
|||
.send(props.body as object);
|
||||
},
|
||||
/**
|
||||
* Creates a new SIEM rules migration using the original vendor rules provided
|
||||
* Creates a new rule migration and returns the corresponding migration_id
|
||||
*/
|
||||
createRuleMigration(props: CreateRuleMigrationProps, kibanaSpace: string = 'default') {
|
||||
createRuleMigration(kibanaSpace: string = 'default') {
|
||||
return supertest
|
||||
.put(routeWithNamespace('/internal/siem_migrations/rules', kibanaSpace))
|
||||
.set('kbn-xsrf', 'true')
|
||||
.set(ELASTIC_HTTP_VERSION_HEADER, '1')
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana');
|
||||
},
|
||||
/**
|
||||
* Adds original vendor rules to an already existing migration. Can be called multiple times to add more rules
|
||||
*/
|
||||
createRuleMigrationRules(
|
||||
props: CreateRuleMigrationRulesProps,
|
||||
kibanaSpace: string = 'default'
|
||||
) {
|
||||
return supertest
|
||||
.post(
|
||||
routeWithNamespace(
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}', props.params),
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}/rules', props.params),
|
||||
kibanaSpace
|
||||
)
|
||||
)
|
||||
|
|
@ -535,6 +551,21 @@ The difference between the `id` and `rule_id` is that the `id` is a unique rule
|
|||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana')
|
||||
.query(props.query);
|
||||
},
|
||||
/**
|
||||
* Deletes a rule migration document stored in the system given the rule migration id
|
||||
*/
|
||||
deleteRuleMigration(props: DeleteRuleMigrationProps, kibanaSpace: string = 'default') {
|
||||
return supertest
|
||||
.delete(
|
||||
routeWithNamespace(
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}', props.params),
|
||||
kibanaSpace
|
||||
)
|
||||
)
|
||||
.set('kbn-xsrf', 'true')
|
||||
.set(ELASTIC_HTTP_VERSION_HEADER, '1')
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana');
|
||||
},
|
||||
/**
|
||||
* Delete one or more Timelines or Timeline templates.
|
||||
*/
|
||||
|
|
@ -1018,7 +1049,7 @@ finalize it.
|
|||
.query(props.query);
|
||||
},
|
||||
/**
|
||||
* Retrieves the rule documents stored in the system given the rule migration id
|
||||
* Retrieves the rule migration document stored in the system given the rule migration id
|
||||
*/
|
||||
getRuleMigration(props: GetRuleMigrationProps, kibanaSpace: string = 'default') {
|
||||
return supertest
|
||||
|
|
@ -1030,8 +1061,7 @@ finalize it.
|
|||
)
|
||||
.set('kbn-xsrf', 'true')
|
||||
.set(ELASTIC_HTTP_VERSION_HEADER, '1')
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana')
|
||||
.query(props.query);
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana');
|
||||
},
|
||||
/**
|
||||
* Retrieves all related integrations
|
||||
|
|
@ -1114,6 +1144,22 @@ finalize it.
|
|||
.set(ELASTIC_HTTP_VERSION_HEADER, '1')
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana');
|
||||
},
|
||||
/**
|
||||
* Retrieves the the list of rules included in a migration given the migration id
|
||||
*/
|
||||
getRuleMigrationRules(props: GetRuleMigrationRulesProps, kibanaSpace: string = 'default') {
|
||||
return supertest
|
||||
.get(
|
||||
routeWithNamespace(
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}/rules', props.params),
|
||||
kibanaSpace
|
||||
)
|
||||
)
|
||||
.set('kbn-xsrf', 'true')
|
||||
.set(ELASTIC_HTTP_VERSION_HEADER, '1')
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana')
|
||||
.query(props.query);
|
||||
},
|
||||
/**
|
||||
* Retrieves the stats of a SIEM rules migration using the migration id provided
|
||||
*/
|
||||
|
|
@ -1644,7 +1690,7 @@ The difference between the `id` and `rule_id` is that the `id` is a unique rule
|
|||
*/
|
||||
startRuleMigration(props: StartRuleMigrationProps, kibanaSpace: string = 'default') {
|
||||
return supertest
|
||||
.put(
|
||||
.post(
|
||||
routeWithNamespace(
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}/start', props.params),
|
||||
kibanaSpace
|
||||
|
|
@ -1672,7 +1718,7 @@ The difference between the `id` and `rule_id` is that the `id` is a unique rule
|
|||
*/
|
||||
stopRuleMigration(props: StopRuleMigrationProps, kibanaSpace: string = 'default') {
|
||||
return supertest
|
||||
.put(
|
||||
.post(
|
||||
routeWithNamespace(
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}/stop', props.params),
|
||||
kibanaSpace
|
||||
|
|
@ -1739,11 +1785,11 @@ The difference between the `id` and `rule_id` is that the `id` is a unique rule
|
|||
.send(props.body as object);
|
||||
},
|
||||
/**
|
||||
* Updates rules migrations attributes
|
||||
* Updates rules migrations data
|
||||
*/
|
||||
updateRuleMigration(props: UpdateRuleMigrationProps, kibanaSpace: string = 'default') {
|
||||
return supertest
|
||||
.put(
|
||||
.patch(
|
||||
routeWithNamespace(
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}', props.params),
|
||||
kibanaSpace
|
||||
|
|
@ -1751,6 +1797,24 @@ The difference between the `id` and `rule_id` is that the `id` is a unique rule
|
|||
)
|
||||
.set('kbn-xsrf', 'true')
|
||||
.set(ELASTIC_HTTP_VERSION_HEADER, '1')
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana');
|
||||
},
|
||||
/**
|
||||
* Updates rules migrations attributes
|
||||
*/
|
||||
updateRuleMigrationRules(
|
||||
props: UpdateRuleMigrationRulesProps,
|
||||
kibanaSpace: string = 'default'
|
||||
) {
|
||||
return supertest
|
||||
.patch(
|
||||
routeWithNamespace(
|
||||
replaceParams('/internal/siem_migrations/rules/{migration_id}/rules', props.params),
|
||||
kibanaSpace
|
||||
)
|
||||
)
|
||||
.set('kbn-xsrf', 'true')
|
||||
.set(ELASTIC_HTTP_VERSION_HEADER, '1')
|
||||
.set(X_ELASTIC_INTERNAL_ORIGIN_REQUEST, 'kibana')
|
||||
.send(props.body as object);
|
||||
},
|
||||
|
|
@ -1823,9 +1887,9 @@ export interface CreatePrivMonUserProps {
|
|||
export interface CreateRuleProps {
|
||||
body: CreateRuleRequestBodyInput;
|
||||
}
|
||||
export interface CreateRuleMigrationProps {
|
||||
params: CreateRuleMigrationRequestParamsInput;
|
||||
body: CreateRuleMigrationRequestBodyInput;
|
||||
export interface CreateRuleMigrationRulesProps {
|
||||
params: CreateRuleMigrationRulesRequestParamsInput;
|
||||
body: CreateRuleMigrationRulesRequestBodyInput;
|
||||
}
|
||||
export interface CreateTimelinesProps {
|
||||
body: CreateTimelinesRequestBodyInput;
|
||||
|
|
@ -1850,6 +1914,9 @@ export interface DeletePrivMonUserProps {
|
|||
export interface DeleteRuleProps {
|
||||
query: DeleteRuleRequestQueryInput;
|
||||
}
|
||||
export interface DeleteRuleMigrationProps {
|
||||
params: DeleteRuleMigrationRequestParamsInput;
|
||||
}
|
||||
export interface DeleteTimelinesProps {
|
||||
body: DeleteTimelinesRequestBodyInput;
|
||||
}
|
||||
|
|
@ -1952,7 +2019,6 @@ export interface GetRuleExecutionResultsProps {
|
|||
params: GetRuleExecutionResultsRequestParamsInput;
|
||||
}
|
||||
export interface GetRuleMigrationProps {
|
||||
query: GetRuleMigrationRequestQueryInput;
|
||||
params: GetRuleMigrationRequestParamsInput;
|
||||
}
|
||||
export interface GetRuleMigrationPrebuiltRulesProps {
|
||||
|
|
@ -1965,6 +2031,10 @@ export interface GetRuleMigrationResourcesProps {
|
|||
export interface GetRuleMigrationResourcesMissingProps {
|
||||
params: GetRuleMigrationResourcesMissingRequestParamsInput;
|
||||
}
|
||||
export interface GetRuleMigrationRulesProps {
|
||||
query: GetRuleMigrationRulesRequestQueryInput;
|
||||
params: GetRuleMigrationRulesRequestParamsInput;
|
||||
}
|
||||
export interface GetRuleMigrationStatsProps {
|
||||
params: GetRuleMigrationStatsRequestParamsInput;
|
||||
}
|
||||
|
|
@ -2087,7 +2157,10 @@ export interface UpdateRuleProps {
|
|||
}
|
||||
export interface UpdateRuleMigrationProps {
|
||||
params: UpdateRuleMigrationRequestParamsInput;
|
||||
body: UpdateRuleMigrationRequestBodyInput;
|
||||
}
|
||||
export interface UpdateRuleMigrationRulesProps {
|
||||
params: UpdateRuleMigrationRulesRequestParamsInput;
|
||||
body: UpdateRuleMigrationRulesRequestBodyInput;
|
||||
}
|
||||
export interface UpdateWorkflowInsightProps {
|
||||
params: UpdateWorkflowInsightRequestParamsInput;
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue