mirror of
https://github.com/elastic/kibana.git
synced 2025-06-27 10:40:07 -04:00
b03b2fd477
## Summary - related [epic](https://github.com/elastic/security-team/issues/6196) - introduces new ES|QL rule type in Technical Preview Stage - historical POC architecture [document](https://docs.google.com/document/d/1hcKzNrDEIrmoWwWoqas1YZ-bd8Kk5NRjJNSUaCvSntM/edit#heading=h.gheuu8zcz481)(internal link). Some of the information there can be outdated, but might be useful for historical context of some tech decision. In future, detailed technical documentation will be added ### UI ES|QL query component introduced in rule edit/creation form Rule name override supports values returned from ES|QL query As agreed on Adv. correlation WG, we don't introduce similar possibility for risk score/severity override at this point <details> <summary>How it looks like in UI</summary> <img width="2082" alt="Screenshot 2023-09-21 at 11 52 59" src="14c94e36-ca90-496d-a7a5-4a31899d25b6"> <img width="2079" alt="Screenshot 2023-09-21 at 11 53 14" src="9abd53ec-a0f4-4481-8b1f-4ecccdc5feae"> <img width="2072" alt="Screenshot 2023-09-21 at 12 14 17" src="58e4f9eb-c15f-4849-bba0-bc1b92e8c945"> </details> ### Context We introduced concept of Aggregating and Non-aggregating rules for ES|QL. It depends on, whether STATS..BY command used in query **Aggregating rule** - is a rule that uses [stats…by](https://esql.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/esql-stats-by.html) grouping commands. So, its result can not be matched to a particular document in ES. This can lead to possibly duplicated alerts, since we are using document `id` to deduplicate alerts. We are going to introduce suppression for all rule types in future, that would help to mitigate this case ``` FROM logs* | STATS count = COUNT(host.name) BY host.name | SORT host.name ``` **Non-aggregating rule** - is a rule that does not use [stats…by](https://esql.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/esql-stats-by.html) grouping commands. Each row in result can be tracked to a source document in ES. For this type of rule operator \`[metadata _id, _index, _version]\` is required to be used after defining index source. This would allow deduplicate alerts and link them with the source document. ``` FROM logs* [metadata _id, _index, _version] | WHERE event.id == "test" | LIMIT 10 ``` ### Serverless Feature Flag ES|QL won't be available for Serverless as for 8.11 release, so it will be hidden by Security experimental feature flag `esqlRulesDisabled`. All UI changes will be hidden (it's mostly Form creation) and rule type won't be registered, which prevents rule to be created, returned in search if it exists or execute. ### Test envs - Serverless qa, [admin link to project](https://admin.qa.cld.elstc.co/projects/security/ef79684f92d64f27b69e1b04de86eb1a), disabled there - internal [link](https://elastic.slack.com/archives/C03E8TR26HE/p1693848029955229) to test env for Stateful ### Rule schema changes introduces value `esql` to `type` property introduces value `esql` to `language` property ### Tests coverage - cypress tests (as per 27/09/2023 added cypress tests for rule creation/edit/details,bulk_edit)) - functional tests for rule execution(exceptions, overrides, preview and actual rule execution) - functional tests for bulk_edit #### Flaky test runner - [cypress esql tests](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3233#_), non failed of added ### Checklist Delete any items that are not applicable to this PR. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/)) - [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US)) - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server)) - [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
43 lines
1.4 KiB
YAML
43 lines
1.4 KiB
YAML
# Security Project config
|
|
|
|
## Disable plugins
|
|
enterpriseSearch.enabled: false
|
|
xpack.apm.enabled: false
|
|
xpack.infra.enabled: false
|
|
xpack.observabilityLogExplorer.enabled: false
|
|
xpack.observability.enabled: false
|
|
xpack.uptime.enabled: false
|
|
xpack.legacy_uptime.enabled: false
|
|
|
|
## Enable the Security Solution Serverless plugin
|
|
xpack.securitySolutionServerless.enabled: true
|
|
xpack.securitySolutionServerless.productTypes:
|
|
[
|
|
{ product_line: 'security', product_tier: 'complete' },
|
|
{ product_line: 'endpoint', product_tier: 'complete' },
|
|
]
|
|
|
|
## Set the home route
|
|
uiSettings.overrides.defaultRoute: /app/security/get_started
|
|
|
|
## Set the dev project switcher current type
|
|
xpack.serverless.plugin.developer.projectSwitcher.currentType: 'security'
|
|
|
|
# Specify in telemetry the project type
|
|
telemetry.labels.serverless: security
|
|
|
|
# Fleet specific configuration
|
|
xpack.fleet.internal.registry.capabilities: ['security']
|
|
xpack.fleet.internal.registry.spec.max: '3.0'
|
|
# Disabled until packages implement the new spec https://github.com/elastic/kibana/issues/166742
|
|
# xpack.fleet.internal.registry.kibanaVersionCheckEnabled: false
|
|
# xpack.fleet.internal.registry.spec.min: '3.0'
|
|
|
|
# Serverless security specific options
|
|
xpack.securitySolution.enableExperimental:
|
|
- discoverInTimeline
|
|
- esqlRulesDisabled
|
|
|
|
xpack.ml.ad.enabled: true
|
|
xpack.ml.dfa.enabled: true
|
|
xpack.ml.nlp.enabled: false
|