github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-06-27 18:51:07 -04:00
Code Issues Projects Releases Packages Wiki Activity
kibana/x-pack/packages/security-solution
History
Vitalii Dmyterko b03b2fd477
[Security Solution][Detection Engine] adds ES|QL rule type to Security Detections rules (#165450) 
## Summary

- related [epic](https://github.com/elastic/security-team/issues/6196)
- introduces new ES|QL rule type in Technical Preview Stage

- historical POC architecture
[document](https://docs.google.com/document/d/1hcKzNrDEIrmoWwWoqas1YZ-bd8Kk5NRjJNSUaCvSntM/edit#heading=h.gheuu8zcz481)(internal
link). Some of the information there can be outdated, but might be
useful for historical context of some tech decision. In future, detailed
technical documentation will be added

### UI

ES|QL query component introduced in rule edit/creation form

Rule name override supports values returned from ES|QL query
As agreed on Adv. correlation WG, we don't introduce similar possibility
for risk score/severity override at this point

<details>
<summary>How it looks like in UI</summary>
<img width="2082" alt="Screenshot 2023-09-21 at 11 52 59"
src="14c94e36-ca90-496d-a7a5-4a31899d25b6">
<img width="2079" alt="Screenshot 2023-09-21 at 11 53 14"
src="9abd53ec-a0f4-4481-8b1f-4ecccdc5feae">
<img width="2072" alt="Screenshot 2023-09-21 at 12 14 17"
src="58e4f9eb-c15f-4849-bba0-bc1b92e8c945">


</details>


### Context

We introduced concept of Aggregating and Non-aggregating rules for
ES|QL. It depends on, whether STATS..BY command used in query

**Aggregating rule** - is a rule that uses
[stats…by](https://esql.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/esql-stats-by.html)
grouping commands. So, its result can not be matched to a particular
document in ES. This can lead to possibly duplicated alerts, since we
are using document `id` to deduplicate alerts. We are going to introduce
suppression for all rule types in future, that would help to mitigate
this case
```
FROM logs*
| STATS count = COUNT(host.name) BY host.name
| SORT host.name
```

**Non-aggregating rule** - is a rule that does not use
[stats…by](https://esql.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/esql-stats-by.html)
grouping commands. Each row in result can be tracked to a source
document in ES. For this type of rule operator \`[metadata _id, _index,
_version]\` is required to be used after defining index source. This
would allow deduplicate alerts and link them with the source document.

```
FROM logs* [metadata _id, _index, _version]
| WHERE event.id == "test"
| LIMIT 10
```

### Serverless Feature Flag

ES|QL won't be available for Serverless as for 8.11 release, so it will
be hidden by Security experimental feature flag `esqlRulesDisabled`. All
UI changes will be hidden (it's mostly Form creation) and rule type
won't be registered, which prevents rule to be created, returned in
search if it exists or execute.

### Test envs
- Serverless qa, [admin link to
project](https://admin.qa.cld.elstc.co/projects/security/ef79684f92d64f27b69e1b04de86eb1a),
disabled there
- internal
[link](https://elastic.slack.com/archives/C03E8TR26HE/p1693848029955229)
to test env for Stateful


### Rule schema changes

introduces value `esql` to `type` property
introduces value `esql` to `language` property

### Tests coverage
- cypress tests (as per 27/09/2023 added cypress tests for rule
creation/edit/details,bulk_edit))
- functional tests for rule execution(exceptions, overrides, preview and
actual rule execution)
  - functional tests for bulk_edit

 #### Flaky test runner
- [cypress esql
tests](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3233#_),
non failed of added


### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-09-30 09:45:34 +01:00
..
data_table Upgrade EUI to v87.1.0 (#163961) 2023-08-21 15:16:39 -07:00
ecs_data_quality_dashboard [Security Solution][Elastic AI Assistant] Adds Model Evaluation Tooling (#167220) 2023-09-29 09:32:24 -06:00
features [Security Solution][Detection Engine] adds ES|QL rule type to Security Detections rules (#165450) 2023-09-30 09:45:34 +01:00
navigation [Security Solution] Fix tab navigation scrolling to top (#166655) 2023-09-19 11:32:30 +02:00
side_nav [Security Solution][Unified IA] New sections with landing page (#163102) 2023-08-15 09:31:57 +02:00
storybook/config [SecuritySolution] Refactor security packages (#155365) 2023-04-26 05:05:36 -07:00
upselling [SecuritySolution] Fix TypeScript errors (#167147) 2023-09-27 08:25:45 +02:00
README.mdx [SecuritySolution] Refactor security packages (#155365) 2023-04-26 05:05:36 -07:00

README.mdx 

## Security Solution package

This package compiles components used by Security Solution

### Lazy by default

All components are exported to be lazily-loaded with a default `React.Suspense` default most appropriate to its nature.

If a page needs to alter the `React.Suspense` behavior, (e.g. a different "loading" component), one can import the `Lazy[ComponentName]` version and surround it with a custom `React.Suspense` component.