mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 17:59:23 -04:00
Your window into the Elastic Stack
ae068a62f1
## Summary This PR adds a new Risk Scoring API endpoint. Its functionality is meant to replace the current transform-based solution. ### Contents of this PR: - New feature flag: `riskScoringRoutesEnabled` - A new POST endpoint at `/internal/risk_scores/preview` - An OpenAPI doc for the endpoint - Unit and integration tests ### Current behavior, and short-term plans The endpoint as specified in this branch is _read-only_. When the endpoint is hit, it triggers some aggregations in elasticsearch, and a formatted response is returned; there is no persistence at this time. This endpoint was originally written as a POC to demonstrate the new Risk Engine's functionality, but it will now drive the [Preview Risk Scoring](https://github.com/elastic/security-team/issues/6443) feature. The main path for the Risk Engine is going to be a _scheduled task_ that calculates Risk Scores and writes them to a persistent datastream that we own. (https://github.com/elastic/security-team/issues/6450). To accomplish this, we will decompose the full functionality of this endpoint into constituent pieces (i.e. `calculate | persist, get`) ## How to review I've created a Postman collection that can be used to exercise this endpoint. It was generated by Postman from the OpenAPI spec, and modified by me to contain a valid subset of request parameters; please peruse the spec and/or feel free to generate your own scripts/tools from the spec. ``` curl -L -H 'Authorization: 10c7f646373aa116' -o 'Risk Scoring API.postman_collection.json' https://upload.elastic.co/d/007a57857fc40c791835629ea6dd692d2a8a290860f2917329d688be78c03b1d ``` ### Review against the PR instance I've created a [demo instance](https://rylnd-pr-155966-risk-score-api.kbndev.co/) containing the code on this branch, along with some realistic(ish) alert data (~200k alerts). While you can use this instance as a convenience, you will need to [set up kibana-remote-dev](https://github.com/elastic/kibana-remote-dev#access-kibana-es-locally-without-sso) and forward ports in order to be able to access the instance's API from a local machine: 1. Configure kibana-remote-dev with your SSH key and GitHub token. 2. Configure kibana-remote-dev to specify `GITHUB_USERNAME=rylnd` * This allows you to bypass kibana-remote-dev code that assumes projects are owned by you 3. Forward local ports to my instance: `./ports rd-rylnd-pr-155966-risk-score-api` 4. Use postman to talk to `http://localhost:5601`, which will be forwarded to the cloud instance via the previous command ### Review manually 1. Check out this branch 3. Enable the feature flag 4. Populate some event data and generate some alerts 5. Navigate to the new endpoint, and observe that the `host.name`s and `user.name`s from those alerts have been aggregated into these "risk scores" in the response 6. Play with the request options to see how these affect the scores (and see docs/test for more details on how those work) ## _What_ to review * Are the scores internally consistent? I.e. do they add up as expected? Does the corresponding "level" make sense? * Do parameters apply as expected? E.g. do weights predictably scale the results? * Are there discrepancies between the spec and the actual implementation? * Does pagination make sense? (i.e. the `after_keys` stuff)? #### TODO (for @rylnd) - [x] Add `description`s to the OpenAPI docs - [x] Remove remaining TODOs from code ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) Related ticket: https://github.com/elastic/security-team/issues/4211 --------- Co-authored-by: Khristinin Nikita <nikita.khristinin@elastic.co> |
||
|---|---|---|
| .buildkite | ||
| .ci | ||
| .github | ||
| api_docs | ||
| config | ||
| dev_docs | ||
| docs | ||
| examples | ||
| kbn_pm | ||
| legacy_rfcs | ||
| licenses | ||
| packages | ||
| plugins | ||
| scripts | ||
| src | ||
| test | ||
| typings | ||
| vars | ||
| x-pack | ||
| .backportrc.json | ||
| .bazelignore | ||
| .bazeliskversion | ||
| .bazelrc | ||
| .bazelrc.common | ||
| .bazelversion | ||
| .browserslistrc | ||
| .editorconfig | ||
| .eslintignore | ||
| .eslintrc.js | ||
| .gitattributes | ||
| .gitignore | ||
| .i18nrc.json | ||
| .node-version | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| .stylelintignore | ||
| .stylelintrc | ||
| .telemetryrc.json | ||
| .yarnrc | ||
| BUILD.bazel | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| FAQ.md | ||
| fleet_packages.json | ||
| github_checks_reporter.json | ||
| Jenkinsfile | ||
| kibana.d.ts | ||
| LICENSE.txt | ||
| nav-kibana-dev.docnav.json | ||
| NOTICE.txt | ||
| package.json | ||
| preinstall_check.js | ||
| README.md | ||
| renovate.json | ||
| RISK_MATRIX.mdx | ||
| SECURITY.md | ||
| STYLEGUIDE.mdx | ||
| tsconfig.base.json | ||
| tsconfig.browser.json | ||
| tsconfig.browser_bazel.json | ||
| tsconfig.json | ||
| TYPESCRIPT.md | ||
| versions.json | ||
| WORKSPACE.bazel | ||
| yarn.lock | ||
Kibana
Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.
- Getting Started
- Documentation
- Version Compatibility with Elasticsearch
- Questions? Problems? Suggestions?
Getting Started
If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.
If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.
Using a Kibana Release
If you want to use a Kibana release in production, give it a test run, or just play around:
- Download the latest version on the Kibana Download Page.
- Learn more about Kibana's features and capabilities on the Kibana Product Page.
- We also offer a hosted version of Kibana on our Cloud Service.
Building and Running Kibana, and/or Contributing Code
You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:
- CONTRIBUTING.md will help you get Kibana up and running.
- If you would like to contribute code, please follow our STYLEGUIDE.mdx.
- For all other questions, check out the FAQ.md and wiki.
Documentation
Visit Elastic.co for the full Kibana documentation.
For information about building the documentation, see the README in elastic/docs.
Version Compatibility with Elasticsearch
Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.
Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.
| Situation | Example Kibana version | Example ES version | Outcome |
|---|---|---|---|
| Versions are the same. | 7.15.1 | 7.15.1 | 💚 OK |
| ES patch number is newer. | 7.15.0 | 7.15.1 | ⚠️ Logged warning |
| ES minor number is newer. | 7.14.2 | 7.15.0 | ⚠️ Logged warning |
| ES major number is newer. | 7.15.1 | 8.0.0 | 🚫 Fatal error |
| ES patch number is older. | 7.15.1 | 7.15.0 | ⚠️ Logged warning |
| ES minor number is older. | 7.15.1 | 7.14.2 | 🚫 Fatal error |
| ES major number is older. | 8.0.0 | 7.15.1 | 🚫 Fatal error |
Questions? Problems? Suggestions?
- If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
- Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.