Your window into the Elastic Stack
Find a file
Yngrid Coello 64df229998
[Dataset quality] degradedDocs rule (#216026)
Closes https://github.com/elastic/kibana/issues/179173.

## Rule type
A dedicated stack rule type was created `datasetQuality.degradedDocs`.
<img width="1759" alt="image"
src="https://github.com/user-attachments/assets/5004a08d-6f12-4f5e-b27f-5f4db242dcf0"
/>
<img width="2318" alt="image"
src="https://github.com/user-attachments/assets/f8b2664a-f1c6-48c5-a617-c6f1b79bf0f7"
/>


This new rule is aggregated by default using `_index` and could be
further configured by the user (e.g. user can also aggregate by
`cloud.provider`).

A new rule type was needed to be created since there is no actual way to
aggregate all documents in a dataStream if we use a DataView like
`logs-*-*`. Inside datasStream documents there is no indication about
the dataStream where they belong to, instead we just have `_index` which
contains backingIndexName instead of actual index.

It's important to note, that this rule type is also visible from
`Observability > Alerts`, which is useful specially for serverless.


https://github.com/user-attachments/assets/000aee51-4895-4f4c-9484-924ace4325c5


## Role Based Access-Control (RBAC)
RBAC for dataset quality alerts is defined within dataQuality kibana
feature.
We have three privileges defined:
1. `all`: This privilege now contains a subFeature `manage_rules` that
will allow for more granularity on alerting level. It's by default
assigned to `all` but can be disabled.
2. `read`: This privilege is only related to serverless (when we don't
have yet custom roles).

https://github.com/user-attachments/assets/70ed5bde-bf45-4024-b448-228799fcaf71
3. `none`: This privilege is only relevant for stateful (in serverless
we don't have custom roles).

## 🎥 Demo
### Serverless
#### `all` privileges

https://github.com/user-attachments/assets/8dad6e30-a261-4a69-979f-6dfc2a41c888

#### `read` privileges

https://github.com/user-attachments/assets/e1cb108d-22a0-4e7f-b252-9cc12d1e9d65

### Stateful
#### `all` privileges

https://github.com/user-attachments/assets/d96f3b70-35b2-466b-aa59-a07190d24d93

#### `all` privileges with subFeature disabled

https://github.com/user-attachments/assets/808ab811-9320-43e4-b2a6-06d530a78b82

#### `none` privileges (Stateful)

https://github.com/user-attachments/assets/18f2a2d6-d825-4713-acea-0d72f451e9ab

## How to test?
1. run synthrace scenario `degraded_logs` in live mode
```
node scripts/synthtrace degraded_logs --live
```
2. Open dataset quality page (/app/management/data/data_quality)
3. Select `synth.3` dataset
(/app/management/data/data_quality/details?pageState=(dataStream:logs-synth.3-default)
4. Click on `Actions` and then select `Create rule`
5. Fill out the alert form
6. Go to `Observability > Alerts` or `Stack management > Alerts`
(/app/observability/alerts)

## Release note
Adds the Create alert rule action to dataset quality page and dataset
quality details. This allows you to generate an alert when the
percentage of degraded docs on the chart crosses a certain threshold.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@elastic.co>
Co-authored-by: Faisal Kanout <faisal.kanout@elastic.co>
2025-06-23 10:30:14 +01:00
.buildkite [Detections Response] Remove unused test (#224108) 2025-06-20 12:39:25 -07:00
.devcontainer Upgrade to Storybook 8 (#195148) 2025-03-14 15:41:03 -07:00
.github docs-builder: add pull-requests: write permission to docs-build workflow (#224714) 2025-06-23 08:30:24 +00:00
api_docs [api-docs] 2025-06-23 Daily api_docs build (#224792) 2025-06-23 07:04:45 +01:00
config [Obs AI Assistant] Disable in Logs Essentials mode (#223236) 2025-06-20 12:47:25 +02:00
dev_docs Update anatomy_of_a_plugin.mdx (#222857) 2025-06-23 10:46:00 +02:00
docs APEX-72 Enforce Kibana >= 8.18.0; Remove switchToModelVersionAt (#220985) 2025-06-20 16:41:54 +02:00
examples [Search] Remove website search guided onboarding (#224538) 2025-06-19 19:34:32 +02:00
kbn_pm add es-cache to cleaned folders (#223373) 2025-06-12 10:58:05 +02:00
legacy_rfcs Optimize existing image assets with lossless compression (#223998) 2025-06-19 16:44:13 +02:00
licenses Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
oas_docs Security Entity Analytics - Privileged user monitoring - Privileged access detection support (#224008) 2025-06-22 00:36:09 -05:00
packages Create @kbn/css-utils package (#223933) 2025-06-20 22:48:44 +02:00
plugins
scripts [Inference] Instrument inference with OpenTelemetry (#218694) 2025-05-07 11:44:29 +02:00
src [Dataset quality] degradedDocs rule (#216026) 2025-06-23 10:30:14 +01:00
typings chore: remove react-syntax-highlighter leftovers (#213076) 2025-03-04 15:35:34 +01:00
x-pack [Dataset quality] degradedDocs rule (#216026) 2025-06-23 10:30:14 +01:00
.backportrc.json chore(NA): prepare replacement of 8.x branch with 8.19 (#218514) 2025-04-17 04:02:40 +01:00
.bazelignore Remove references to deleted .ci folder (#177168) 2024-02-20 19:54:21 +01:00
.bazeliskversion
.bazelrc
.bazelrc.common Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
.bazelversion chore(NA): revert bazel upgrade for v5.2.0 (#135096) 2022-06-24 03:57:21 +01:00
.browserslistrc Add Firefox ESR to browserlistrc (#184462) 2024-05-29 17:53:18 -05:00
.editorconfig
.eslintignore chore(fullstory): serve the snippet as an asset (#220368) 2025-05-07 23:06:24 +02:00
.eslintrc.js [ska] relocate guided_onboarding tests that import search solution src code (#224390) 2025-06-18 16:01:56 +02:00
.gitattributes
.gitignore Upgrade Node.js to 22.16.0 (#205983) 2025-06-18 08:55:18 -05:00
.i18nrc.json [Discover] Support Lens fetches across tabs (#218506) 2025-05-02 13:39:25 -03:00
.node-version Upgrade Node.js to 22.16.0 (#205983) 2025-06-18 08:55:18 -05:00
.npmrc [npmrc] Fix puppeteer_skip_download configuration (#177673) 2024-02-22 18:59:01 -07:00
.nvmrc Upgrade Node.js to 22.16.0 (#205983) 2025-06-18 08:55:18 -05:00
.prettierignore
.prettierrc
.puppeteerrc Add .puppeteerrc (#179847) 2024-04-03 09:14:39 -05:00
.stylelintignore
.stylelintrc Bump stylelint to ^14 (#136693) 2022-07-20 10:11:00 -05:00
.telemetryrc.json Sustainable Kibana Architecture: Move modules owned by @elastic/kibana-core (#201653) 2025-01-04 11:47:24 -07:00
.yarnrc
BUILD.bazel Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
catalog-info.yaml Configures PagerDuty Backstage Integration for kbn (#208440) 2025-01-27 23:29:11 +00:00
CODE_OF_CONDUCT.md
CONTRIBUTING.md Docs: fix broken links in CONTRIBUTING.md (#219158) 2025-04-24 17:25:00 -06:00
FAQ.md Fix small typos in the root md files (#134609) 2022-06-23 09:36:11 -05:00
fleet_packages.json [main] Sync bundled packages with Package Storage (#224452) 2025-06-20 10:40:03 +02:00
github_checks_reporter.json
kibana.d.ts Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
LICENSE.txt Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
NOTICE.txt [api-docs] 2025-01-01 Daily api_docs build (#205342) 2025-01-01 01:37:13 -06:00
package.json Update dependency form-data to ^4.0.3 (main) (#223590) 2025-06-20 17:03:19 -05:00
preinstall_check.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
README.md
renovate.json Renovate - move immer redux toolkit owner, move csstype to react dependency owner (#223852) 2025-06-19 12:42:24 +02:00
RISK_MATRIX.mdx
run_fleet_setup_parallel.sh Sustainable Kibana Architecture: Move modules owned by @elastic/fleet (#202422) 2024-12-24 15:32:43 +01:00
SECURITY.md
sonar-project.properties [sonarqube] update memory, cpu (#190547) 2024-09-09 16:16:30 -05:00
STYLEGUIDE.mdx [styleguide] update path to scss theme (#140742) 2022-09-15 10:41:14 -04:00
tsconfig.base.json Create @kbn/css-utils package (#223933) 2025-06-20 22:48:44 +02:00
tsconfig.browser.json
tsconfig.browser_bazel.json
tsconfig.json Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
TYPESCRIPT.md Fix small typos in the root md files (#134609) 2022-06-23 09:36:11 -05:00
updatecli-compose.yaml deps(updatecli): bump all policies (#195865) 2024-10-15 07:37:12 -05:00
versions.json chore(NA): update versions after v8.17.8 bump (#222415) 2025-06-03 21:39:35 +01:00
WORKSPACE.bazel Upgrade Node.js to 22.16.0 (#205983) 2025-06-18 08:55:18 -05:00
yarn.lock Update dependency form-data to ^4.0.3 (main) (#223590) 2025-06-20 17:03:19 -05:00

Kibana

Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.

Getting Started

If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.

If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.

Using a Kibana Release

If you want to use a Kibana release in production, give it a test run, or just play around:

Building and Running Kibana, and/or Contributing Code

You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:

Documentation

Visit Elastic.co for the full Kibana documentation.

For information about building the documentation, see the README in elastic/docs.

Version Compatibility with Elasticsearch

Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.

Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.

Situation Example Kibana version Example ES version Outcome
Versions are the same. 7.15.1 7.15.1 💚 OK
ES patch number is newer. 7.15.0 7.15.1 ⚠️ Logged warning
ES minor number is newer. 7.14.2 7.15.0 ⚠️ Logged warning
ES major number is newer. 7.15.1 8.0.0 🚫 Fatal error
ES patch number is older. 7.15.1 7.15.0 ⚠️ Logged warning
ES minor number is older. 7.15.1 7.14.2 🚫 Fatal error
ES major number is older. 8.0.0 7.15.1 🚫 Fatal error

Questions? Problems? Suggestions?

  • If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
  • Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.