Your window into the Elastic Stack
Find a file
Chris Cowan 872d103b7e
[8.x] [Streams 🌊] Add initial telemetry to Streams (#214027) (#215200)
# Backport

This will backport the following commits from `main` to `8.x`:
- [[Streams 🌊] Add initial telemetry to Streams
(#214027)](https://github.com/elastic/kibana/pull/214027)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Chris
Cowan","email":"chris@elastic.co"},"sourceCommit":{"committedDate":"2025-03-17T19:34:52Z","message":"[Streams
🌊] Add initial telemetry to Streams (#214027)\n\n## Summary\n\nThis PR
adds EBT telemetry to the Streams backend and UI. For the\nbackend APIs,
we are only measuring latency on the \"write\" endpoints\nthat have the
highest potential for being slow:\n\n- `POST
/api/streams/{name}/processing/_simulate 2023-10-31`\n- `POST
/api/streams/{name}/processing/_suggestions 2023-10-31`\n- `POST
/api/streams/{name}/_fork 2023-10-31`\n- `PUT
/api/streams/{name}/dashboards/{dashboardId} 2023-10-31`\n- `PUT
/api/streams/{name} 2023-10-31`\n- `PUT /api/streams/{name}/_group
2023-10-31`\n- `PUT /api/streams/{name}/_ingest 2023-10-31`\n- `DELETE
/api/streams/{name} 2023-10-31`\n- `POST /api/streams/_enable
2023-10-31`\n- `POST /api/streams/_disable 2023-10-31`\n- `POST
/api/streams/_resync 2023-10-31`\n\nThis is controlled by an allow list
located in the server's telemetry\nclient.\n\nFor the UI, I've added the
following tracking:\n\n- **Number of Dashboards associated with a
Stream** – This captures the\nstream name and the number of dashboards
associated with it. It's\nattached to the dashboard hook, this means
that we only track dashboard\ncounts on streams that users are actually
using.\n- **Clicks on Dashboards** – This captures the name of the
dashboard,\nthe dashboard id, and the stream name\n- **Latency of AI
Grok Suggestions** – This captures the name of the\nstream, the field
used, the number of suggested patterns, the success\nrates of the
suggest patterns, and the connector id of the LLM. This\nwill also
capture if the AI returned ZERO suggestions.\n- **When a user accepts an
AI Grok Suggestion** – This captures the name\nof the stream, the name
of the field, the success rate of the pattern,\nand number of fields
detected.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"508c89fc15fd611971d28e8f9434b214a945efcf","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:version","Feature:Streams","v9.1.0","v8.19.0"],"title":"[Streams
🌊] Add initial telemetry to
Streams","number":214027,"url":"https://github.com/elastic/kibana/pull/214027","mergeCommit":{"message":"[Streams
🌊] Add initial telemetry to Streams (#214027)\n\n## Summary\n\nThis PR
adds EBT telemetry to the Streams backend and UI. For the\nbackend APIs,
we are only measuring latency on the \"write\" endpoints\nthat have the
highest potential for being slow:\n\n- `POST
/api/streams/{name}/processing/_simulate 2023-10-31`\n- `POST
/api/streams/{name}/processing/_suggestions 2023-10-31`\n- `POST
/api/streams/{name}/_fork 2023-10-31`\n- `PUT
/api/streams/{name}/dashboards/{dashboardId} 2023-10-31`\n- `PUT
/api/streams/{name} 2023-10-31`\n- `PUT /api/streams/{name}/_group
2023-10-31`\n- `PUT /api/streams/{name}/_ingest 2023-10-31`\n- `DELETE
/api/streams/{name} 2023-10-31`\n- `POST /api/streams/_enable
2023-10-31`\n- `POST /api/streams/_disable 2023-10-31`\n- `POST
/api/streams/_resync 2023-10-31`\n\nThis is controlled by an allow list
located in the server's telemetry\nclient.\n\nFor the UI, I've added the
following tracking:\n\n- **Number of Dashboards associated with a
Stream** – This captures the\nstream name and the number of dashboards
associated with it. It's\nattached to the dashboard hook, this means
that we only track dashboard\ncounts on streams that users are actually
using.\n- **Clicks on Dashboards** – This captures the name of the
dashboard,\nthe dashboard id, and the stream name\n- **Latency of AI
Grok Suggestions** – This captures the name of the\nstream, the field
used, the number of suggested patterns, the success\nrates of the
suggest patterns, and the connector id of the LLM. This\nwill also
capture if the AI returned ZERO suggestions.\n- **When a user accepts an
AI Grok Suggestion** – This captures the name\nof the stream, the name
of the field, the success rate of the pattern,\nand number of fields
detected.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"508c89fc15fd611971d28e8f9434b214a945efcf"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/214027","number":214027,"mergeCommit":{"message":"[Streams
🌊] Add initial telemetry to Streams (#214027)\n\n## Summary\n\nThis PR
adds EBT telemetry to the Streams backend and UI. For the\nbackend APIs,
we are only measuring latency on the \"write\" endpoints\nthat have the
highest potential for being slow:\n\n- `POST
/api/streams/{name}/processing/_simulate 2023-10-31`\n- `POST
/api/streams/{name}/processing/_suggestions 2023-10-31`\n- `POST
/api/streams/{name}/_fork 2023-10-31`\n- `PUT
/api/streams/{name}/dashboards/{dashboardId} 2023-10-31`\n- `PUT
/api/streams/{name} 2023-10-31`\n- `PUT /api/streams/{name}/_group
2023-10-31`\n- `PUT /api/streams/{name}/_ingest 2023-10-31`\n- `DELETE
/api/streams/{name} 2023-10-31`\n- `POST /api/streams/_enable
2023-10-31`\n- `POST /api/streams/_disable 2023-10-31`\n- `POST
/api/streams/_resync 2023-10-31`\n\nThis is controlled by an allow list
located in the server's telemetry\nclient.\n\nFor the UI, I've added the
following tracking:\n\n- **Number of Dashboards associated with a
Stream** – This captures the\nstream name and the number of dashboards
associated with it. It's\nattached to the dashboard hook, this means
that we only track dashboard\ncounts on streams that users are actually
using.\n- **Clicks on Dashboards** – This captures the name of the
dashboard,\nthe dashboard id, and the stream name\n- **Latency of AI
Grok Suggestions** – This captures the name of the\nstream, the field
used, the number of suggested patterns, the success\nrates of the
suggest patterns, and the connector id of the LLM. This\nwill also
capture if the AI returned ZERO suggestions.\n- **When a user accepts an
AI Grok Suggestion** – This captures the name\nof the stream, the name
of the field, the success rate of the pattern,\nand number of fields
detected.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"508c89fc15fd611971d28e8f9434b214a945efcf"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
2025-03-19 20:00:40 +01:00
.buildkite [8.x] [ES|QL] Update grammars (#213006) (#215058) 2025-03-18 23:45:41 +02:00
.devcontainer [8.x] Upgrade to Storybook 8 (#195148) (#214681) 2025-03-18 23:20:24 +02:00
.github [8.x] [Streams] Replay loghub data with synthtrace (#212120) (#214916) 2025-03-19 10:57:02 +01:00
api_docs [8.x] [Streams 🌊] Introduce GroupStreams (#208126) (#209871) 2025-02-06 14:09:59 +01:00
config [8.x] [Inference Connector] Enable inference connector for ESS by default, disable it for Serverless (#209197) (#209865) 2025-02-06 17:17:01 +01:00
dev_docs [8.x] [Dev Docs] Add VS Code configurations to Dev Docs Debugging Tutorial (#212807) (#213413) 2025-03-06 18:09:18 +01:00
docs [DOCS][8.18] New sub-feature priv for case assignees (#215162) 2025-03-19 09:48:36 -04:00
examples [8.x] [Discover] Replace DiscoverInternalStateContainer with Redux based InternalStateStore (#208784) (#213630) 2025-03-14 16:16:37 +00:00
kbn_pm [8.x] Sustainable Kibana Architecture: Move CodeEditor related packages #205587 (#205738) (#205919) 2025-01-10 11:20:26 +00:00
legacy_rfcs [8.x] SKA: Relocate "platform" packages that remain on /packages (#208704) (#212474) 2025-02-28 10:12:01 +00:00
licenses Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
oas_docs [8.x] [DOCS] Logstash configuration management APIs (#214497) (#214896) 2025-03-19 17:55:11 +02:00
packages [8.x] SKA: Relocate /test to /src/platform/test (#210956) (#214791) 2025-03-18 14:48:04 +01:00
plugins
scripts [8.x] SKA: Relocate /test to /src/platform/test (#210956) (#214791) 2025-03-18 14:48:04 +01:00
src [8.x] [Security Solution] Cleanup alerts table rendering and reference issues (#213649) (#215205) 2025-03-19 20:02:14 +02:00
typings [8.x] make emotion typing global (#200958) (#203162) 2024-12-05 14:10:16 -06:00
x-pack [8.x] [Streams 🌊] Add initial telemetry to Streams (#214027) (#215200) 2025-03-19 20:00:40 +01:00
.backportrc.json chore(NA): adds 8.16 into backportrc (#187530) 2024-07-04 19:09:25 +01:00
.bazelignore Remove references to deleted .ci folder (#177168) 2024-02-20 19:54:21 +01:00
.bazeliskversion chore(NA): upgrade bazelisk into v1.11.0 (#125070) 2022-02-09 20:43:57 +00:00
.bazelrc chore(NA): use new and more performant BuildBuddy servers (#130350) 2022-04-18 02:01:38 +01:00
.bazelrc.common Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
.bazelversion chore(NA): revert bazel upgrade for v5.2.0 (#135096) 2022-06-24 03:57:21 +01:00
.browserslistrc Add Firefox ESR to browserlistrc (#184462) 2024-05-29 17:53:18 -05:00
.editorconfig
.eslintignore [8.x] SKA: Relocate "platform" packages that remain on /packages (#208704) (#212474) 2025-02-28 10:12:01 +00:00
.eslintrc.js [8.x] [scout] use project deps as global hooks for parallel tests (#211409) (#215066) 2025-03-19 12:17:51 +01:00
.gitattributes
.gitignore [8.x] SKA: Relocate /test to /src/platform/test (#210956) (#214791) 2025-03-18 14:48:04 +01:00
.i18nrc.json [8.x] SKA: Fix kebab-case issues in security-threat-hunting packages (#211349) (#211732) 2025-02-19 13:45:13 +01:00
.node-version [8.x] Upgrade Node.js to 20.18.2 (#207431) (#207894) 2025-01-22 19:37:48 +00:00
.npmrc [npmrc] Fix puppeteer_skip_download configuration (#177673) 2024-02-22 18:59:01 -07:00
.nvmrc [8.x] Upgrade Node.js to 20.18.2 (#207431) (#207894) 2025-01-22 19:37:48 +00:00
.prettierignore
.prettierrc
.puppeteerrc Add .puppeteerrc (#179847) 2024-04-03 09:14:39 -05:00
.stylelintignore
.stylelintrc Bump stylelint to ^14 (#136693) 2022-07-20 10:11:00 -05:00
.telemetryrc.json [8.x] Sustainable Kibana Architecture: Move modules owned by @elastic/kibana-core (#201653) (#205563) 2025-01-05 16:32:00 +01:00
.yarnrc
BUILD.bazel Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
catalog-info.yaml [sonarqube] Disable cron (#190611) 2024-08-15 09:19:09 -05:00
CODE_OF_CONDUCT.md
CONTRIBUTING.md
FAQ.md Fix small typos in the root md files (#134609) 2022-06-23 09:36:11 -05:00
fleet_packages.json [8.x] Sync bundled packages with Package Storage (#214912) 2025-03-18 10:47:33 +00:00
github_checks_reporter.json
kibana.d.ts Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
LICENSE.txt Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
NOTICE.txt [8.x] [ES|QL] capitalize &#x60;FROM&#x60; in recommended queries (#205122) (#205352) 2025-01-02 04:27:03 -06:00
package.json [8.x] [Streams] Replay loghub data with synthtrace (#212120) (#214916) 2025-03-19 10:57:02 +01:00
preinstall_check.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
README.md
renovate.json [8.x] Upgrade to Storybook 8 (#195148) (#214681) 2025-03-18 23:20:24 +02:00
RISK_MATRIX.mdx
run_fleet_setup_parallel.sh [8.x] Sustainable Kibana Architecture: Move modules owned by @elastic/fleet (#202422) (#205145) 2024-12-24 15:17:23 -06:00
SECURITY.md
sonar-project.properties [sonarqube] update memory, cpu (#190547) 2024-09-09 16:16:30 -05:00
STYLEGUIDE.mdx [styleguide] update path to scss theme (#140742) 2022-09-15 10:41:14 -04:00
tsconfig.base.json [8.x] [Streams] Replay loghub data with synthtrace (#212120) (#214916) 2025-03-19 10:57:02 +01:00
tsconfig.browser.json
tsconfig.browser_bazel.json
tsconfig.json Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
TYPESCRIPT.md Fix small typos in the root md files (#134609) 2022-06-23 09:36:11 -05:00
versions.json [ci] Update version tracking for 7.17.25 (#192477) 2024-09-10 20:54:04 -05:00
WORKSPACE.bazel [8.x] Upgrade Node.js to 20.18.2 (#207431) (#207894) 2025-01-22 19:37:48 +00:00
yarn.lock [8.x] [Streams] Replay loghub data with synthtrace (#212120) (#214916) 2025-03-19 10:57:02 +01:00

Kibana

Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.

Getting Started

If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.

If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.

Using a Kibana Release

If you want to use a Kibana release in production, give it a test run, or just play around:

Building and Running Kibana, and/or Contributing Code

You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:

Documentation

Visit Elastic.co for the full Kibana documentation.

For information about building the documentation, see the README in elastic/docs.

Version Compatibility with Elasticsearch

Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.

Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.

Situation Example Kibana version Example ES version Outcome
Versions are the same. 7.15.1 7.15.1 💚 OK
ES patch number is newer. 7.15.0 7.15.1 ⚠️ Logged warning
ES minor number is newer. 7.14.2 7.15.0 ⚠️ Logged warning
ES major number is newer. 7.15.1 8.0.0 🚫 Fatal error
ES patch number is older. 7.15.1 7.15.0 ⚠️ Logged warning
ES minor number is older. 7.15.1 7.14.2 🚫 Fatal error
ES major number is older. 8.0.0 7.15.1 🚫 Fatal error

Questions? Problems? Suggestions?

  • If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
  • Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.