mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 09:48:58 -04:00
Your window into the Elastic Stack
7b31ca96dd
## Summary Ticket: https://github.com/elastic/kibana/issues/161201 An initial pass at adding usageCollection telemetry for cloud_defend (D4C), as well as some click tracking around the SessionView plugin. The cloud_defend telemetry schema mirrors that of CSP (see: https://docs.elastic.dev/security-solution/cloud-security-posture/telemetry/technical-index) but with metrics relevant to the cloud-defend service instead of kspm cspm etc... (e.g findings) The cloud_defend daily telemetry schema can be seen here: x-pack/plugins/cloud_defend/server/lib/telemetry/collectors/schema.ts The biggest difference is that instead of counts around findings/vuln, it is showing file/process/alert document counts, as well as sending up yaml and json versions of the cloud-defend policy schema. The json policy fields are all typed and can be used to run any aggregate query needed to dig into a a customer's policy usage. e.g which selector conditions they use, and if they are blocking any operations like 'fork', 'exec', 'createFile', 'deleteFile', etc... Documentation on how cloud-defend policies work can be found here: https://github.com/elastic/integrations/tree/main/packages/cloud_defend#policy-example TODO: - cloud-defend binary needs to start populating kubernetes_version. I imagine we could make use of https://www.elastic.co/guide/en/ecs/8.5/ecs-orchestrator.html#field-orchestrator-cluster-version for this? cc @norrietaylor The following click tracking events have been added to session_view: ``` export type SessionViewTelemetryKey = | 'loaded_from_cloud_defend_log' | 'loaded_from_cloud_defend_alert' | 'loaded_from_endpoint_log' | 'loaded_from_endpoint_alert' | 'loaded_from_unknown_log' | 'loaded_from_unknown_alert' | 'refresh_clicked' | 'process_selected' | 'collapse_tree' | 'children_opened' | 'children_closed' | 'alerts_opened' | 'alerts_closed' | 'details_opened' | 'details_closed' | 'output_clicked' | 'alert_details_loaded' | 'disabled_tty_clicked' // tty button clicked when disabled (no data or not enabled) | 'tty_loaded' // tty player succesfully loaded | 'tty_playback_started' | 'tty_playback_stopped' | 'verbose_mode_enabled' | 'verbose_mode_disabled' | 'timestamp_enabled' | 'timestamp_disabled' | 'search_performed' | 'search_next' | 'search_previous'; ``` Sample output for cloud_defend daily telemetry: ``` "cloud_defend": { "indices": { "alerts": { "doc_count": 116, "deleted": 0, "size_in_bytes": 203482, "last_doc_timestamp": "2023-07-15T02:11:16.478Z" }, "file": { "doc_count": 44, "deleted": 0, "size_in_bytes": 168313, "last_doc_timestamp": "2023-07-15T02:11:16.478Z" }, "process": { "doc_count": 85353, "deleted": 0, "size_in_bytes": 54157433, "last_doc_timestamp": "2023-07-15T02:15:47.214Z" }, "latestPackageVersion": "1.0.7", "packageStatus": { "status": "indexed", "installedPackagePolicies": 1, "healthyAgents": 0 } }, "accounts_stats": [ { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "total_doc_count": 85513, "file_doc_count": 160, "process_doc_count": 85353, "alert_doc_count": 116, "kubernetes_version": null, "cloud_provider": "gcp", "agents_count": 3, "nodes_count": 3, "pods_count": 7 } ], "pods_stats": [ { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "pod_name": "pdcsi-node-shrsp", "container_image_name": "gke.gcr.io/csi-node-driver-registrar", "container_image_tag": "v2.8.0-gke.1", "total_doc_count": 19152, "file_doc_count": 0, "process_doc_count": 19152, "alert_doc_count": 0 }, { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "pod_name": "pdcsi-node-6w5nw", "container_image_name": "gke.gcr.io/csi-node-driver-registrar", "container_image_tag": "v2.8.0-gke.1", "total_doc_count": 19149, "file_doc_count": 0, "process_doc_count": 19149, "alert_doc_count": 0 }, { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "pod_name": "pdcsi-node-ltg8s", "container_image_name": "gke.gcr.io/csi-node-driver-registrar", "container_image_tag": "v2.8.0-gke.1", "total_doc_count": 19148, "file_doc_count": 0, "process_doc_count": 19148, "alert_doc_count": 0 }, { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "pod_name": "kube-proxy-gke-kg-dev-default-pool-9347b91e-rqb0", "container_image_name": "gke.gcr.io/kube-proxy-amd64", "container_image_tag": "v1.26.5-gke.1200", "total_doc_count": 9141, "file_doc_count": 0, "process_doc_count": 9141, "alert_doc_count": 0 }, { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "pod_name": "kube-proxy-gke-kg-dev-default-pool-9347b91e-lflp", "container_image_name": "gke.gcr.io/kube-proxy-amd64", "container_image_tag": "v1.26.5-gke.1200", "total_doc_count": 9139, "file_doc_count": 0, "process_doc_count": 9139, "alert_doc_count": 0 }, { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "pod_name": "kube-proxy-gke-kg-dev-default-pool-9347b91e-t9jd", "container_image_name": "gke.gcr.io/kube-proxy-amd64", "container_image_tag": "v1.26.5-gke.1200", "total_doc_count": 9139, "file_doc_count": 0, "process_doc_count": 9139, "alert_doc_count": 0 }, { "account_id": "a9f309fb-d427-42c8-90de-48653f7ea6d7", "pod_name": "elastic-agent-667qf", "container_image_name": "docker.elastic.co/elastic-agent/elastic-agent", "container_image_tag": "8.8.0", "total_doc_count": 645, "file_doc_count": 160, "process_doc_count": 485, "alert_doc_count": 116 } ], "installation_stats": [ { "package_policy_id": "7814c387-58a4-4e5c-8475-38e86f584971", "package_version": "1.0.7", "created_at": "2023-07-12T19:23:19.432Z", "agent_policy_id": "6bece4a0-20e9-11ee-8d36-0d4244506490", "agent_count": 0, "policy_yaml": """process: selectors: - name: allProcesses operation: [fork, exec] responses: - match: [allProcesses] actions: [log] file: selectors: - name: executableChanges operation: [createExecutable, modifyExecutable] responses: - match: [executableChanges] actions: [alert] """, "selectors": [ { "name": "allProcesses", "operation": [ "fork", "exec" ], "type": "process" }, { "name": "executableChanges", "operation": [ "createExecutable", "modifyExecutable" ], "type": "file" } ], "responses": [ { "match": [ "allProcesses" ], "actions": [ "log" ], "type": "process" }, { "match": [ "executableChanges" ], "actions": [ "alert" ], "type": "file" } ] } ] }, ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> |
||
|---|---|---|
| .buildkite | ||
| .ci | ||
| .github | ||
| api_docs | ||
| config | ||
| dev_docs | ||
| docs | ||
| examples | ||
| kbn_pm | ||
| legacy_rfcs | ||
| licenses | ||
| packages | ||
| plugins | ||
| scripts | ||
| src | ||
| test | ||
| typings | ||
| vars | ||
| x-pack | ||
| .backportrc.json | ||
| .bazelignore | ||
| .bazeliskversion | ||
| .bazelrc | ||
| .bazelrc.common | ||
| .bazelversion | ||
| .browserslistrc | ||
| .editorconfig | ||
| .eslintignore | ||
| .eslintrc.js | ||
| .gitattributes | ||
| .gitignore | ||
| .i18nrc.json | ||
| .node-version | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| .stylelintignore | ||
| .stylelintrc | ||
| .telemetryrc.json | ||
| .yarnrc | ||
| BUILD.bazel | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| FAQ.md | ||
| fleet_packages.json | ||
| github_checks_reporter.json | ||
| Jenkinsfile | ||
| kibana.d.ts | ||
| LICENSE.txt | ||
| nav-kibana-dev.docnav.json | ||
| NOTICE.txt | ||
| package.json | ||
| preinstall_check.js | ||
| README.md | ||
| renovate.json | ||
| RISK_MATRIX.mdx | ||
| SECURITY.md | ||
| STYLEGUIDE.mdx | ||
| tsconfig.base.json | ||
| tsconfig.browser.json | ||
| tsconfig.browser_bazel.json | ||
| tsconfig.json | ||
| TYPESCRIPT.md | ||
| versions.json | ||
| WORKSPACE.bazel | ||
| yarn.lock | ||
Kibana
Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.
- Getting Started
- Documentation
- Version Compatibility with Elasticsearch
- Questions? Problems? Suggestions?
Getting Started
If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.
If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.
Using a Kibana Release
If you want to use a Kibana release in production, give it a test run, or just play around:
- Download the latest version on the Kibana Download Page.
- Learn more about Kibana's features and capabilities on the Kibana Product Page.
- We also offer a hosted version of Kibana on our Cloud Service.
Building and Running Kibana, and/or Contributing Code
You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:
- CONTRIBUTING.md will help you get Kibana up and running.
- If you would like to contribute code, please follow our STYLEGUIDE.mdx.
- For all other questions, check out the FAQ.md and wiki.
Documentation
Visit Elastic.co for the full Kibana documentation.
For information about building the documentation, see the README in elastic/docs.
Version Compatibility with Elasticsearch
Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.
Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.
| Situation | Example Kibana version | Example ES version | Outcome |
|---|---|---|---|
| Versions are the same. | 7.15.1 | 7.15.1 | 💚 OK |
| ES patch number is newer. | 7.15.0 | 7.15.1 | ⚠️ Logged warning |
| ES minor number is newer. | 7.14.2 | 7.15.0 | ⚠️ Logged warning |
| ES major number is newer. | 7.15.1 | 8.0.0 | 🚫 Fatal error |
| ES patch number is older. | 7.15.1 | 7.15.0 | ⚠️ Logged warning |
| ES minor number is older. | 7.15.1 | 7.14.2 | 🚫 Fatal error |
| ES major number is older. | 8.0.0 | 7.15.1 | 🚫 Fatal error |
Questions? Problems? Suggestions?
- If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
- Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.