Closes https://github.com/elastic/logs-dev/issues/183, https://github.com/elastic/logs-dev/issues/184 and https://github.com/elastic/logs-dev/issues/185. ## Summary This PR aims to support failure store in dataset quality page. The following acceptance criteria items were resolved ### Dataset quality page - [x] A column for Failed docs is included in the table - [x] A tooltip is placed in the title of the column - [x] A % of documents inside Failure store is calculated for every dataStream - [x] If % is lesser than 0.0001 but greater than 0 we should show ⚠ symbol next to the ~0 value (as we do with degraded docs) - [x] Failed docs percentages greater than 0 should link to discover 🎥 Demo https://github.com/user-attachments/assets/6d9e3f4c-02d9-43ab-88cb-ae70716b05d9 ### Dataset details page - [x] A metric, Failed docs, is included in the Overview panel under Data set quality. This metric includes the number of documents inside the failure store for the specific dataStream. - [x] A tooltip is placed in the title of the Failed docs metric with message: `The percentage of docs sent to failure store due to an issue during ingestion.` - [x] Degraded docs graph section is transformed to Document trends allowing the users to switch between Degraded docs and Failed docs trends over time. - [x] A new chart for failed documents is created with links to discover/Logs explorer using the right dataView 🎥 Demo https://github.com/user-attachments/assets/6a3a1f09-2668-4e83-938e-ecdda798c199 ### Failed docs ingestion issue flyout - [x] Whenever documents are found in failure store we should list Document indexing failed in Quality issues table - [x] User should be able to expand Document indexing failed and see more information in the flyout - [x] The flyout will show Docs count, an aggregation of the number of documents inside failure store for the selected timeframe - [x] The flyout will show Last ocurrence, the datetime registered for the most recent document in the failure store. - [x] The flyout will contain a section called Error messages where a list of unique error messages should be shown, exposing Content (error message) and Type (Error Type). - [x] Type should contain a tooltip where message (`Error message category`) explain users how we are categorising the errors. - [x] Other issues inside Quality issues table will be appended by field ignored and the field will be shown in bold. https://github.com/user-attachments/assets/94dc81f0-9720-4596-b256-c9d289cefd94 Note: This PR was reconstructed from https://github.com/elastic/kibana/pull/199806 which it supersedes. ## How to test 1. Execute `failed_logs` synthtrace scenario 2. Open dataset quality page ## Follow ups - Enable in serverless - Deployment agnostic tests cannot be added until we enable this in serverless - FTR tests will be added as part of https://github.com/elastic/logs-dev/issues/182 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> |
||
---|---|---|
.. | ||
core | ||
kbn-ambient-common-types | ||
kbn-ambient-ftr-types | ||
kbn-ambient-storybook-types | ||
kbn-ambient-ui-types | ||
kbn-apm-synthtrace | ||
kbn-apm-synthtrace-client | ||
kbn-axe-config | ||
kbn-babel-preset | ||
kbn-babel-register | ||
kbn-babel-transform | ||
kbn-bazel-runner | ||
kbn-capture-oas-snapshot-cli | ||
kbn-check-mappings-update-cli | ||
kbn-check-prod-native-modules-cli | ||
kbn-ci-stats-core | ||
kbn-ci-stats-performance-metrics | ||
kbn-ci-stats-reporter | ||
kbn-ci-stats-shipper-cli | ||
kbn-cli-dev-mode | ||
kbn-code-owners | ||
kbn-cypress-config | ||
kbn-dependency-ownership | ||
kbn-dependency-usage | ||
kbn-dev-cli-errors | ||
kbn-dev-cli-runner | ||
kbn-dev-proc-runner | ||
kbn-dev-utils | ||
kbn-docs-utils | ||
kbn-es | ||
kbn-es-archiver | ||
kbn-eslint-config | ||
kbn-eslint-plugin-css | ||
kbn-eslint-plugin-disable | ||
kbn-eslint-plugin-eslint | ||
kbn-eslint-plugin-i18n | ||
kbn-eslint-plugin-imports | ||
kbn-eslint-plugin-telemetry | ||
kbn-expect | ||
kbn-failed-test-reporter-cli | ||
kbn-find-used-node-modules | ||
kbn-ftr-common-functional-services | ||
kbn-ftr-common-functional-ui-services | ||
kbn-ftr-screenshot-filename | ||
kbn-gen-ai-functional-testing | ||
kbn-generate | ||
kbn-generate-console-definitions | ||
kbn-get-repo-files | ||
kbn-import-locator | ||
kbn-import-resolver | ||
kbn-io-ts-utils | ||
kbn-jest-serializers | ||
kbn-journeys | ||
kbn-json-ast | ||
kbn-kibana-manifest-schema | ||
kbn-lint-packages-cli | ||
kbn-lint-ts-projects-cli | ||
kbn-managed-vscode-config | ||
kbn-managed-vscode-config-cli | ||
kbn-management | ||
kbn-manifest | ||
kbn-mock-idp-plugin | ||
kbn-mock-idp-utils | ||
kbn-openapi-bundler | ||
kbn-openapi-generator | ||
kbn-optimizer | ||
kbn-optimizer-webpack-helpers | ||
kbn-peggy | ||
kbn-peggy-loader | ||
kbn-performance-testing-dataset-extractor | ||
kbn-picomatcher | ||
kbn-plugin-check | ||
kbn-plugin-generator | ||
kbn-plugin-helpers | ||
kbn-relocate | ||
kbn-repo-file-maps | ||
kbn-repo-linter | ||
kbn-repo-path | ||
kbn-repo-source-classifier | ||
kbn-repo-source-classifier-cli | ||
kbn-scout | ||
kbn-scout-info | ||
kbn-scout-reporting | ||
kbn-set-map | ||
kbn-some-dev-log | ||
kbn-sort-package-json | ||
kbn-stdio-dev-helpers | ||
kbn-storybook | ||
kbn-styled-components-mapping-cli | ||
kbn-telemetry-tools | ||
kbn-test | ||
kbn-test-eui-helpers | ||
kbn-test-jest-helpers | ||
kbn-test-subj-selector | ||
kbn-tooling-log | ||
kbn-ts-projects | ||
kbn-ts-type-check-cli | ||
kbn-validate-next-docs-cli | ||
kbn-web-worker-stub | ||
kbn-whereis-pkg-cli | ||
kbn-yarn-lock-validator | ||
serverless/storybook/config | ||
README.md |
Kibana-related packages
This folder contains packages that are intended for use in Kibana and Kibana plugins.
tl;dr:
- Don't publish to npm registry
- Always use the
@kbn
namespace - Always set
"private": true
inpackage.json
Using these packages
We no longer publish these packages to the npm registry. Now, instead of specifying a version when including these packages, we rely on yarn workspaces, which sets up a symlink to the package.
For example if you want to use the @kbn/i18n
package in Kibana itself, you
can specify the dependency like this:
"@kbn/i18n": "1.0.0"
However, if you want to use this from a Kibana plugin, you need to use a link:
dependency and account for the relative location of the Kibana repo, so it would
instead be:
"@kbn/i18n": "link:../../kibana/src/platform/packages/shared/kbn-i18n"
then run yarn kbn bootstrap
from the plugin directory.
Creating a new package
Run the following command from the root of the Kibana repo:
node scripts/generate package @kbn/<PACKAGE_NAME> --web --owner @elastic/<TEAM_NAME>
Unit tests for a package
Currently there is only one tool being used in order to test packages which is Jest. Below we will explain how it should be done.
Jest
A package should follow the pattern of having .test.js
files as siblings of the source code files, and these run by Jest.
A package using the .test.js
naming convention will have those tests automatically picked up by Jest and run by the unit test runner, currently mapped to the Kibana test
script in the root package.json
.
yarn test
runs all unit tests.yarn jest
runs all Jest tests in Kibana.
In order for the plugin or package to use Jest, a jest.config.js file must be present in it's root. However, there are safeguards for this in CI should a test file be added without a corresponding config file.
Each package can also specify its own test
script in the package's package.json
, for cases where you'd prefer to run the tests from the local package directory.