Your window into the Elastic Stack
Find a file
Marshall Main 6073eb6dcd
[Security Solution] Swap rule unions out for discriminated unions to improve validation error messages (#171452)
**Epics:** https://github.com/elastic/security-team/issues/8058,
https://github.com/elastic/security-team/issues/6726 (internal)
**Partially addresses:**
https://github.com/elastic/security-team/issues/7991 (internal)

## Summary

The main benefit of this PR is shown in `rule_request_schema.test.ts`,
where the error messages are now more accurate and concise. With regular
unions, `zod` has to try validating the input against all schemas in the
union and reports the errors from every schema in the union. Switching
to discriminated unions, with `type` as the discriminator, allows `zod`
to pick the right rule type schema from the union and only validate
against that rule type. This means the error message reports that either
the discriminator is invalid, in any case where `type` is not valid, or
if `type` is valid but another field is wrong for that type of rule then
the error message is the validation result from only that rule type.

To make it possible to use discriminated unions, we need to switch from
using zod's `.and()` for intersections to `.merge()` because `.and()`
returns an intersection type that is incompatible with discriminated
unions in zod. Similarly, we need to remove the usage of `.transform()`
because it returns a ZodEffect that is incompatible with `.merge()`.

Instead of using `.transform()` to turn properties from optional to
possibly undefined, we can use `requiredOptional` explicitly in specific
places to convert the types. Similarly, the `RequiredOptional` type can
be used on the return type of conversion functions between API and
internal schemas to enforce that all properties are explicitly specified
in the conversion.

Future work:
- better alignment of codegen with OpenAPI definitions of anyOf/oneOf.
https://swagger.io/docs/specification/data-models/oneof-anyof-allof-not/#oneof
oneOf requires that the input match exactly one schema from the list,
which is different from z.union. anyOf should be z.union, oneOf should
be z.discriminatedUnion
- flatten the schema structure further to avoid `Type instantiation is
excessively deep and possibly infinite`. Seems to be a common issue with
zod (https://github.com/microsoft/TypeScript/issues/34933) Limiting the
number of `.merge` and other zod operations needed to build a particular
schema object seems to help resolve the error. Combining
`ResponseRequiredFields` and `ResponseOptionalFields` into a single
object rather than merging them solved the immediate problem. However,
we may still be near the depth limit. Changing `RuleResponse` as seen
below also solved the problem in testing, and may give us more headroom
for future changes if we apply techniques like this here and in other
places. The difference here is that `SharedResponseProps` is only
intersected with the type specific schemas after they're combined in a
discriminated union, whereas in `main` we merge `SharedResponseProps`
with each individual schema then merge them all together.
- combine other Required and Optional schemas, like
QueryRuleRequiredFields and QueryRuleOptionalFields

```ts
export type RuleResponse = z.infer<typeof RuleResponse>;
export const RuleResponse = SharedResponseProps.and(z.discriminatedUnion('type', [
  EqlRuleResponseFields,
  QueryRuleResponseFields,
  SavedQueryRuleResponseFields,
  ThresholdRuleResponseFields,
  ThreatMatchRuleResponseFields,
  MachineLearningRuleResponseFields,
  NewTermsRuleResponseFields,
  EsqlRuleResponseFields,
]));
```

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-11-29 21:11:29 +01:00
.buildkite [build] Rename ubi9 image to ubi (#171424) 2023-11-29 11:50:22 -06:00
.ci Remove CI Composite Storybook (#171258) 2023-11-21 14:59:39 -07:00
.github [ML] Transforms/Data frame analytics: Align data view / destination index creation workflow in wizards. (#171202) 2023-11-29 18:40:35 +00:00
api_docs [api-docs] 2023-11-29 Daily api_docs build (#172126) 2023-11-29 00:59:13 -05:00
config [Serverless][Advanced Settings] Add validations on user input (#170234) 2023-11-16 11:07:51 -07:00
dev_docs async-import plugins in the server side (#170856) 2023-11-15 00:55:56 -07:00
docs [Response Ops][Alerting] Edit text in Elasticsearch query rule and index threshold rule (#171896) 2023-11-28 17:37:29 -07:00
examples async-import plugins in the server side (#170856) 2023-11-15 00:55:56 -07:00
kbn_pm chore(NA): exit early with link for docs when using kbn_pm on windows (#139745) 2023-02-10 15:38:29 +00:00
legacy_rfcs rename @elastic/* packages to @kbn/* (#138957) 2022-08-18 08:54:42 -07:00
licenses Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
packages [Security Solution] Swap rule unions out for discriminated unions to improve validation error messages (#171452) 2023-11-29 21:11:29 +01:00
plugins
scripts Add mock identity provider for serverless (2nd attempt) (#171513) 2023-11-23 01:08:44 -07:00
src [build] Rename ubi9 image to ubi (#171424) 2023-11-29 11:50:22 -06:00
test skip failing es promotion suite (#172215) 2023-11-29 19:15:06 +00:00
typings Upgrade EUI to v88.3.0 (#166676) 2023-09-20 08:03:02 -05:00
vars [RAM] Alert table all column fix 2 (#161054) 2023-07-07 09:57:22 -06:00
x-pack [Security Solution] Swap rule unions out for discriminated unions to improve validation error messages (#171452) 2023-11-29 21:11:29 +01:00
.backportrc.json chore(NA): adds 8.11 into backportrc (#168025) 2023-10-04 20:10:33 +01:00
.bazelignore Bazel config maintenance (#135442) 2022-07-05 10:20:26 -05:00
.bazeliskversion chore(NA): upgrade bazelisk into v1.11.0 (#125070) 2022-02-09 20:43:57 +00:00
.bazelrc chore(NA): use new and more performant BuildBuddy servers (#130350) 2022-04-18 02:01:38 +01:00
.bazelrc.common Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
.bazelversion chore(NA): revert bazel upgrade for v5.2.0 (#135096) 2022-06-24 03:57:21 +01:00
.browserslistrc [browserslist] remove unnecessary browsers (#89186) 2021-01-25 16:30:18 -07:00
.editorconfig .editorconfig MDX files should follow the same rules as MD (#96942) 2021-04-13 11:40:42 -04:00
.eslintignore [NOTICE.txt] Fix notices for Gainsight and FullStory (#146004) 2023-01-13 14:49:43 +01:00
.eslintrc.js Improvements for eslint-i18n-package (#171588) 2023-11-23 05:02:36 -07:00
.gitattributes
.gitignore [APM] Add snapshots to visual regression tests (#165834) 2023-09-11 08:59:16 +01:00
.i18nrc.json Reporting packages for export types (#162845) 2023-11-15 10:48:00 -07:00
.node-version Revert "Upgrade to Node.js 20 (#162696)" 2023-11-21 07:15:23 -06:00
.npmrc chore(NA): assure puppeteer_skip_chromium_download is applied across every yarn install situation (#88346) 2021-01-14 18:00:23 +00:00
.nvmrc Revert "Upgrade to Node.js 20 (#162696)" 2023-11-21 07:15:23 -06:00
.prettierignore [dev] Replace sass-lint with stylelint (#86177) 2021-01-15 11:52:29 -06:00
.prettierrc
.stylelintignore chore(NA): stop grouping bazel out symlink folders (#96066) 2021-04-01 14:16:14 -05:00
.stylelintrc Bump stylelint to ^14 (#136693) 2022-07-20 10:11:00 -05:00
.telemetryrc.json [Telemetry] Fix telemetry-tools TS parser for packages (#149819) 2023-01-31 04:09:09 +03:00
.yarnrc chore(NA): manage npm dependencies within bazel (#92864) 2021-03-03 12:37:20 -05:00
BUILD.bazel Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
catalog-info.yaml Add the buildkite-pipeline to the catalog-info.yaml (#171741) 2023-11-29 02:56:09 -07:00
CODE_OF_CONDUCT.md Add CODE_OF_CONDUCT.md (#87439) 2021-02-23 09:01:51 +01:00
CONTRIBUTING.md Update doc slugs to improve analytic tracking, move to appropriate folders (#113630) 2021-10-04 13:36:45 -04:00
FAQ.md Fix small typos in the root md files (#134609) 2022-06-23 09:36:11 -05:00
fleet_packages.json [main] Sync bundled packages with Package Storage (#170250) 2023-10-31 09:30:00 -07:00
github_checks_reporter.json
Jenkinsfile [CI] Disable tracked branch jobs in Jenkins, enable reporting in Buildkite (#112604) 2021-09-21 11:31:15 -04:00
kibana.d.ts fix all violations 2022-04-16 01:37:30 -05:00
LICENSE.txt Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
nav-kibana-dev.docnav.json Enhance plugin documentation (#146678) 2023-11-28 20:58:46 +01:00
NOTICE.txt [FullStory] Update snippet (#153570) 2023-04-18 04:06:05 -07:00
package.json [ML] Transforms/Data frame analytics: Align data view / destination index creation workflow in wizards. (#171202) 2023-11-29 18:40:35 +00:00
preinstall_check.js Always throw error objects - never strings (#171498) 2023-11-20 09:23:16 -05:00
README.md [README] Update version Compatibility with Elasticsearch (#116040) 2022-01-10 10:31:21 -05:00
renovate.json [chore] Adjust Renovate settings for Storybook (#172032) 2023-11-27 16:14:39 -08:00
RISK_MATRIX.mdx Add "Risk Matrix" section to the PR template (#100649) 2021-06-02 14:43:47 +02:00
SECURITY.md
STYLEGUIDE.mdx [styleguide] update path to scss theme (#140742) 2022-09-15 10:41:14 -04:00
tsconfig.base.json [ML] Transforms/Data frame analytics: Align data view / destination index creation workflow in wizards. (#171202) 2023-11-29 18:40:35 +00:00
tsconfig.browser.json
tsconfig.browser_bazel.json [build_ts_refs] improve caches, allow building a subset of projects (#107981) 2021-08-10 22:12:45 -07:00
tsconfig.json Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
TYPESCRIPT.md Fix small typos in the root md files (#134609) 2022-06-23 09:36:11 -05:00
versions.json chore(NA): update versions after v7.17.16 bump (#171136) 2023-11-14 15:35:15 +00:00
WORKSPACE.bazel Revert "Upgrade to Node.js 20 (#162696)" 2023-11-21 07:15:23 -06:00
yarn.lock [ML] Transforms/Data frame analytics: Align data view / destination index creation workflow in wizards. (#171202) 2023-11-29 18:40:35 +00:00

Kibana

Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.

Getting Started

If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.

If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.

Using a Kibana Release

If you want to use a Kibana release in production, give it a test run, or just play around:

Building and Running Kibana, and/or Contributing Code

You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:

Documentation

Visit Elastic.co for the full Kibana documentation.

For information about building the documentation, see the README in elastic/docs.

Version Compatibility with Elasticsearch

Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.

Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.

Situation Example Kibana version Example ES version Outcome
Versions are the same. 7.15.1 7.15.1 💚 OK
ES patch number is newer. 7.15.0 7.15.1 ⚠️ Logged warning
ES minor number is newer. 7.14.2 7.15.0 ⚠️ Logged warning
ES major number is newer. 7.15.1 8.0.0 🚫 Fatal error
ES patch number is older. 7.15.1 7.15.0 ⚠️ Logged warning
ES minor number is older. 7.15.1 7.14.2 🚫 Fatal error
ES major number is older. 8.0.0 7.15.1 🚫 Fatal error

Questions? Problems? Suggestions?

  • If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
  • Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.