Your window into the Elastic Stack
Find a file
Walter Rafelsberger 0863a7f3a1
[8.0] [ML] APM Correlations: Fix usage in load balancing/HA setups. (#115145) (#117979)
* [ML] APM Correlations: Fix usage in load balancing/HA setups. (#115145)

- The way we customized the use of search strategies caused issues with race conditions when multiple Kibana instances were used for load balancing. This PR migrates away from search strategies and uses regular APM API endpoints.
- The task that manages calling the sequence of queries to run the correlations analysis is now in a custom React hook (useFailedTransactionsCorrelations / useLatencyCorrelations) instead of a task on the Kibana server side. While they show up as new lines/files in the git diff, the code for the hooks is more or less a combination of the previous useSearchStrategy and the server side service files that managed queries and state.
- The consuming React UI components only needed minimal changes. The above mentioned hooks return the same data structure as the previously used useSearchStrategy. This also means functional UI tests didn't need any changes and should pass as is.
- API integration tests have been added for the individual new endpoints. The test files that were previously used for the search strategies are still there to simulate a full analysis run, the assertions for the resulting data have the same values, it's just the structure that had to be adapted.
- Previously all ES queries of the analysis were run sequentially. The new endpoints run ES queries in parallel where possible. Chunking is managed in the hooks on the client side.
- For now the endpoints use the standard current user's esClient. I tried to use the APM client, but it was missing a wrapper for the fieldCaps method and I ran into a problem when trying to construct a random_score query. Sticking to the esClient allowed to leave most of the functions that run the actual queries unchanged. If possible I'd like to pick this up in a follow up. All the endpoints still use withApmSpan() now though.
- The previous use of generators was also refactored away, as mentioned above, the queries are now run in parallel.
Because we might run up to hundreds of similar requests for correlation analysis, we don't want the analysis to fail if just a single query fails like we did in the previous search strategy based task. I created a util splitAllSettledPromises() to handle Promise.allSettled() and split the results and errors to make the handling easier. Better naming suggestions are welcome 😅 . A future improvement could be to not run individual queries but combine them into nested aggs or using msearch. That's out of scope for this PR though.

* [ML] Fix http client types.
2021-11-09 12:42:36 +00:00
.buildkite [CI] Prep for the transition to build bot for github commit statuses (#117685) (#117718) 2021-11-05 20:23:55 +00:00
.ci Bump node to 16.13.0 (#116519) (#117253) 2021-11-03 05:06:32 +00:00
.github Add stale bot for APM issues (#116432) (#116569) 2021-10-29 07:16:05 +02:00
api_docs remove unused enrollment path (#117063) (#117200) 2021-11-02 20:04:27 +00:00
config [8.0] remove kibana.index config property (#112773) 2021-10-25 14:25:24 -07:00
dev_docs [docs] document using your @elastic.co email to commit (#117428) (#117452) 2021-11-04 02:51:37 +00:00
docs [8.0] [Security Solution] [Sourcerer] [Feature Branch] Update to use Kibana Data Views (#114806) (#117670) 2021-11-08 19:01:10 +00:00
examples Revert "[8.0] replace any with unknown in http client and types (#114265) (#117524)" (#117592) 2021-11-04 15:18:29 -05:00
legacy_rfcs Updates Github link references from master to main (#116789) (#116791) 2021-10-29 10:07:44 -07:00
licenses Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
packages Better support M1 users (#117766) (#117956) 2021-11-09 01:14:59 +00:00
plugins [dev/cli] ensure plugins/ and all watch source dirs exist (#78973) 2020-09-30 10:20:44 -07:00
scripts [8.0] [eslint] enable type-specific lint rules (#114184) (#117444) 2021-11-04 03:02:56 +00:00
src [TSVB] Fix reappearing of hidden series on refresh and styles loading (#117311) (#117985) 2021-11-09 12:10:50 +00:00
test [TSVB] Fix reappearing of hidden series on refresh and styles loading (#117311) (#117985) 2021-11-09 12:10:50 +00:00
typings remove unnecessary global typings (#117408) (#117551) 2021-11-04 18:11:33 +00:00
vars [8.0] [Fleet] cypress setup (#113106) (#117138) 2021-11-02 17:40:17 +00:00
x-pack [8.0] [ML] APM Correlations: Fix usage in load balancing/HA setups. (#115145) (#117979) 2021-11-09 12:42:36 +00:00
.backportrc.json chore(NA): adds backport config for 8.1.0 bump (#116431) (#116498) 2021-10-27 21:35:52 +01:00
.bazelignore chore(NA): stop grouping bazel out symlink folders (#96066) 2021-04-01 14:16:14 -05:00
.bazeliskversion Development and build support for ARM based Apple devices (#114879) 2021-10-19 23:01:52 +01:00
.bazelrc [bazel] Remove Bazel cache for now (#116351) 2021-10-26 12:04:17 -07:00
.bazelrc.common [ci] Disable Bazel cache on CI (#115428) 2021-10-19 22:12:31 +01:00
.bazelversion Development and build support for ARM based Apple devices (#114879) 2021-10-19 23:01:52 +01:00
.browserslistrc [browserslist] remove unnecessary browsers (#89186) 2021-01-25 16:30:18 -07:00
.editorconfig .editorconfig MDX files should follow the same rules as MD (#96942) 2021-04-13 11:40:42 -04:00
.eslintignore [kbn/ui-shared-deps] split into two packages (#110558) 2021-09-16 12:06:46 -07:00
.eslintrc.js [kibanaUtils] Don't import full semver client side (#114986) 2021-10-18 13:20:44 +02:00
.fossa.yml Adds FOSSA CLI configuration file (#70137) 2020-07-02 08:37:37 -07:00
.gitattributes [canvas] Color fixes + Storybook 5 (#34075) 2019-04-02 11:21:51 -05:00
.gitignore [8.0] [eslint] enable type-specific lint rules (#114184) (#117444) 2021-11-04 03:02:56 +00:00
.i18nrc.json [MetricVis] Move the expression function and renderer to a separate plugin. (#113427) 2021-10-08 08:42:44 +03:00
.node-version Bump node to 16.13.0 (#116519) (#117253) 2021-11-03 05:06:32 +00:00
.npmrc chore(NA): assure puppeteer_skip_chromium_download is applied across every yarn install situation (#88346) 2021-01-14 18:00:23 +00:00
.nvmrc Bump node to 16.13.0 (#116519) (#117253) 2021-11-03 05:06:32 +00:00
.prettierignore [dev] Replace sass-lint with stylelint (#86177) 2021-01-15 11:52:29 -06:00
.prettierrc Increase prettier line width to 100 (#20535) 2018-07-09 22:50:37 +02:00
.stylelintignore chore(NA): stop grouping bazel out symlink folders (#96066) 2021-04-01 14:16:14 -05:00
.stylelintrc Amsterdam helpers (#93701) 2021-03-10 10:27:16 -06:00
.telemetryrc.json [Usage collection] Collect non-default kibana configs (#97368) 2021-04-20 11:02:27 -04:00
.yarnrc chore(NA): manage npm dependencies within bazel (#92864) 2021-03-03 12:37:20 -05:00
api-documenter.json Normalize EOL symbol in platform docs (#56021) 2020-01-27 18:42:45 +01:00
BUILD.bazel [build_ts_refs] improve caches, allow building a subset of projects (#107981) 2021-08-10 22:12:45 -07:00
CODE_OF_CONDUCT.md Add CODE_OF_CONDUCT.md (#87439) 2021-02-23 09:01:51 +01:00
CONTRIBUTING.md Update doc slugs to improve analytic tracking, move to appropriate folders (#113630) 2021-10-04 13:36:45 -04:00
FAQ.md propose language changes (#10709) 2017-03-05 12:10:32 -05:00
github_checks_reporter.json implementing github checks - second attempt (#35757) 2019-05-01 16:02:33 -05:00
Jenkinsfile [CI] Disable tracked branch jobs in Jenkins, enable reporting in Buildkite (#112604) 2021-09-21 11:31:15 -04:00
jest.config.integration.js chore(NA): introduce preset for jest-integration tests on @kbn/test (#105144) 2021-07-14 20:53:12 +01:00
jest.config.js [Canvas] Expression tagcloud (#108036) 2021-08-23 06:52:19 -04:00
kibana.d.ts Remove /src/legacy (#95510) 2021-04-06 09:25:36 +02:00
LICENSE.txt Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
NOTICE.txt [Detections] Adds automatic updating for Prebuilt Security Detection Rules package (#101846) 2021-06-24 15:31:25 -06:00
package.json Better support M1 users (#117766) (#117956) 2021-11-09 01:14:59 +00:00
preinstall_check.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
README.md STYLEGUIDE reference .md => .mdx (#107440) 2021-08-02 14:05:58 -05:00
RISK_MATRIX.mdx Add "Risk Matrix" section to the PR template (#100649) 2021-06-02 14:43:47 +02:00
SECURITY.md Add security policy to the Kibana repository (#85407) 2020-12-10 09:26:00 -05:00
STYLEGUIDE.mdx remove all references to v7 theme (#113570) (#116824) 2021-11-01 02:27:29 +00:00
tsconfig.base.json Adds missing DOM.Iterable (#115218) 2021-10-18 13:22:49 -04:00
tsconfig.bazel.json [build_ts_refs] improve caches, allow building a subset of projects (#107981) 2021-08-10 22:12:45 -07:00
tsconfig.browser.json Introduce TS incremental builds & move src/test_utils to TS project (#76082) 2020-09-03 14:20:04 +02:00
tsconfig.browser_bazel.json [build_ts_refs] improve caches, allow building a subset of projects (#107981) 2021-08-10 22:12:45 -07:00
tsconfig.json Add interactive setup CLI (#114493) 2021-10-20 22:17:45 +01:00
tsconfig.types.json [build_ts_refs] improve caches, allow building a subset of projects (#107981) 2021-08-10 22:12:45 -07:00
TYPESCRIPT.md Fixed grammar (#74725) 2020-08-11 06:40:22 -04:00
WORKSPACE.bazel Bump node to 16.13.0 (#116519) (#117253) 2021-11-03 05:06:32 +00:00
yarn.lock [Charts] fix legend breaking on words (#117681) 2021-11-08 18:08:24 +00:00

Kibana

Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.

Getting Started

If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.

If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.

Using a Kibana Release

If you want to use a Kibana release in production, give it a test run, or just play around:

Building and Running Kibana, and/or Contributing Code

You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:

Documentation

Visit Elastic.co for the full Kibana documentation.

For information about building the documentation, see the README in elastic/docs.

Version Compatibility with Elasticsearch

Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.

Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.

Situation Example Kibana version Example ES version Outcome
Versions are the same. 5.1.2 5.1.2 💚 OK
ES patch number is newer. 5.1.2 5.1.5 ⚠️ Logged warning
ES minor number is newer. 5.1.2 5.5.0 ⚠️ Logged warning
ES major number is newer. 5.1.2 6.0.0 🚫 Fatal error
ES patch number is older. 5.1.2 5.1.0 ⚠️ Logged warning
ES minor number is older. 5.1.2 5.0.0 🚫 Fatal error
ES major number is older. 5.1.2 4.0.0 🚫 Fatal error

Questions? Problems? Suggestions?

  • If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
  • Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.