kibana/scripts
Dario Gieselaar 6d0e4b8d28
[8.19] [Inference] Instrument inference with OpenTelemetry (#218694) (#220349)
# Backport

This will backport the following commits from `main` to `8.19`:
- [[Inference] Instrument inference with OpenTelemetry
(#218694)](https://github.com/elastic/kibana/pull/218694)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Dario
Gieselaar","email":"dario.gieselaar@elastic.co"},"sourceCommit":{"committedDate":"2025-05-07T09:44:29Z","message":"[Inference]
Instrument inference with OpenTelemetry (#218694)\n\nInstrument the
inference chatComplete API with OpenTelemetry, and export\nhelper
functions to create spans w/ the right semconv
attributes.\nAdditionally, optionally export to Langfuse or
Phoenix.\n\n## Centralizes OpenTelemetry setup\n\nAs this is the first
instance of OpenTelemetry based _tracing_ (we\nalready have metrics in
the MonitoringCollection plugin), some\nbootstrapping code is necessary
to centrally configure OpenTelemetry. To\nthis end, I've added the
following config settings:\n\n- `telemetry.tracing.enabled`: whether
OpenTelemetry tracing is enabled\n(defaults to undefined, if undefined,
falls back to `telemetry.enabled`)\n- `telemetry.tracing.sample_rate`
(defaults to 1)\n\nThe naming of these configuration settings is mostly
in-line with [the\nElasticsearch
tracing\nsettings](https://github.com/elastic/elasticsearch/blob/main/TRACING.md).\n\nThe
following packages (containing bootstrapping logic, utility\nfunctions,
types and config schemas) were added:\n- `@kbn/telemetry`\n-
`@kbn/telemetry-config` \n- `@kbn/tracing`\n\nThe OpenTelemetry
bootstrapping depends on @kbn/apm-config-loader, as it\nhas the same
constraints - it needs to run before any other code, and it\nneeds to
read the raw config.\n\nAdditionally, a root `telemetry` logger was
added that captures\nOpenTelemetry logs.\n\nNote that there is no
default exporter for spans, which means that\nalthough spans are being
recorded, they do not get exported.\n\n## Instrument chatComplete
calls\n\nCalls to `chatComplete` now create OpenTelemetry spans,
roughly\nfollowing semantic conventions (which for GenAI are very much
in flux).\nSome helper functions were added to create other inference
spans. These\nhelper functions use baggage to determine whether the
created inference\nspan is the \"root\" of an inference trace. This
allows us to export these\nspans as if it were root spans - something
that is needed to be able to\neasily visualize these in other
tools.\n\nLeveraging these inference spans, two exporters are added. One
for\n[Phoenix](https://github.com/Arize-ai/phoenix) and one
for\n[Langfuse](https://github.com/langfuse/langfuse/tree/main):
two\nopen-source LLM Observability suites. This allows engineers that
use the\nInference plugin to be able to inspect and improve their
LLM-based\nworkflows with much less effort.\n\nFor both Phoenix and
Langfuse, two service scripts were added. Run `node\nscripts/phoenix` or
`node scripts/langfuse` to get started. Both scripts\nwork with
zero-config - they will log generated Kibana config to
stdout.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"2387e3b88c83317ddd5354dff751a1ac0fb62bb9","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Obs
AI
Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[Inference]
Instrument inference with
OpenTelemetry","number":218694,"url":"https://github.com/elastic/kibana/pull/218694","mergeCommit":{"message":"[Inference]
Instrument inference with OpenTelemetry (#218694)\n\nInstrument the
inference chatComplete API with OpenTelemetry, and export\nhelper
functions to create spans w/ the right semconv
attributes.\nAdditionally, optionally export to Langfuse or
Phoenix.\n\n## Centralizes OpenTelemetry setup\n\nAs this is the first
instance of OpenTelemetry based _tracing_ (we\nalready have metrics in
the MonitoringCollection plugin), some\nbootstrapping code is necessary
to centrally configure OpenTelemetry. To\nthis end, I've added the
following config settings:\n\n- `telemetry.tracing.enabled`: whether
OpenTelemetry tracing is enabled\n(defaults to undefined, if undefined,
falls back to `telemetry.enabled`)\n- `telemetry.tracing.sample_rate`
(defaults to 1)\n\nThe naming of these configuration settings is mostly
in-line with [the\nElasticsearch
tracing\nsettings](https://github.com/elastic/elasticsearch/blob/main/TRACING.md).\n\nThe
following packages (containing bootstrapping logic, utility\nfunctions,
types and config schemas) were added:\n- `@kbn/telemetry`\n-
`@kbn/telemetry-config` \n- `@kbn/tracing`\n\nThe OpenTelemetry
bootstrapping depends on @kbn/apm-config-loader, as it\nhas the same
constraints - it needs to run before any other code, and it\nneeds to
read the raw config.\n\nAdditionally, a root `telemetry` logger was
added that captures\nOpenTelemetry logs.\n\nNote that there is no
default exporter for spans, which means that\nalthough spans are being
recorded, they do not get exported.\n\n## Instrument chatComplete
calls\n\nCalls to `chatComplete` now create OpenTelemetry spans,
roughly\nfollowing semantic conventions (which for GenAI are very much
in flux).\nSome helper functions were added to create other inference
spans. These\nhelper functions use baggage to determine whether the
created inference\nspan is the \"root\" of an inference trace. This
allows us to export these\nspans as if it were root spans - something
that is needed to be able to\neasily visualize these in other
tools.\n\nLeveraging these inference spans, two exporters are added. One
for\n[Phoenix](https://github.com/Arize-ai/phoenix) and one
for\n[Langfuse](https://github.com/langfuse/langfuse/tree/main):
two\nopen-source LLM Observability suites. This allows engineers that
use the\nInference plugin to be able to inspect and improve their
LLM-based\nworkflows with much less effort.\n\nFor both Phoenix and
Langfuse, two service scripts were added. Run `node\nscripts/phoenix` or
`node scripts/langfuse` to get started. Both scripts\nwork with
zero-config - they will log generated Kibana config to
stdout.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"2387e3b88c83317ddd5354dff751a1ac0fb62bb9"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/218694","number":218694,"mergeCommit":{"message":"[Inference]
Instrument inference with OpenTelemetry (#218694)\n\nInstrument the
inference chatComplete API with OpenTelemetry, and export\nhelper
functions to create spans w/ the right semconv
attributes.\nAdditionally, optionally export to Langfuse or
Phoenix.\n\n## Centralizes OpenTelemetry setup\n\nAs this is the first
instance of OpenTelemetry based _tracing_ (we\nalready have metrics in
the MonitoringCollection plugin), some\nbootstrapping code is necessary
to centrally configure OpenTelemetry. To\nthis end, I've added the
following config settings:\n\n- `telemetry.tracing.enabled`: whether
OpenTelemetry tracing is enabled\n(defaults to undefined, if undefined,
falls back to `telemetry.enabled`)\n- `telemetry.tracing.sample_rate`
(defaults to 1)\n\nThe naming of these configuration settings is mostly
in-line with [the\nElasticsearch
tracing\nsettings](https://github.com/elastic/elasticsearch/blob/main/TRACING.md).\n\nThe
following packages (containing bootstrapping logic, utility\nfunctions,
types and config schemas) were added:\n- `@kbn/telemetry`\n-
`@kbn/telemetry-config` \n- `@kbn/tracing`\n\nThe OpenTelemetry
bootstrapping depends on @kbn/apm-config-loader, as it\nhas the same
constraints - it needs to run before any other code, and it\nneeds to
read the raw config.\n\nAdditionally, a root `telemetry` logger was
added that captures\nOpenTelemetry logs.\n\nNote that there is no
default exporter for spans, which means that\nalthough spans are being
recorded, they do not get exported.\n\n## Instrument chatComplete
calls\n\nCalls to `chatComplete` now create OpenTelemetry spans,
roughly\nfollowing semantic conventions (which for GenAI are very much
in flux).\nSome helper functions were added to create other inference
spans. These\nhelper functions use baggage to determine whether the
created inference\nspan is the \"root\" of an inference trace. This
allows us to export these\nspans as if it were root spans - something
that is needed to be able to\neasily visualize these in other
tools.\n\nLeveraging these inference spans, two exporters are added. One
for\n[Phoenix](https://github.com/Arize-ai/phoenix) and one
for\n[Langfuse](https://github.com/langfuse/langfuse/tree/main):
two\nopen-source LLM Observability suites. This allows engineers that
use the\nInference plugin to be able to inspect and improve their
LLM-based\nworkflows with much less effort.\n\nFor both Phoenix and
Langfuse, two service scripts were added. Run `node\nscripts/phoenix` or
`node scripts/langfuse` to get started. Both scripts\nwork with
zero-config - they will log generated Kibana config to
stdout.\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"2387e3b88c83317ddd5354dff751a1ac0fb62bb9"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2025-05-09 07:35:24 -04:00
..
codeql [8.x] [CodeQL] Local run script (#194272) (#197989) 2024-10-28 09:23:36 -05:00
archive_migration_functions.sh [8.x] SKA: Relocate /test to /src/platform/test (#210956) (#214791) 2025-03-18 14:48:04 +01:00
backport.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
build.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
build_api_docs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
build_kibana_platform_plugins.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
build_plugin_list_docs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
build_product_doc_artifacts.js [8.x] [KB] create @kbn/product-doc-artifact-builder package (#193847) (#196206) 2024-10-15 08:02:35 +02:00
capture_oas_snapshot.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
check_file_casing.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
check_ftr_code_owners.js [8.x] [ftr] Speed up FTR code owner check (#205093) (#205415) 2025-01-02 12:53:22 -06:00
check_ftr_configs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
check_jest_configs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
check_licenses.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
check_mappings_update.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
check_prod_native_modules.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
chromium_version.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
classify_source.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
create_observability_rules.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
delete_kibana_indices.sh Script to delete .kibana indices (#157491) 2023-05-16 14:28:01 +02:00
dependency_usage.sh [Backport] Dependency Usage CLI (#201815) 2024-11-26 21:48:26 +01:00
dev_docs.sh [Docs] Added OAS dev tutorial (#187573) 2024-07-05 08:47:14 +00:00
docs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
download_pr_list.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
eis.js [8.19] [Inference] Run EIS locally (#215475) (#218909) 2025-05-01 16:16:44 +02:00
enabled_ftr_configs.js [8.19] Updated js-yaml to v4 (#190678) (#219852) 2025-05-02 13:16:51 -04:00
es.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
es_archiver.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
eslint.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
eslint_with_types.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
extract_performance_testing_dataset.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
find_babel_runtime_helpers_in_use.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
find_node_libs_browser_polyfills_in_use.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
functional_test_runner.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
functional_tests.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
functional_tests_server.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
generate.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
generate_console_definitions.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
generate_openapi.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
generate_plugin.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
generate_team_assignments.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
i18n_check.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
i18n_extract.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
i18n_integrate.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
ingest_coverage.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
jest.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
jest_integration.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kbn.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kbn_archiver.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kibana.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kibana_encryption_keys.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kibana_keystore.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kibana_plugin.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kibana_setup.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
kibana_verification_code.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
langfuse.js [8.19] [Inference] Instrument inference with OpenTelemetry (#218694) (#220349) 2025-05-09 07:35:24 -04:00
licenses_csv_report.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
lint_packages.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
lint_ts_projects.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
makelogs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
manifest.js [8.x] Add ESLINT constraints to detect inter-group dependencies (#194810) (#197670) 2024-10-24 13:20:30 -07:00
notice.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
phoenix.js [8.19] [Inference] Instrument inference with OpenTelemetry (#218694) (#220349) 2025-05-09 07:35:24 -04:00
plugin_check.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
plugin_helpers.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
precommit_hook.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
prettier_topology_check.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
profile.js [8.x] @kbn/profiler-cli: collect and display CPU profiles (#216356) (#216774) 2025-04-03 07:38:42 -04:00
quick_checks.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
read_jest_help.mjs [8.x] SKA: Relocate "platform" packages that remain on /packages (#208704) (#212474) 2025-02-28 10:12:01 +00:00
README.md [8.x] SKA: Relocate /test to /src/platform/test (#210956) (#214791) 2025-03-18 14:48:04 +01:00
register_git_hook.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
relocate.js [8.x] Sustainable Kibana Architecture: Relocate script v5 (#204522) 2024-12-17 13:57:27 +01:00
report_failed_tests.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
report_performance_metrics.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
rewrite_buildkite_definitions.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
run_performance.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
run_scalability.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
saved_objs_info.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
scout.js [8.x] [kbn-scout] Custom event-oriented test reporter & persistence (#202906) (#204696) 2024-12-18 05:06:58 -06:00
ship_ci_stats.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
snapshot_plugin_types.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
spec_to_console.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
storybook.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
styled_components_mapping.js [8.x] Update styled_components_files.js to include all files that import styled-components (#206084) 2025-01-10 22:11:34 +00:00
stylelint.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
synthtrace.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
telemetry_check.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
telemetry_extract.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
test_hardening.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
type_check.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
update_prs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
update_vscode_config.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
validate_next_docs.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
whereis_pkg.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00
yarn_deduplicate.js Adds AGPL 3.0 license (#192025) 2024-09-06 19:02:41 -06:00

Kibana Dev Scripts

This directory contains scripts useful for interacting with Kibana tools in development. Use the node executable and --help flag to learn about how they work:

node scripts/{{script name}} --help

For Developers

This directory is excluded from the build and tools within it should help users discover their capabilities. Each script in this directory must:

  • require src/setup_node_env to bootstrap NodeJS environment
  • call out to source code in the src or packages directories
  • react to the --help flag
  • run everywhere OR check and fail fast when a required OS or toolchain is not available

Functional Test Scripts

node scripts/functional_tests [--config src/platform/test/functional/config.base.js --config test/api_integration/config.js]

Runs all the functional tests: selenium tests and api integration tests. List configs with multiple --config arguments. Uses the @kbn/test library to run Elasticsearch and Kibana servers and tests against those servers, for multiple server+test setups. In particular, calls out to runTests(). Can be run on a single config.

node scripts/functional_tests_server [--config src/platform/test/functional/config.base.js]

Starts just the Elasticsearch and Kibana servers given a single config, i.e. via --config src/platform/test/functional/config.base.js or --config test/api_integration/config. Allows the user to start just the servers with this script, and keep them running while running tests against these servers. The idea is that the same config file configures both Elasticsearch and Kibana servers. Uses the startServers() method from @kbn/test library.

Example. Start servers and run tests, separately, but using the same config:

# Just the servers
node scripts/functional_tests_server --config path/to/config

In another terminal:

# Just the tests--against the running servers
node scripts/functional_test_runner --config path/to/config

For details on how the internal methods work, read this readme.

ES archiver

Loading data

If you wish to load up specific es archived data for your test, you can do so via:

node scripts/es_archiver.js load <archive> [--es-url=http://username:password@localhost:9200] [--kibana-url=http://username:password@localhost:5601/{basepath?}]

That will load the specified archive located in the archive directory specified by the default functional config file, located in src/platform/test/functional/config.base.js. To load archives from other function config files you can pass --config path/to/config.js.

Note: The --es-url and --kibana-url options may or may not be neccessary depending on your current Kibana configuration settings, and their values may also change based on those settings (for example if you are not running with security you will not need the username:password portion).

Saving data

You can save existing data into an archive by using the save command:

node scripts/es_archiver.js save <archive name for kibana data> [space separated list of index patterns to include]

You may want to store the .kibana index separate from data. Since adding a lot of data will bloat our repo size, we have many tests that reuse the same data indices but use their own .kibana index.