kibana/scripts
Robert Oskamp 6272d5af6f
Add script to check test file code ownership (#173411)
## Summary

This PR adds a script that determines GitHub code ownership for
functional test files in the Kibana repository.

### Why do we need this?

We want to be able to determine test ownership to allow teams to get a
better overview of their tests (number of tests, number of skipped
tests, number of failures in the last x days, etc).

### What does this PR bring?

This PR is a first step on closing the test ownership gaps. It adds
functionality to determine the GitHub code owner for a given file (in
the `@kbn/code-owners` package) and adds a script that makes use of this
to check if all functional test files have a code owner, reporting the
gaps.

### Future plans

The idea is to include the test ownership information in our ingested
test results, such that we can create dashboards, reports, etc based on
it.
At some point (once all ownership gaps are closed), we might consider
running this check on CI to prevent new test files without owners.

### How to run?

```
node scripts/check_ftr_code_owners.js
```
The script lists the functional test files that are not covered by code
owners and also gives a summary like this:
```
ERROR Found 2592 test files without code owner (checked 7550 test files in 12.73 s)
```
2023-12-18 17:41:39 +01:00
..
archive_migration_functions.sh [QA] Fixup flaky test (#156373) 2023-05-08 11:32:53 +01:00
backport.js Bump backport to 8.5.0 (#133848) 2022-06-09 16:18:52 +01:00
build.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
build_api_docs.js Automatically generated Api documentation (#86232) 2021-02-24 19:23:19 -05:00
build_kibana_platform_plugins.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
build_plugin_list_docs.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
check_file_casing.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
check_ftr_code_owners.js Add script to check test file code ownership (#173411) 2023-12-18 17:41:39 +01:00
check_ftr_configs.js [CI] Add a check to ensure all FTR configs are in the manifest (#132857) 2022-05-26 11:18:13 -04:00
check_jest_configs.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
check_licenses.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
check_mappings_update.js [Saved Objects] Compatible mappings PR check (#148656) 2023-04-27 15:42:31 +02:00
chromium_version.js [Reporting/Build Chromium] Add a developer script to find the Chromium commit (#94033) 2021-03-10 10:57:29 -07:00
classify_source.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
create_observability_rules.js Add alerting_test_data package and a script to create rules for manual testing (#168493) 2023-10-18 12:00:09 +02:00
delete_kibana_indices.sh Script to delete .kibana indices (#157491) 2023-05-16 14:28:01 +02:00
dev_docs.sh [docs] Update yarn dev-docs config format (#137753) 2022-08-01 13:23:04 -07:00
docs.js [docs] rewrite docs cli to show logs and use modern apis (#117767) 2021-11-06 20:31:22 -05:00
download_pr_list.js Save github PR list in csv (#123276) 2022-01-20 20:28:09 +01:00
download_re2.sh Upgrade re2 from v1.17.7 to v1.20.1 (#162880) 2023-08-04 20:13:13 +02:00
enabled_ftr_configs.js [Archive Migrations] Bash functions to facilitate es to kbn archive migrations (#135929) 2022-07-13 10:31:32 +01:00
es.js Add mock identity provider for serverless (2nd attempt) (#171513) 2023-11-23 01:08:44 -07:00
es_archiver.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
eslint.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
eslint_with_types.js [eslint] enable type-specific lint rules (#114184) 2021-11-03 16:56:17 -06:00
extract_performance_testing_dataset.js [packages] add kbn-performance-testing-dataset-extractor (#131631) 2022-05-06 04:11:51 +02:00
find_babel_runtime_helpers_in_use.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
find_node_libs_browser_polyfills_in_use.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
functional_test_runner.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
functional_tests.js [ftr] automatically determine config run order (#130983) 2022-05-04 17:05:58 -05:00
functional_tests_server.js [ftr] add first-class support for playwrite journeys (#140680) 2022-09-22 01:06:46 -07:00
generate.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
generate_console_definitions.js [Console] Use ES specification for autocomplete definitions (#159241) 2023-06-23 12:05:25 -07:00
generate_openapi.js [Security Solution] Extract OpenAPI codegen to a package (#166269) 2023-09-25 10:51:40 +02:00
generate_plugin.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
generate_team_assignments.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
i18n_check.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
i18n_extract.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
i18n_integrate.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
ingest_coverage.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
jest.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
jest_integration.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
kbn.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
kbn_archiver.js [kbn/test] add import/export support to KbnClient (#92526) 2021-02-25 16:13:27 -07:00
kibana.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
kibana_encryption_keys.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
kibana_keystore.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
kibana_plugin.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
kibana_setup.js Add interactive setup CLI (#114493) 2021-10-20 22:17:45 +01:00
kibana_verification_code.js Verification code CLI (#111707) 2021-09-14 21:58:25 +01:00
licenses_csv_report.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
lint_packages.js Implement package linter (#148496) 2023-01-09 16:49:29 -07:00
lint_ts_projects.js Implement package linter (#148496) 2023-01-09 16:49:29 -07:00
makelogs.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
notice.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
plugin_helpers.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
precommit_hook.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
read_jest_help.mjs Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
README.md [ftr] automatically determine config run order (#130983) 2022-05-04 17:05:58 -05:00
register_git_hook.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
report_failed_tests.js [ftr] add first-class support for playwrite journeys (#140680) 2022-09-22 01:06:46 -07:00
report_performance_metrics.js ingest performance metrics to ci-stats (#134792) 2022-06-22 15:44:11 -07:00
run_performance.js Performance runner in js (#146129) 2022-11-29 19:43:35 +02:00
run_scalability.js [scalability testing] typescript runner (#147002) 2022-12-06 22:21:52 +01:00
saved_objs_info.js [QA][SO INFO SVC] Add cli (#108353) 2021-08-16 13:36:22 +01:00
ship_ci_stats.js [ci-stats] move shipper to a package, validate limits in on-merge job (#149474) 2023-01-25 08:20:40 -07:00
snapshot_plugin_types.js [Ops] Create SO migration snapshot comparion script (#168623) 2023-10-27 11:29:24 +02:00
spec_to_console.js chore(NA): moving @kbn/spec-to-console into bazel (#103470) 2021-06-28 15:58:41 +01:00
storybook.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
stylelint.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
synthtrace.js [packages] migrate all plugins to packages (#148130) 2023-02-08 21:06:50 -06:00
telemetry_check.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
telemetry_extract.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
test_hardening.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
type_check.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
update_prs.js Elastic License 2.0 (#90099) 2021-02-03 18:12:39 -08:00
update_vscode_config.js Transpile packages on demand, validate all TS projects (#146212) 2022-12-22 19:00:29 -06:00
validate_next_docs.js [ci] build next docs in PRs when relevant files change (#149991) 2023-02-09 21:57:10 -07:00
whereis_pkg.js chore(NA): adds @kbn/whereis-pkg-cli to quickly find a pkg location (#148696) 2023-01-13 17:45:24 -07:00
yarn_deduplicate.js [ci] Add node scripts/yarn_deduplicate check (#169165) 2023-10-19 18:13:48 -05:00

Kibana Dev Scripts

This directory contains scripts useful for interacting with Kibana tools in development. Use the node executable and --help flag to learn about how they work:

node scripts/{{script name}} --help

For Developers

This directory is excluded from the build and tools within it should help users discover their capabilities. Each script in this directory must:

  • require src/setup_node_env to bootstrap NodeJS environment
  • call out to source code in the src or packages directories
  • react to the --help flag
  • run everywhere OR check and fail fast when a required OS or toolchain is not available

Functional Test Scripts

node scripts/functional_tests [--config test/functional/config.base.js --config test/api_integration/config.js]

Runs all the functional tests: selenium tests and api integration tests. List configs with multiple --config arguments. Uses the @kbn/test library to run Elasticsearch and Kibana servers and tests against those servers, for multiple server+test setups. In particular, calls out to runTests(). Can be run on a single config.

node scripts/functional_tests_server [--config test/functional/config.base.js]

Starts just the Elasticsearch and Kibana servers given a single config, i.e. via --config test/functional/config.base.js or --config test/api_integration/config. Allows the user to start just the servers with this script, and keep them running while running tests against these servers. The idea is that the same config file configures both Elasticsearch and Kibana servers. Uses the startServers() method from @kbn/test library.

Example. Start servers and run tests, separately, but using the same config:

# Just the servers
node scripts/functional_tests_server --config path/to/config

In another terminal:

# Just the tests--against the running servers
node scripts/functional_test_runner --config path/to/config

For details on how the internal methods work, read this readme.

ES archiver

Loading data

If you wish to load up specific es archived data for your test, you can do so via:

node scripts/es_archiver.js load <archive> [--es-url=http://username:password@localhost:9200] [--kibana-url=http://username:password@localhost:5601/{basepath?}]

That will load the specified archive located in the archive directory specified by the default functional config file, located in test/functional/config.base.js. To load archives from other function config files you can pass --config path/to/config.js.

Note: The --es-url and --kibana-url options may or may not be neccessary depending on your current Kibana configuration settings, and their values may also change based on those settings (for example if you are not running with security you will not need the username:password portion).

Saving data

You can save existing data into an archive by using the save command:

node scripts/es_archiver.js save <archive name for kibana data> [space separated list of index patterns to include]

You may want to store the .kibana index separate from data. Since adding a lot of data will bloat our repo size, we have many tests that reuse the same data indices but use their own .kibana index.