afb09ccf8a
## Dearest Reviewers 👋 I've been working on this branch with @mistic and @tylersmalley and we're really confident in these changes. Additionally, this changes code in nearly every package in the repo so we don't plan to wait for reviews to get in before merging this. If you'd like to have a concern addressed, please feel free to leave a review, but assuming that nobody raises a blocker in the next 24 hours we plan to merge this EOD pacific tomorrow, 12/22. We'll be paying close attention to any issues this causes after merging and work on getting those fixed ASAP. 🚀 --- The operations team is not confident that we'll have the time to achieve what we originally set out to accomplish by moving to Bazel with the time and resources we have available. We have also bought ourselves some headroom with improvements to babel-register, optimizer caching, and typescript project structure. In order to make sure we deliver packages as quickly as possible (many teams really want them), with a usable and familiar developer experience, this PR removes Bazel for building packages in favor of using the same JIT transpilation we use for plugins. Additionally, packages now use `kbn_references` (again, just copying the dx from plugins to packages). Because of the complex relationships between packages/plugins and in order to prepare ourselves for automatic dependency detection tools we plan to use in the future, this PR also introduces a "TS Project Linter" which will validate that every tsconfig.json file meets a few requirements: 1. the chain of base config files extended by each config includes `tsconfig.base.json` and not `tsconfig.json` 1. the `include` config is used, and not `files` 2. the `exclude` config includes `target/**/*` 3. the `outDir` compiler option is specified as `target/types` 1. none of these compiler options are specified: `declaration`, `declarationMap`, `emitDeclarationOnly`, `skipLibCheck`, `target`, `paths` 4. all references to other packages/plugins use their pkg id, ie: ```js // valid { "kbn_references": ["@kbn/core"] } // not valid { "kbn_references": [{ "path": "../../../src/core/tsconfig.json" }] } ``` 5. only packages/plugins which are imported somewhere in the ts code are listed in `kbn_references` This linter is not only validating all of the tsconfig.json files, but it also will fix these config files to deal with just about any violation that can be produced. Just run `node scripts/ts_project_linter --fix` locally to apply these fixes, or let CI take care of automatically fixing things and pushing the changes to your PR. > **Example:** [` |
||
|---|---|---|
| .. | ||
| archive_migration_functions.sh | ||
| backport.js | ||
| build.js | ||
| build_api_docs.js | ||
| build_kibana_platform_plugins.js | ||
| build_plugin_list_docs.js | ||
| check_file_casing.js | ||
| check_ftr_configs.js | ||
| check_jest_configs.js | ||
| check_licenses.js | ||
| chromium_version.js | ||
| classify_source.js | ||
| dev_docs.sh | ||
| docs.js | ||
| download_pr_list.js | ||
| enabled_ftr_configs.js | ||
| es.js | ||
| es_archiver.js | ||
| eslint.js | ||
| eslint_with_types.js | ||
| extract_performance_testing_dataset.js | ||
| find_babel_runtime_helpers_in_use.js | ||
| find_node_libs_browser_polyfills_in_use.js | ||
| find_plugins_ready_to_migrate_to_ts_refs.js | ||
| find_plugins_with_circular_deps.js | ||
| find_plugins_without_ts_refs.js | ||
| functional_test_runner.js | ||
| functional_tests.js | ||
| functional_tests_server.js | ||
| generate.js | ||
| generate_plugin.js | ||
| generate_team_assignments.js | ||
| i18n_check.js | ||
| i18n_extract.js | ||
| i18n_integrate.js | ||
| ingest_coverage.js | ||
| jest.js | ||
| jest_integration.js | ||
| kbn.js | ||
| kbn_archiver.js | ||
| kibana.js | ||
| kibana_encryption_keys.js | ||
| kibana_keystore.js | ||
| kibana_plugin.js | ||
| kibana_setup.js | ||
| kibana_verification_code.js | ||
| licenses_csv_report.js | ||
| makelogs.js | ||
| notice.js | ||
| plugin_helpers.js | ||
| precommit_hook.js | ||
| read_jest_help.mjs | ||
| README.md | ||
| register_git_hook.js | ||
| report_failed_tests.js | ||
| report_performance_metrics.js | ||
| run_performance.js | ||
| run_scalability.js | ||
| saved_objs_info.js | ||
| ship_ci_stats.js | ||
| spec_to_console.js | ||
| storybook.js | ||
| stylelint.js | ||
| synthtrace.js | ||
| telemetry_check.js | ||
| telemetry_extract.js | ||
| test_hardening.js | ||
| ts_project_linter.js | ||
| type_check.js | ||
| update_prs.js | ||
| update_vscode_config.js | ||
Kibana Dev Scripts
This directory contains scripts useful for interacting with Kibana tools in development. Use the node executable and --help flag to learn about how they work:
node scripts/{{script name}} --help
For Developers
This directory is excluded from the build and tools within it should help users discover their capabilities. Each script in this directory must:
- require
src/setup_node_envto bootstrap NodeJS environment - call out to source code in the
srcorpackagesdirectories - react to the
--helpflag - run everywhere OR check and fail fast when a required OS or toolchain is not available
Functional Test Scripts
node scripts/functional_tests [--config test/functional/config.base.js --config test/api_integration/config.js]
Runs all the functional tests: selenium tests and api integration tests. List configs with multiple --config arguments. Uses the @kbn/test library to run Elasticsearch and Kibana servers and tests against those servers, for multiple server+test setups. In particular, calls out to runTests(). Can be run on a single config.
node scripts/functional_tests_server [--config test/functional/config.base.js]
Starts just the Elasticsearch and Kibana servers given a single config, i.e. via --config test/functional/config.base.js or --config test/api_integration/config. Allows the user to start just the servers with this script, and keep them running while running tests against these servers. The idea is that the same config file configures both Elasticsearch and Kibana servers. Uses the startServers() method from @kbn/test library.
Example. Start servers and run tests, separately, but using the same config:
# Just the servers
node scripts/functional_tests_server --config path/to/config
In another terminal:
# Just the tests--against the running servers
node scripts/functional_test_runner --config path/to/config
For details on how the internal methods work, read this readme.
ES archiver
Loading data
If you wish to load up specific es archived data for your test, you can do so via:
node scripts/es_archiver.js load <archive> [--es-url=http://username:password@localhost:9200] [--kibana-url=http://username:password@localhost:5601/{basepath?}]
That will load the specified archive located in the archive directory specified by the default functional config file, located in test/functional/config.base.js. To load archives from other function config files you can pass --config path/to/config.js.
Note: The --es-url and --kibana-url options may or may not be neccessary depending on your current Kibana configuration settings, and their values
may also change based on those settings (for example if you are not running with security you will not need the username:password portion).
Saving data
You can save existing data into an archive by using the save command:
node scripts/es_archiver.js save <archive name for kibana data> [space separated list of index patterns to include]
You may want to store the .kibana index separate from data. Since adding a lot of data will bloat our repo size, we have many tests that reuse the same
data indices but use their own .kibana index.