* [ftr] automatically determine config run order
* split lens config into two groups
* support ftr configs always running against CI
* Split detection_engine_api_integration rule exception list tests
* Add configs from previous commit
* [ftr] remove testMetadata and maintain a unique lifecycle instance per run
* Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run"
This reverts commit d2b4fdb824.
* Split alerting_api_integration/security_and_spaces tests
* Add groups to yaml
* Revert "Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run""
This reverts commit 56232eea68.
* stop ES more forcefully and fix timeout
* only cleanup lifecycle phases when the cleanup is totally complete
* only use kill when cleaning up an esTestInstance
* fix broken import
* fix runOptions.alwaysUseSource implementation
* fix config access
* fix x-pack/ccs config
* fix ml import file paths
* update kibana build id
* revert array.concat() change
* fix baseConfig usage
* fix pie chart data
* split up maps tests
* pull in all of group5 so that es archives are loaded correctly
* add to ftr configs.yml
* fix pie chart data without breaking legacy version
* fix more pie_chart stuff in new vis lib
* restore normal PR tasks
* bump kibana-buildkite-library
* remove ciGroup validation
* remove the script which is no longer called from checks.sh
* [CI] Auto-commit changed files from 'yarn kbn run build -i @kbn/pm'
* adapt flaky test runner scripts to handle ftrConfig paths
* fix types in alerting_api_integration
* improve flaky config parsing and use non-local var name for passing explicit configs to ftr_configs.sh
* Split xpack dashboard tests
* Add configs
* [flaky] remove key from ftr-config steps
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
* restore cypress builds
* remove ciGroups from FTR config files
* fixup some docs
* add temporary script to hunt for FTR config files
* use config.base.js naming for clarity
* use script to power ftr_configs.yml
* remove usage of removed x-pack/scripts/functional_tests
* fix test names in dashboard snapshots
* bump kibana-buildkite-library
* Try retrying only failed configs
* be a little quieter about trying to get testStats from configs with testRunners defined
* Remove test code
* bump kibana-buildkite-library
* update es_snapshot and on_merge jobs too
* track duration and exit code for each config and print it at the end of the script
* store results in order, rather than by key, in case there are duplicates in $config
* bash is hard
* fix env source and use +e rather than disabling e for whole file
* bash sucks
* print config summary in jest jobs too
* define results in jest_parallel.sh
* simplify config summary print, format times a little better
* fix reference to unbound time variable, use better variable name
* skip the newline between each result
* finish with the nitpicking
* sync changes with ftr_configs.sh
* refuse to execute config files which aren't listed in the .buildkite/ftr_configs.yml
* fix config.edge.js base config import paths
* fix some readmes
* resolve paths from ftr_configs manifest
* fix readConfigFile tests
* just allow __fixtures__ configs
* list a few more cypress config files
* install the main branch of kibana-buildkite-library
* split up lens group1
* move ml data_visualizer tests to their own config
* fix import paths
* fix more imports
* install specific commit of buildkite-pipeline-library
* sort configs in ftr_configs.yml
* bump kibana-buildkite-library
* remove temporary script
* fix env var for limiting config types
* Update docs/developer/contributing/development-functional-tests.asciidoc
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* produce a JUnit report for saved objects field count
* apply standard concurrency limits from flaky test runner
* support customizing FTR concurrency via the env
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* Migrated Discover visualize tests to use CCS.
* Fixed.
* Forgot to add the config file to the functional tests command,
* Fixed nits in PR.
* Removed unused references.
* Removed unused references.
* Removed test file that was removed in the merge.
* Added saved_queries test to run when CCS is enabled and put test that wasn't supposed to be removed back.
* Used conditional in index pattern.
* Added CCS test for data view functionality.
* Changed how we are approaching the test. Just adding a remote to exercise the CCS functionality.
* Adjusted test to use correct port for remote cluster.
* Added and corrected config for the tests to be able to run.
* Changed test title.
* Added functional ccs suite to functional test runner script.
* Changed config.ts to config.js
* Moved es to before clause.
* Added await for an async call.
* Added await for an async call.
* Updated saved queries test to use CCS. One test is failing, all others are passing.
* Merge branch 'main' of github.com:elastic/kibana into CCS_Functional_Test_POC
* Fixed a reference.
* Removed the OSS tests since we created CCS versions of them.
* Removed tests from index file.
* Restored test files per comment with matthias.
* Did some cleanup of comments.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [code coverage] collect coverage on server side
* fix script
* copy server coverage to functional
* [code coverage] run all e2e and api tests
* enable all configs for coverage, fix reportDir
* [code coverage] use KIBANA_DIR var
* create target/kibana-coverage/functional if not exist
* remove empty lines and unused code
* adjust include/exclude filters
* exclude more test-related paths
* Update functional nyc config
* Update server nyc config
* exclude more dirs
* exclude stub/
* remove unnecessary ts-ignore
* add context propagation to x-opaque-id header tests
* run tests on CI
* simplify logging. the action purpose follows from the context name
* extend tests with the assertion against execution_context from the Kibana logs
* split JSON log records only
* apply suggestions proposed by Spencer
* mark legacy ES client types as deprecated
* expose es client to plugins and update mocks
* ElasticSearchClientMock --> ElasticsearchClientMock
* expose es client mocks
* expose es client via RequestHandlerContext
* convert test/plugin_functional/config into ts
* convert top_nav test into ts
* add an integration test for the es client
* update comments to refer to the new es client
* fix import paths. do not use extensions
temp
* update docs
* fix other refs
* add test for a custom client
* fix context
* add test for scoped client
* update docs
* Plugging in DashboardStart dependency
* Create embeddable by reference and navigate back to dashboard
* Trying to feature flag the new flow
* Feature flagging new visualize flow
* Removing unnecessary console statement
* Fixing typescript errors
* Adding a functional test for new functionality
* Adding a functional test for new functionality
* Fixing test name
* Changing test name
* Moving functional test to a separate folder
* Trying to fix the config file
* Adding an index file
* Remove falsly included file
* Adding aggs and params to vis input
* Serializing vis before passing it as an input
* Incorporating new state transfer logic
* Remove dashboardStart as a dependency
* Trying to get the test to run
* Remove unused import
* Readding spaces
* Fixing type errors
* Incorporating new changes
* Added base folder structure for Newsfeed plugin
* Added base folders for lib and component
* Added newsfeed button to navigation controls on the right side
* add getApi() to return api data observable (#49581)
* Added flyout base body and provided EuiHeaderAlert component inside the newsfeed plugin
* Moved newsfeed plugin to OSS and added for the styles purpose new folder for legacy plugin 'newsfeed' with the same id to support this
* Added subscribe on fetch newsfeed change
* Add NewsfeedApiDriver class (#49710)
* add NewsfeedApiDriver class
* fix xpack prefix
* add corner case handling
* Added data binding to the ui
* added EuiHeaderAlert style overrides (#49739)
* Fixed due to comments on PR
* add missing fields to NewsfeedItem and FetchResult
* fix templating of service url
* gracefully handle temporary request failure
* Mapped missing fields for data and badge
* Fixed typos issues
* integrate i18n.getLocale()
* allow service url root to be changed in dev mode
* replace a lot of consts with config
* fix flyout height (#49809)
* Add "error" field to FetchResult: Error | null
* simplify fetch error handling
* Do not store hash for items that are filtered out
* add expireOn in case it is useful to UI
* always use staging url for dev config
* unit test for newsfeed api driver
* simplify modelItems
* Fixed eslint errors
* Fixed label translations
* Add unit test for concatenating the stored hashes with the new
* add newsfeed to i18n.json
* Fixed expression error
* --wip-- [skip ci]
* fix parse error
* fix test
* test(newsfeed): Added testing endpoint which simulates the Elastic Newsfeed for consumption in functional tests
* add tests for getApi()
* add tests for getApi
* Added no news page
* fix fetch not happening after page refresh with sessionStorage primed
* test(newsfeed): Added testing endpoint which simulates the Elastic Newsfeed for consumption in functional tests
* Added loading screen
* Small fixes due to comments
* Fixed issue with stop fetching news on error catch
* test(newsfeed): Configure FTS to point newsfeed to the simulated newsfeed endpoit
* Fixed browser error message: Invariant Violation: [React Intl] Could not find required `intl` object. <IntlProvider> needs to exist in the component ancestry.
* Fixed typo issue in label name
* polish the code changes
* Add simple jest/enzyme tests for the components
* honor utc format
* Filter pre-published items
* Fall back to en
* retry tests
* comment clarfication
* Setup newsfeed service fixture from test/common/config
* Added base functional tests for newsfeed functionality
* valid urlroot is for prod
* add documentation for the supported enabled setting
* more urlRoot
* --wip-- [skip ci]
* add the before for fn
* add ui_capabilties test
* update jest snapshot
* Fixed failing test
* finish newsfeed error functional test
* include ui_capability config
* error case testing in ci group 6
* refactor(newsfeed): moved newsfeed api call so that it is done before its use
* code polish
* enabled newsfeed_err test in CI
Right now the functional tests are run against a distributable of Kibana in CI, but that distributable is running with `--env.name=development`. That causes the optimizer to run again before the tests can start and prevents the functional tests from running against the actual version of the application users will end up getting. This seems necessary for some tests, but not all of them, but I would like to get all of the tests running against the production version of Kibana soon.
This PR implements a second ftr config, `test/functional_production` that uses basically a copy of the `test/functional` config but with a few minor adjustments, removing the `--env.name=development` kbnServerArg and using a unique junit report name. To accomplish this I needed to modify the `@kbn/test` module to only pass the `--dev` flag to the Kibana server if it is being run in development mode, which it currently does by testing the args for `--env.name=development` or two args next to each other: `'--env.name'` and `'development'`. It does this by converting the `extraKbnOpts` option into an `addExtraKbnArgs` function, which is called with the final args just before passing them to the proc runner and given a chance to modify then after all other args are resolved (pulling from different places in config based on the build type, etc.)
Over the next couple weeks I'll push up PRs for individual test suites, migrating them over to the new production config, hopefully in a short period of time we will have all the function test suites back under that `test/function` config and can run them all against the Kibana server in production mode.
* [kbn-test] convert kibana-install-dir flag to installDir option
* [kbn-test] replicate kibana-install-dir handling to startServers
* [ftr] try running functional tests in production in CI
* Revert "[ftr] try running functional tests in production in CI"
This reverts commit e5b94aa024.
* [core/public/legacyPlatform] exclude ui/test_harness from the distributable
* [optimizer] fix `process.env.IS_KIBANA_DISTRIBUTABLE` definition
* [optimizer] only define `process.env.IS_KIBANA_DISTRIBUTABLE` when needed
Adding a `webpack.DefinePlugin` slows down the optimizer a small amount,
so only apply it when it is necessary, and skip it if it is going to
be defined as "false".
* [kbn-test/startServer] don't run in --dev mode if running from dist
* [ftr/kibanaServer/version] attach `-SNAPSHOT` suffix to version if running build_snapshot
In order to make the license that applies to each file as clear as possible, and to be consistent with elasticsearch, we are adding Apache 2.0 license headers to the top of each file.
Existence of this header is enforced by eslint and tslint and missing headers were automatically added in the last commit by running:
```
node scripts/eslint --fix && node scripts/tslint --fix
```
* feat(12976): node version validation at runtime.
* refact(12976): move the code into a static utilities class.
* test(12976): added first test case using jest.
* test(12976): added test cases for node_version.
* feat(12976): create setup env node to bootstrap babel, ts-node and node version validator.
* refact(12976): migrated node version code from es6 to es5.
* feat(12976): node version validation at runtime.
* refact(12976): move the code into a static utilities class.
* test(12976): added first test case using jest.
* test(12976): added test cases for node_version.
* feat(12976): create setup env node to bootstrap babel, ts-node and node version validator.
* refact(12976): migrated node version code from es6 to es5.
* fix(12976): remove one level from ts node register cache directory link.
* chore(12976): added caret to semver dependecy in order to support minor versions.
* refact(12976): small change from named import to default import on node version validator.
* refact(12976): removed ts_node_register and add the code to babel_register.
* feat(12976): split eslint config in order to properly support files built to run before and after node version validator. refact(12976): convert script files to es5 code. refact(12976): delete inline eslint configs from node version check related files.
* refact(12976): remove ts node register file.
* refact(12976): completely port setup_node_env to es5.
* refact(12976): remove babel_register invokation from external dependencies in scripts.
* refact(12976): move node_version code directly into node_version_validator inside setup_node_env folder.
* refact(12976): only node version validator for kbn script.
Restructure testing with kbn-test package
- Run with multiple configs, move cli options to config
- Package-ify kbn-test
- Eventually we'll have functional_test_runner live in a package
of its own, and then this kbn-test will use that as a dependency,
probably still as a devDependency.
- Implement functional_tests_server
- Collapse single and multiple config apis into one command
Use kbn-es
Replace es_test_cluster + es_test_config with kbn/test utils
Implement new createEsTestCluster
Improve scripts, jsdocs, cli top-level tools
Lift error handling to the top level
