* [partial backport] extract kbnServer test utils from #12554
* Implement esTestCluster test util (#13099)
* [es/tests] remove unused module
* [testUtil/es] add utility for starting es nodes in tests
* [ftr/tests] use esTestCluster util to start es
* [es/tests/routes] use esTestCluster to start own es
* [testUtils/kbnServer] disable uiSettings unless plugins are enabled
* [testUtils/esTestCluster] use standard test es port by default
* [server/http/tests] add esTestCluster to setup
* [test/config] unify es test config into a single module
* [testUtils/esTestCluster] directory is no longer configurable
* [testUtils/esTestCluster] throw when es.start() is called again without es.stop()
* [testUtils/esTestCluster] is* checks should not mutate state
(cherry picked from commit 6748b22d03)
* [testUtils/esTestCluster] use more standard api style (#13197)
(cherry picked from commit d36080bca8)
* [scanner] use new esTestConfig service
* [indexPatterns] support cross cluster patterns
* [vis] remove unused `hasTimeField` param
* [indexPatterns/create] fix method name in view
* [indexPatterns/create] disallow expanding with ccs
* [indexPatterns/create] field fetching is cheaper, react faster
* [indexPatterns/resolveTimePattern/tests] increase readability
* [tests/apiIntegration/indexPatterns] test conflict field output
* [indexPatterns/fieldCaps/readFieldCapsResponse] add unit tests
* [test/apiIntegration] ensure random word will not be valid
* [indexPatterns/ui/client] remove unused import
* remove use of auto-release-sinon
* [indexPatterns/create] don't allow expand when cross cluster
* [indexPatternsApiClient/stub] use angular promises
* [indexPatterns/create] add tests for base create ui behaviors
(cherry picked from commit e5deca679c)
* [tests/functional] move screenshots to their own service
* [ftr] add testFailure and testHookFailure lifecycle hooks
* [tests/functional/screenshots] cleanup old screenshots at startup
* [test/functional/screenshots] take screenshots when tests fail
* [cli_plugin/install] fix test
* [ui/scanner] fix test
(cherry picked from commit 2e7fed87fd)
* [ftr] remove digdug, use chromedriver directly
why?
- digdug is meant to be used by intern, and expects a certain lifecycle that the FTR has tried and continuously fails to mimic
- rather than continue trying to force digdug into the stack, just spawn chromedriver ourselves. We know how to handle it
- cleans up verbose remote logging while we're in there, since selenium-standalone-server went with digdug, which was previously doing the verbose logging
- deprecate config.servers.webdriver
- add config.chromedriver.url
- if url at config.chromedriver.url points to a server that responds to http requests use it as the chromedriver instance, enables running chrome in a VM or container
- if pings to config.chromedriver.url fail a local chromedriver is started
* address review requests
(cherry picked from commit f76bef46c0)
* [esArchiver] combine elasticDump and ScenarioManager (#10359)
* As a part of bringing functional testing to plugins, esArchiver gives these plugins a way to capture and reload es indexes without needing to write a bunch of custom code. It works similarly to the elasticDump and ScenarioManager tools that it replaces.
Differences:
- Streaming implementation allows for much larger archives
- CLI for creating and using archives
- Configurable archive location
- Stores the data in gzipped files (better for source control, searching, large archives)
- Automatically identifies and upgrades Kibana config documents
Methods:
- `#load(name)`: import an archive
- `#loadIfNeeded(name)`: import an archive, but skip the documents what belong to any existing index
- `#unload(name)`: delete the indexes stored in an archive
CLI operations:
- `./bin/es_archiver save <name> [index patterns...]`: save the mapping and documents in one or more indexes that match the wild-card patterns into an the `<name>` archive
- `./bin/es_archiver load <name>`: load the mapping and documents from the `<name>` archive
* [functional_tests/common/nagivate] check for statusPage
* [es_archiver] move bins into new scripts dir
* [functional_tests/apps/context] use esArchiver
* [esArchiver] general improvements after showing to a few folks
- remove auto-upgrading config doc logic (until we have better access to kibana version info)
- export unload command
- remove preemptive checks in favor of reacting to errors
- use type "doc" vs "hit" for doc records (consistency)
- wrote a bunch of pending tests to think though and plan
* [esArchiver] make log a stream that writes to itself
* [esArchiver] fill in stats and archive format tests
* [esArchiver] splitup action logic
* [esArchiver/cli] fix cli --help output and comment
* [esArchiver] remove type-based param coercion
* [esArchiver/log] use strings for log levels
* [esArchvier] remove unused var
* [esArchiver/indexDocRecordsStream] add tests
* [esArchive] fill in remaining tests
* [esArchiver] fix dem tests
* [eslint] remove unused vars
* [esArchiver/loadIfNeeded] fix call to load()
* [esArchiver] remove loadDumpData helpers
* [esArchiver] update archives for 5.x
* [functionalTestRunner] replace intern (#10910)
* [functional_test_runner] replace functional testing tools with custom/pluggable solution
* [functional_test_runner] Convert unit tests to commonjs format
* [functional_test_runner] Fix dashboard test in wrong mode
* [functional_test_runner] Add dashboardLandingPage test subject
* [functional_test_runner] Get Visualize page object
* [functional_test_runner] Fix outdated references
* [functional_test_runner] Fix more outdated refs
* [functional_test_runner] Remove duplicate tests
* [functional_test_runner] Improve test readability
* [functional_test_runner] 😞 So many duplicate methods
* [functional_test_runner] Move mgmt `before` outside toplevel describe
* [functional_test_runner] Settings page obj missing methods
* [functional_test_runner] Add improvements from @gammon
* [functional_test_runner] Fix return statements in async funcs
* [functional_test_runner] Move before() to correct scope
* [functional_test_runner] Add after() hooks to remove index patterns
* [functional_test_runner] Attempt to fix vertical bar chart tests
* [functional_test_runner] Clean up
* [functional_test_runner] Reinstate unit tests
* [functional_test_runner] Set default loglevel back to info
* [functional_test_runner] Replace `context`s with `describe`s
* [functional_test_runner] Better error handling
* [functional_test_runner] Add in new Tile Map tests
* Incorporate changes from master
* [functional_test_runner] validate that every test file has a single top-level suite
* Update contributing doc with link to full doc
* [docs] Spelling and grammar fixes
* docs: writing and running functional tests
* [docs] Move plugin doc to plugin area
* [docs] Housekeeping. Doc in wrong place
* [docs] Remove dup doc file
* [grunt] Only run mocha_setup when running tests, not every grunt task
* [eslint] remove use of context()
* Create KuiButton, KuiLinkButton, KuiSubmitButton, and KuiButtonIcon React components in UI Framework.
* Add Jest test coverage for UI Framework, generate report in ui_framework/jest/report.
* Add UI Framework to linting task.
* Update UI Framework README with instructions on creating and testing React components.
* Add both React and HTML examples.
* Add UI Framework Jest tests to npm test script. Create separate scripts for watching and generating coverage reports.
* Fix appearance of kuiButtons with icons throughout Kibana, by adding a flexbox wrapper.
* Improve accessibility of kuiButtonIcon.
* Remove disabled attribute from KuiLinkButton.
* Remove kuiButton-isDisabled class from KuiButton and KuiSubmitButton.
Backports PR #9133
**Commit 1:**
Update PageObjects for Visualize, Dashboard, and Discover to use data-test-subj selectors for menu items.
* Original sha: 84cf9ba9de
* Authored by CJ Cenizal <cj@cenizal.com> on 2016-11-17T21:28:09Z
**Commit 2:**
Run checkPlugins task as part of test:ui:runner task.
* Original sha: 7cdf829629
* Authored by CJ Cenizal <cj@cenizal.com> on 2016-11-19T18:07:47Z
Backports PR #8981
**Commit 1:**
Add checkPlugins task to check for plugins before running tests.
* Original sha: 9e71c779ba
* Authored by CJ Cenizal <cj@cenizal.com> on 2016-11-05T05:02:13Z
**Commit 2:**
Remove checkPlugins npm script. Gracefully handle error case.
* Original sha: 98ec182915
* Authored by CJ Cenizal <cj@cenizal.com> on 2016-11-09T20:51:47Z
**Commit 3:**
Move checkPlugins task into its own module.
* Original sha: bba727461d
* Authored by CJ Cenizal <cj@cenizal.com> on 2016-11-15T23:40:41Z
---------
**Commit 1:**
[tests/functional] auto-rebuild assets in test:ui:server task
While working on some functional tests I had to re-run the `test:ui:server` task several times to rebuild the front-end assets. I'm not sure why that should be necessary, so this updates the server used in that specific task to auto-rebuild the assets.
* Original sha: ea303935a1
* Authored by spalger <email@spalger.com> on 2016-09-19T20:15:04Z
The tests in master are currently failing regularly because our current browser tests are serious memory hogs. Investigation reveals that nearly every test is retaining all of the memory it causes to be allocated. We have made some progress to being able to diagnose the problems, but we expect that problem to take some serious work to fix. We need a short-term solution though, and this is it.
Rather than modify the bundling process, we will shard the top-level test suites by name. For now, we've created 4 shards, but adding new shards is trivial if we need to.
Sharding is accomplished by creating a murmur3 hash of the top level suite names, then bucketing based on the hash output. If a test suite resolves to shard2, but we are running shard1, we simply never pass the function to `mocha.describe()`. Rather than redefine every describe statement, we have shimmed the global `window.describe()` function to accomplish this.
- Add 'test:visualRegression' grunt task.
- Run 'test:visualRegression' as part of npm script 'test'.
- Add 'clean:screenshots task'.
- Clean screenshots/session when funtional tests are run.
Adding a 'suites' property to the existing intern configuration caused
an error to be thrown at the beginning of the functional test run. Even
if the value of 'suites' was just an empty array. The existence of the
property seemed to enabled execution of the config file in the selenium
browser because it complained about not having the node require
function. To fix this, I created a separate api test config file without
the node require and removed the 'suites' property from intern.js.
