[functionalTestRunner] replace intern (#10910)

* [functional_test_runner] replace functional testing tools with custom/pluggable solution

* [functional_test_runner] Convert unit tests to commonjs format

* [functional_test_runner] Fix dashboard test in wrong mode

* [functional_test_runner] Add dashboardLandingPage test subject

* [functional_test_runner] Get Visualize page object

* [functional_test_runner] Fix outdated references

* [functional_test_runner] Fix more outdated refs

* [functional_test_runner] Remove duplicate tests

* [functional_test_runner] Improve test readability

* [functional_test_runner] 😞 So many duplicate methods

* [functional_test_runner] Move mgmt `before` outside toplevel describe

* [functional_test_runner] Settings page obj missing methods

* [functional_test_runner] Add improvements from @gammon

* [functional_test_runner] Fix return statements in async funcs

* [functional_test_runner] Move before() to correct scope

* [functional_test_runner] Add after() hooks to remove index patterns

* [functional_test_runner] Attempt to fix vertical bar chart tests

* [functional_test_runner] Clean up

* [functional_test_runner] Reinstate unit tests

* [functional_test_runner] Set default loglevel back to info

* [functional_test_runner] Replace `context`s with `describe`s

* [functional_test_runner] Better error handling

* [functional_test_runner] Add in new Tile Map tests

* Incorporate changes from master

* [functional_test_runner] validate that every test file has a single top-level suite

* Update contributing doc with link to full doc

* [docs] Spelling and grammar fixes

* docs: writing and running functional tests

* [docs] Move plugin doc to plugin area

* [docs] Housekeeping. Doc in wrong place

* [docs] Remove dup doc file

* [grunt] Only run mocha_setup when running tests, not every grunt task

CONTRIBUTING.md

@@ -308,12 +308,7 @@ npm run test:ui:runner
 
 ##### Browser Automation Notes
 
-- Using Page Objects pattern (https://theintern.github.io/intern/#writing-functional-test)
-- At least the initial tests for the Settings, Discover, and Visualize tabs all depend on a very specific set of logstash-type data (generated with makelogs).  Since that is a static set of data, all the Discover and Visualize tests use a specific Absolute time range.  This guarantees the same results each run.
-- These tests have been developed and tested with Chrome and Firefox browser.  In theory, they should work on all browsers (that's the benefit of Intern using Leadfoot).
-- These tests should also work with an external testing service like https://saucelabs.com/ or https://www.browserstack.com/ but that has not been tested.
-- https://theintern.github.io/
-- https://theintern.github.io/leadfoot/module-leadfoot_Element.html
+[Read about the `FunctionalTestRunner`](https://www.elastic.co/guide/en/kibana/current/development-functional-tests.html) to learn more about how you can run and develop functional tests for Kibana core and plugins.
 
 ### Building OS packages
 
@@ -374,4 +369,4 @@ Remember, someone is blocked by a pull awaiting review, make it count. Be thorou
 1. **Hand it off** If you're the first reviewer and everything looks good but the changes are more than a few lines, hand the pull to someone else to take a second look. Again, try to find the right person to assign it to.
 1. **Merge the code** When everything looks good, put in a `LGTM` (looks good to me) comment. Merge into the target branch. Check the labels on the pull to see if backporting is required, and perform the backport if so.
 
-Thank you so much for reading our guidelines! :tada:
+Thank you so much for reading our guidelines! :tada:

Gruntfile.js

@@ -57,7 +57,7 @@ module.exports = function (grunt) {
     init: true,
     config: config,
     loadGruntTasks: {
-      pattern: ['grunt-*', '@*/grunt-*', 'gruntify-*', '@*/gruntify-*', 'intern']
+      pattern: ['grunt-*', '@*/grunt-*', 'gruntify-*', '@*/gruntify-*']
     }
   });
 

docs/development/core-development.asciidoc

@@ -5,6 +5,7 @@
 * <<development-dependencies>>
 * <<development-modules>>
 * <<development-elasticsearch>>
+* <<development-functional-tests>>
 
 include::core/development-basepath.asciidoc[]
 
@@ -12,4 +13,6 @@ include::core/development-dependencies.asciidoc[]
 
 include::core/development-modules.asciidoc[]
 
-include::plugin/development-elasticsearch.asciidoc[]
+include::core/development-elasticsearch.asciidoc[]
+
+include::core/development-functional-tests.asciidoc[]

docs/development/core/development-functional-tests.asciidoc

@@ -0,0 +1,383 @@
+[[development-functional-tests]]
+=== Functional Testing
+
+We use functional tests to make sure the Kibana UI works as expected. It replaces hours of manual testing by automating user interaction. To have better control over our functional test environment, and to make it more accessible to plugin authors, Kibana uses a tool called the `FunctionalTestRunner`.
+
+[float]
+==== Running functional tests
+
+The `FunctionalTestRunner` is very bare bones and gets most of its functionality from its config file, located at {blob}test/functional/config.js[test/functional/config.js]. If you’re writing a plugin you will have your own config file. See <<development-plugin-functional-tests>> for more info.
+
+Execute the `FunctionalTestRunner`'s script with node.js to run the tests with Kibana's default configuration:
+
+["source","shell"]
+-----------
+node scripts/functional_test_runner
+-----------
+
+When run without any arguments the `FunctionalTestRunner` automatically loads the configuration in the standard location, but you can override that behavior with the `--config` flag. There are also command line flags for `--bail` and `--grep`, which behave just like their mocha counterparts. The logging can also be customized with `--quiet`, `--debug`, or `--verbose` flags.
+
+Use the `--help` flag for more options.
+
+[float]
+==== Writing functional tests
+
+[float]
+===== Environment
+
+The tests are written in https://mochajs.org[mocha] using https://github.com/Automattic/expect.js[expect] for assertions.
+
+We use https://sites.google.com/a/chromium.org/chromedriver/[chromedriver], https://theintern.github.io/leadfoot[leadfoot], and https://github.com/theintern/digdug[digdug] for automating Chrome. When the `FunctionalTestRunner` launches, digdug opens a `Tunnel` which starts chromedriver and a stripped-down instance of Chrome. It also creates an instance of https://theintern.github.io/leadfoot/module-leadfoot_Command.html[Leadfoot's `Command`] class, which is available via the `remote` service. The `remote` communicates to Chrome through the digdug `Tunnel`. See the https://theintern.github.io/leadfoot/module-leadfoot_Command.html[leadfoot/Command API] docs for all the commands you can use with `remote`.
+
+The `FunctionalTestRunner` automatically transpiles functional tests using babel, so that tests can use the same ECMAScript features that Kibana source code uses. See [style_guides/js_style_guide.md]({blob}style_guides/js_style_guide.md).
+
+[float]
+===== Definitions
+
+**Provider:**
+
+Code run by the `FunctionalTestRunner` is wrapped in a function so it can be passed around via config files and be parameterized. Any of these Provider functions may be asynchronous and should return/resolve-to the value they are meant to *provide*. Provider functions will always be called with a single argument: a provider API (see Provider API section).
+
+A config provder:
+
+["source","js"]
+-----------
+// config and test files use `export default`
+export default function (/* { providerAPI } */) {
+  return {
+    // ...
+  }
+}
+-----------
+
+**Services:**
+
+Services are named singleton values produced by a Service Provider. Tests and other services can retrieve service instances by asking for them by name. All functionality except the mocha API is exposed via services.
+
+**Page objects:**
+
+Page objects are a special type of service that encapsulate behaviors common to a particular page or plugin. When you write your own plugin, you’ll likely want to add a page object (or several) that describes the common interactions your tests need to execute.
+
+**Test Files:**
+
+The `FunctionalTestRunner`'s primary purpose is to execute test files. These files export a Test Provider that is called with a Provider API but is not expected to return a value. Instead Test Providers define a suite using https://mochajs.org/#bdd[mocha's BDD interface].
+
+**Test Suite:**
+
+A test suite is a collection of tests defined by calling `describe()`, and then populated with tests and setup/teardown hooks by calling `it()`, `before()`, `beforeEach()`, etc. Every test file must define only one top level test suite, and test suites can have as many nested test suites as they like.
+
+[float]
+===== Anatomy of a test file
+
+The annotated example file below shows the basic structure every test suite uses. It starts by importing https://github.com/Automattic/expect.js[`expect.js`] and defining its default export: an anonymous Test Provider. The test provider then destructures the Provider API for the `getService()` and `getPageObjects()` functions. It uses these functions to collect the dependencies of this suite. The rest of the test file will look pretty normal to mocha.js users. `describe()`, `it()`, `before()` and the lot are used to define suites that happen to automate a browser via services and objects of type `PageObject`.
+
+["source","js"]
+----
+import expect from 'expect.js';
+// test files must `export default` a function that defines a test suite
+export default function ({ getService, getPageObject }) {
+
+  // most test files will start off by loading some services
+  const retry = getService('retry');
+  const testSubjects = getService('testSubjects');
+  const esArchiver = getService('esArchiver');
+
+  // for historical reasons, PageObjects are loaded in a single API call
+  // and returned on an object with a key/value for each requested PageObject
+  const PageObjects = getPageObjects(['common', 'visualize']);
+
+  // every file must define a top-level suite before defining hooks/tests
+  describe('My Test Suite', () => {
+
+    // most suites start with a before hook that navigates to a specific
+    // app/page and restores some archives into elasticsearch with esArchiver
+    before(async () => {
+  await Promise.all([
+        // start with an empty .kibana index
+        esArchiver.load('empty_kibana'),
+        // load some basic log data only if the index doesn't exist
+        esArchiver.loadIfNeeded('makelogs')
+      ]);
+      // go to the page described by `apps.visualize` in the config
+      await PageObjects.common.navigateTo('visualize');
+    });
+
+    // right after the before() hook definition, add the teardown steps
+    // that will tidy up elasticsearch for other test suites
+    after(async () => {
+      // we unload the empty_kibana archive but not the makelogs
+      // archive because we don't make any changes to it, and subsequent
+      // suites could use it if they call `.loadIfNeeded()`.
+      await esArchiver.unload('empty_kibana');
+    });
+
+    // This series of tests illustrate how tests generally verify
+    // one step of a larger process and then move on to the next in
+    // a new test, each step building on top of the previous
+    it('Vis Listing Page is empty');
+    it('Create a new vis');
+    it('Shows new vis in listing page');
+    it('Opens the saved vis');
+    it('Respects time filter changes');
+    it(...
+  });
+
+}
+----
+
+[float]
+==== Provider API
+
+The first and only argument to all providers is a Provider API Object. This object can be used to load service/page objects and config/test files.
+
+Within config files the API has the following properties
+
+* `log` - An instance of the {blob}src/utils/tooling_log/tooling_log.js[`ToolingLog`] that is ready for use
+
+* `readConfigFile(path)` - Returns a promise that will resolve to a Config instance that provides the values from the config file at `path`
+
+Within service and PageObject Providers the API is:
+
+* `getService(name)` - Load and return the singleton instance of a service by name
+
+* `getPageObjects(names)` - Load the singleton instances of `PageObject`s and collect them on an object where each name is the key to the singleton instance of that PageObject
+
+Within a test Provider the API is exactly the same as the service providers API but with an additional method:
+
+* `loadTestFile(path)` - Load the test file at path in place. Use this method to nest suites from other files into a higher-level suite
+
+[float]
+==== Service Index
+
+[float]
+===== Built-in Services
+
+The `FunctionalTestRunner` comes with three built-in services:
+
+**config:**
+
+* Source: {blob}src/functional_test_runner/lib/config/config.js[src/functional_test_runner/lib/config/config.js]
+
+* Schema: {blob}src/functional_test_runner/lib/config/schema.js[src/functional_test_runner/lib/config/schema.js]
+
+* Use `config.get(path)` to read any value from the config file
+
+**log:**
+
+* Source: {blob}src/utils/tooling_log/tooling_log.js[src/utils/tooling_log/tooling_log.js]
+* `ToolingLog` instances are readable streams. The instance provided by this service is automatically piped to stdout by the `FunctionalTestRunner` CLI
+* `log.verbose()`, `log.debug()`, `log.info()`, `log.warning()` all work just like console.log but produce more organized output
+
+**lifecycle:**
+
+* Source: {blob}src/functional_test_runner/lib/lifecycle.js[src/functional_test_runner/lib/lifecycle.js]
+* Designed primary for use in services
+* Exposes lifecycle events for basic coordination. Handlers can return a promise and resolve/fail asynchronously
+* Phases include: `beforeLoadTests`, `beforeTests`, `beforeEachTest`, `cleanup`, `phaseStart`, `phaseEnd`
+
+[float]
+===== Kibana Services
+
+The Kibana functional tests define the vast majority of the actual functionality used by tests.
+
+**retry:**
+
+* Source: {blob}test/functional/services/retry.js[test/functional/services/retry.js]
+* Helpers for retrying operations
+* Popular methods:
+    * `retry.try(fn)` - execute `fn` in a loop until it succeeds or the default try timeout elapses
+    * `retry.tryForTime(ms, fn)` execute fn in a loop until it succeeds or `ms` milliseconds elapses
+
+**testSubjects:**
+
+* Source: {blob}test/functional/services/test_subjects.js[test/functional/services/test_subjects.js]
+* Test subjects are elements that are tagged specifically for selecting from tests
+* Use `testSubjects` over CSS selectors when possible
+* Usage:
+    * Tag your test subject with a `data-test-subj` attribute:
+
+["source","html"]
+-----------
+<div id="container”>
+  <button id="clickMe” data-test-subj=”containerButton” />
+</div>
+-----------
+
+* Click this button using the `testSubjects` helper:
+
+["source","js"]
+-----------
+await testSubjects.click(‘containerButton’);
+-----------
+
+* Popular methods:
+    * `testSubjects.find(testSubjectSelector)` - Find a test subject in the page; throw if it can't be found after some time
+    * `testSubjects.click(testSubjectSelector)` - Click a test subject in the page; throw if it can't be found after some time
+
+**find:**
+
+* Source: {blob}test/functional/services/find.js[test/functional/services/find.js]
+* Helpers for `remote.findBy*` methods that log and manage timeouts
+* Popular methods:
+    * `find.byCssSelector()`
+    * `find.allByCssSelector()`
+
+**kibanaServer:**
+
+* Source: {blob}test/functional/services/kibana_server/kibana_server.js[test/functional/services/kibana_server/kibana_server.js]
+* Helpers for interacting with Kibana's server
+* Commonly used methods:
+    * `kibanaServer.uiSettings.update()`
+    * `kibanaServer.version.get()`
+    * `kibanaServer.status.getOverallState()`
+
+**esArchiver:**
+
+* Source: {blob}test/functional/services/es_archiver.js[test/functional/services/es_archiver.js]
+* Load/unload archives created with the `esArchiver`
+* Popular methods:
+    * `esArchiver.load(name)`
+    * `esArchiver.loadIfNeeded(name)`
+    * `esArchiver.unload(name)`
+
+**docTable:**
+
+* Source: {blob}test/functional/services/doc_table.js[test/functional/services/doc_table.js]
+* Helpers for interacting with doc tables
+
+**pointSeriesVis:**
+
+* Source: {blob}test/functional/services/point_series_vis.js[test/functional/services/point_series_vis.js]
+* Helpers for interacting with point series visualizations
+
+**Low-level utilities:**
+
+* es
+    * Source: {blob}test/functional/services/es.js[test/functional/services/es.js]
+    * Elasticsearch client
+    * Higher level options: `kibanaServer.uiSettings` or `esArchiver`
+
+* remote
+    * Source: {blob}test/functional/services/remote/remote.js[test/functional/services/remote/remote.js]
+    * Instance of https://theintern.github.io/leadfoot/module-leadfoot_Command.html[Leadfoot's `Command]` class
+    * Responsible for all communication with the browser
+    * Higher level options: `testSubjects`, `find`, and `PageObjects.common`
+    * See the https://theintern.github.io/leadfoot/module-leadfoot_Command.html[leadfoot/Command API] for full API
+
+[float]
+===== Custom Services
+
+Services are intentionally generic. They can be literally anything (even nothing). Some services have helpers for interacting with a specific types of UI elements, like `pointSeriesVis`, and others are more foundational, like `log` or `config`. Whenever you want to provide some functionality in a reusable package, consider making a custom service.
+
+To create a custom service `somethingUseful`:
+
+* Create a `test/functional/services/something_useful.js` file that looks like this:
+
+["source","js"]
+-----------
+// Services are defined by Provider functions that receive the ServiceProviderAPI
+export function SomethingUsefulProvider({ getService }) {
+  const log = getService('log');
+
+  class SomethingUseful {
+    doSomething() {
+    }
+  }
+  return new SomethingUseful();
+}
+-----------
+
+* Re-export your provider from `services/index.js`
+
+* Import it into `src/functional/config.js` and add it to the services config:
+
+["source","js"]
+-----------
+import { SomethingUsefulProvider } from './services';
+
+export default function () {
+  return {
+    // … truncated ...
+    services: {
+      somethingUseful: SomethingUsefulProvider
+    }
+  }
+}
+-----------
+
+[float]
+==== PageObjects
+
+The purpose for each PageObject is pretty self-explanatory. The visualize PageObject provides helpers for interacting with the visualize app, dashboard is the same for the dashboard app, and so on.
+
+One exception is the "common" PageObject. A holdover from the intern implementation, the common PageObject is a collection of helpers useful across pages. Now that we have shareable services, and those services can be shared with other `FunctionalTestRunner` configurations, we will continue to move functionality out of the common PageObject and into services.
+
+Please add new methods to existing or new services rather than further expanding the CommonPage class.
+
+[float]
+==== Gotchas
+
+Remember that you can’t run an individual test in the file (`it` block) because the whole `describe` needs to be run in order. There should only be one top level `describe` in a file.
+
+[float]
+===== Functional Test Timing
+
+Another important gotcha is writing stable tests by being mindful of timing. All methods on `remote` run asynchronously. It’s better to write interactions that wait for changes on the UI to appear before moving onto the next step.
+
+For example, rather than writing an interaction that simply clicks a button, write an interaction with the a higher-level purpose in mind:
+
+Bad example: `PageObjects.app.clickButton()`
+
+["source","js"]
+-----------
+class AppPage {
+  // what can people who call this method expect from the
+  // UI after the promise resolves? Since the reaction to most
+  // clicks is asynchronous the behavior is dependant on timing
+  // and likely to cause test that fail unexpectedly
+  async clickButton () {
+    await testSubjects.click(‘menuButton’);
+  }
+}
+-----------
+
+Good example: `PageObjects.app.openMenu()`
+
+["source","js"]
+-----------
+class AppPage {
+  // unlike `clickButton()`, callers of `openMenu()` know
+  // the state that the UI will be in before they move on to
+  // the next step
+  async openMenu () {
+    await testSubjects.click(‘menuButton’);
+    await testSubjects.exists(‘menu’);
+  }
+}
+-----------
+
+Writing in this way will ensure your test timings are not flaky or based on assumptions about UI updates after interactions.
+
+[float]
+==== Debugging
+
+From the command line run:
+
+["source","shell"]
+-----------
+node --debug-brk --inspect scripts/functional_test_runner
+-----------
+
+This prints out a URL that you can visit in Chrome and debug your functional tests in the browser.
+
+You can also see additional logs in the terminal by running the `FunctionalTestRunner` with the `--debug` or `--verbose` flag. Add more logs with statements in your tests like
+
+["source","js"]
+-----------
+// load the log service
+const log = getService(‘log’);
+
+// log.debug only writes when using the `--debug` or `--verbose` flag.
+log.debug(‘done clicking menu’);
+-----------
+

docs/development/plugin-development.asciidoc

@@ -8,8 +8,11 @@ The Kibana plugin interfaces are in a state of constant development.  We cannot
 
 * <<development-plugin-resources>>
 * <<development-uiexports>>
+* <<development-plugin-functional-tests>>
 
 
 include::plugin/development-plugin-resources.asciidoc[]
 
 include::plugin/development-uiexports.asciidoc[]
+
+include::plugin/development-plugin-functional-tests.asciidoc[]

docs/development/plugin/development-plugin-functional-tests.asciidoc

@@ -0,0 +1,91 @@
+[[development-plugin-functional-tests]]
+=== Functional Tests for Plugins
+
+Plugins use the `FunctionalTestRunner` by running it out of the Kibana repo. Ensure that your Kibana Development Environment is setup properly before continuing.
+
+[float]
+==== Writing your own configuration
+
+Every project or plugin should have its own `FunctionalTestRunner` config file. Just like Kibana's, this config file will define all of the test files to load, providers for Services and PageObjects, as well as configuration options for certain services.
+
+To get started copy and paste this example to `test/functional/config.js`:
+
+["source","js"]
+-----------
+import { resolve } from 'path';
+import { MyServiceProvider } from './services/my_service';
+import { MyAppPageProvider } from './services/my_app_page;
+
+// allow overriding the default kibana directory
+// using the KIBANA_DIR environment variable
+const KIBANA_CONFIG_PATH = resolve(process.env.KIBANA_DIR || '../kibana', 'test/functional/config.js');
+
+// the default export of config files must be a config provider
+// that returns an object with the projects config values
+export default async function ({ readConfigFile }) {
+
+  // read the Kibana config file so that we can utilize some of
+  // its services and PageObjects
+  const kibanaConfig = await readConfigFile(KIBANA_CONFIG_PATH);
+
+  return {
+    // list paths to the files that contain your plugins tests
+    testFiles: [
+      resolve(__dirname, './my_test_file.js'),
+    ],
+
+    // define the name and providers for services that should be
+    // available to your tests. If you don't specify anything here
+    // only the built-in services will be avaliable
+    services: {
+      ...kibanaConfig.get('services'),
+      myService: MyServiceProvider,
+    },
+
+    // just like services, PageObjects are defined as a map of
+    // names to Providers. Merge in Kibana's or pick specific ones
+    pageObjects: {
+      management: kibanaConfig.get('pageObjects.management'),
+      myApp: MyAppPageProvider,
+    },
+
+    // the apps section defines the urls that
+    // `PageObjects.common.navigateTo(appKey)` will use.
+    // Merge urls for your plugin with the urls defined in
+    // Kibana's config in order to use this helper
+    apps: {
+      ...kibanaConfig.get('apps'),
+      myApp: {
+        pathname: '/app/my_app',
+      }
+    },
+
+    // choose where esArchiver should load archives from
+    esArchiver: {
+      directory: resolve(__dirname, './es_archives'),
+    },
+
+    // choose where screenshots should be saved
+    screenshots: {
+      directory: resolve(__dirname, './tmp/screenshots'),
+    }
+
+    // more settings, like timeouts, mochaOpts, etc are
+    // defined in the config schema. See {blob}src/functional_test_runner/lib/config/schema.js[src/functional_test_runner/lib/config/schema.js]
+  };
+}
+
+-----------
+
+From the root of your repo you should now be able to run the `FunctionalTestRunner` script from your plugin project.
+
+["source","shell"]
+-----------
+node ../kibana/scripts/functional_test_runner
+-----------
+
+[float]
+==== Using esArchiver
+
+We're working on documentation for this, but for now the best place to look is the original {pull}10359[pull request].
+

docs/index.asciidoc

@@ -16,6 +16,7 @@ release-state can be: released | prerelease | unreleased
 :issue:          {repo}issues/
 :pull:           {repo}pull/
 :commit:         {repo}commit/
+:blob:           {repo}blob/{branch}/
 :security:       https://www.elastic.co/community/security/
 
 

package.json

@@ -126,8 +126,8 @@
     "expose-loader": "0.7.0",
     "extract-text-webpack-plugin": "0.8.2",
     "file-loader": "0.8.4",
-    "font-awesome": "4.4.0",
     "flot-charts": "0.8.3",
+    "font-awesome": "4.4.0",
     "glob": "5.0.13",
     "glob-all": "3.0.1",
     "good-squeeze": "2.1.0",
@@ -209,8 +209,9 @@
     "chance": "1.0.6",
     "cheerio": "0.22.0",
     "chokidar": "1.6.0",
-    "chromedriver": "2.24.1",
+    "chromedriver": "2.28.0",
     "classnames": "2.2.5",
+    "digdug": "1.6.3",
     "enzyme": "2.7.0",
     "enzyme-to-json": "1.4.5",
     "eslint": "3.11.1",
@@ -239,7 +240,6 @@
     "html-loader": "0.4.3",
     "husky": "0.8.1",
     "image-diff": "1.6.0",
-    "intern": "3.2.3",
     "istanbul-instrumenter-loader": "0.1.3",
     "jest": "19.0.0",
     "jest-cli": "19.0.0",
@@ -252,6 +252,7 @@
     "karma-mocha": "0.2.0",
     "karma-safari-launcher": "0.1.1",
     "keymirror": "0.1.1",
+    "leadfoot": "1.7.1",
     "license-checker": "5.1.2",
     "load-grunt-config": "0.19.2",
     "makelogs": "3.2.3",

scripts/functional_test_runner.js

@@ -0,0 +1,2 @@
+require('../src/optimize/babel/register');
+require('../src/functional_test_runner/cli');

src/core_plugins/kibana/public/dashboard/listing/dashboard_listing.html

@@ -12,7 +12,10 @@
   </div>
 </kbn-top-nav>
 
-<div class="kuiViewContent kuiViewContent--constrainedWidth">
+<div
+  class="kuiViewContent kuiViewContent--constrainedWidth"
+  data-test-subj="dashboardLandingPage"
+>
   <!-- ControlledTable -->
   <div class="kuiViewContentItem kuiControlledTable kuiVerticalRhythm">
     <!-- ToolBar -->

src/es_archiver/actions/load.js

@@ -9,7 +9,7 @@ import {
   isGzip,
   createStats,
   prioritizeMappings,
-  getArchiveFiles,
+  readDirectory,
   createParseArchiveStreams,
   createCreateIndexStream,
   createIndexDocRecordsStream,
@@ -19,7 +19,7 @@ export async function loadAction({ name, skipExisting, client, dataDir, log }) {
   const inputDir = resolve(dataDir, name);
   const stats = createStats(name, log);
 
-  const files = prioritizeMappings(await getArchiveFiles(inputDir));
+  const files = prioritizeMappings(await readDirectory(inputDir));
   for (const filename of files) {
     log.info('[%s] Loading %j', name, filename);
 

src/es_archiver/actions/rebuild_all.js

@@ -1,7 +1,6 @@
 import { resolve } from 'path';
 import {
   rename,
-  readdir,
   createReadStream,
   createWriteStream
 } from 'fs';
@@ -14,18 +13,18 @@ import {
 
 import {
   prioritizeMappings,
-  getArchiveFiles,
+  readDirectory,
   isGzip,
   createParseArchiveStreams,
   createFormatArchiveStreams,
 } from '../lib';
 
 export async function rebuildAllAction({ dataDir, log }) {
-  const archiveNames = await fromNode(cb => readdir(dataDir, cb));
+  const archiveNames = await readDirectory(dataDir);
 
   for (const name of archiveNames) {
     const inputDir = resolve(dataDir, name);
-    const files = prioritizeMappings(await getArchiveFiles(inputDir));
+    const files = prioritizeMappings(await readDirectory(inputDir));
     for (const filename of files) {
       log.info('[%s] Rebuilding %j', name, filename);
 

src/es_archiver/actions/unload.js

@@ -9,7 +9,7 @@ import {
   isGzip,
   createStats,
   prioritizeMappings,
-  getArchiveFiles,
+  readDirectory,
   createParseArchiveStreams,
   createFilterRecordsStream,
   createDeleteIndexStream
@@ -19,7 +19,7 @@ export async function unloadAction({ name, client, dataDir, log }) {
   const inputDir = resolve(dataDir, name);
   const stats = createStats(name, log);
 
-  const files = prioritizeMappings(await getArchiveFiles(inputDir));
+  const files = prioritizeMappings(await readDirectory(inputDir));
   for (const filename of files) {
     log.info('[%s] Unloading indices from %j', name, filename);
 

src/es_archiver/cli.js

@@ -6,12 +6,14 @@
 
 import { resolve } from 'path';
 import { readFileSync } from 'fs';
+import { format as formatUrl } from 'url';
 
 import { Command } from 'commander';
 import elasticsearch from 'elasticsearch';
 
 import { EsArchiver } from './es_archiver';
-import { createLog } from './lib';
+import { createToolingLog } from '../utils';
+import { readConfigFile } from '../functional_test_runner';
 
 const cmd = new Command('node scripts/es_archiver');
 
@@ -20,6 +22,7 @@ cmd
   .option('--es-url [url]', 'url for elasticsearch')
   .option(`--dir [path]`, 'where archives are stored')
   .option('--verbose', 'turn on verbose logging')
+  .option('--config [path]', 'path to a functional test config file to use for default values')
   .on('--help', () => {
     console.log(readFileSync(resolve(__dirname, './cli_help.txt'), 'utf8'));
   });
@@ -49,9 +52,16 @@ if (missingCommand) {
 
 async function execute(operation, ...args) {
   try {
-    const log = createLog(cmd.verbose ? 'debug' : 'info');
+    const log = createToolingLog(cmd.verbose ? 'debug' : 'info');
     log.pipe(process.stdout);
 
+    if (cmd.config) {
+      // load default values from the specified config file
+      const config = await readConfigFile(log, resolve(cmd.config));
+      if (!cmd.esUrl) cmd.esUrl = formatUrl(config.get('servers.elasticsearch'));
+      if (!cmd.dir) cmd.dir = config.get('esArchiver.directory');
+    }
+
     // log and count all validation errors
     let errorCount = 0;
     const error = (msg) => {
@@ -61,10 +71,10 @@ async function execute(operation, ...args) {
 
     if (!operation) error('Missing or invalid command');
     if (!cmd.esUrl) {
-      error('You must specify either --es-url flag');
+      error('You must specify either --es-url or --config flags');
     }
     if (!cmd.dir) {
-      error('You must specify either --dir flag');
+      error('You must specify either --dir or --config flags');
     }
 
     // if there was a validation error display the help
@@ -84,7 +94,7 @@ async function execute(operation, ...args) {
       const esArchiver = new EsArchiver({
         log,
         client,
-        dataDir: resolve(cmd.dir)
+        dataDir: resolve(cmd.dir),
       });
       await esArchiver[operation](...args);
     } finally {

src/es_archiver/lib/__tests__/stats.js

@@ -2,9 +2,10 @@ import expect from 'expect.js';
 import { uniq } from 'lodash';
 import sinon from 'sinon';
 
-import { createStats, createLog } from '../';
+import { createStats } from '../';
 
 import {
+  createToolingLog,
   createConcatStream,
   createPromiseFromStreams
 } from '../../../utils';
@@ -47,14 +48,14 @@ function assertDeepClones(a, b) {
 describe('esArchiver: Stats', () => {
   describe('#skippedIndex(index)', () => {
     it('marks the index as skipped', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.skippedIndex('index-name');
       const indexStats = stats.toJSON()['index-name'];
       expect(indexStats).to.have.property('skipped', true);
     });
 
     it('logs that the index was skipped', async () => {
-      const log = createLog('debug');
+      const log = createToolingLog('debug');
       const stats = createStats('name', log);
       stats.skippedIndex('index-name');
       expect(await drain(log)).to.contain('Skipped');
@@ -63,13 +64,13 @@ describe('esArchiver: Stats', () => {
 
   describe('#deletedIndex(index)', () => {
     it('marks the index as deleted', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.deletedIndex('index-name');
       const indexStats = stats.toJSON()['index-name'];
       expect(indexStats).to.have.property('deleted', true);
     });
     it('logs that the index was deleted', async () => {
-      const log = createLog('debug');
+      const log = createToolingLog('debug');
       const stats = createStats('name', log);
       stats.deletedIndex('index-name');
       expect(await drain(log)).to.contain('Deleted');
@@ -78,20 +79,20 @@ describe('esArchiver: Stats', () => {
 
   describe('#createdIndex(index, [metadata])', () => {
     it('marks the index as created', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.createdIndex('index-name');
       const indexStats = stats.toJSON()['index-name'];
       expect(indexStats).to.have.property('created', true);
     });
     it('logs that the index was created', async () => {
-      const log = createLog('debug');
+      const log = createToolingLog('debug');
       const stats = createStats('name', log);
       stats.createdIndex('index-name');
       expect(await drain(log)).to.contain('Created');
     });
     describe('with metadata', () => {
       it('debug-logs each key from the metadata', async () => {
-        const log = createLog('debug');
+        const log = createToolingLog('debug');
         const stats = createStats('name', log);
         stats.createdIndex('index-name', {
           foo: 'bar'
@@ -103,7 +104,7 @@ describe('esArchiver: Stats', () => {
     });
     describe('without metadata', () => {
       it('no debug logging', async () => {
-        const log = createLog('debug');
+        const log = createToolingLog('debug');
         const stats = createStats('name', log);
         stats.createdIndex('index-name');
         const output = await drain(log);
@@ -114,20 +115,20 @@ describe('esArchiver: Stats', () => {
 
   describe('#archivedIndex(index, [metadata])', () => {
     it('marks the index as archived', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.archivedIndex('index-name');
       const indexStats = stats.toJSON()['index-name'];
       expect(indexStats).to.have.property('archived', true);
     });
     it('logs that the index was archived', async () => {
-      const log = createLog('debug');
+      const log = createToolingLog('debug');
       const stats = createStats('name', log);
       stats.archivedIndex('index-name');
       expect(await drain(log)).to.contain('Archived');
     });
     describe('with metadata', () => {
       it('debug-logs each key from the metadata', async () => {
-        const log = createLog('debug');
+        const log = createToolingLog('debug');
         const stats = createStats('name', log);
         stats.archivedIndex('index-name', {
           foo: 'bar'
@@ -139,7 +140,7 @@ describe('esArchiver: Stats', () => {
     });
     describe('without metadata', () => {
       it('no debug logging', async () => {
-        const log = createLog('debug');
+        const log = createToolingLog('debug');
         const stats = createStats('name', log);
         stats.archivedIndex('index-name');
         const output = await drain(log);
@@ -150,7 +151,7 @@ describe('esArchiver: Stats', () => {
 
   describe('#indexedDoc(index)', () => {
     it('increases the docs.indexed count for the index', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.indexedDoc('index-name');
       expect(stats.toJSON()['index-name'].docs.indexed).to.be(1);
       stats.indexedDoc('index-name');
@@ -161,7 +162,7 @@ describe('esArchiver: Stats', () => {
 
   describe('#archivedDoc(index)', () => {
     it('increases the docs.archived count for the index', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.archivedDoc('index-name');
       expect(stats.toJSON()['index-name'].docs.archived).to.be(1);
       stats.archivedDoc('index-name');
@@ -172,13 +173,13 @@ describe('esArchiver: Stats', () => {
 
   describe('#toJSON()', () => {
     it('returns the stats for all indexes', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.archivedIndex('index1');
       stats.archivedIndex('index2');
       expect(Object.keys(stats.toJSON())).to.eql(['index1', 'index2']);
     });
     it('returns a deep clone of the stats', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.archivedIndex('index1');
       stats.archivedIndex('index2');
       stats.deletedIndex('index3');
@@ -189,7 +190,7 @@ describe('esArchiver: Stats', () => {
 
   describe('#forEachIndex(fn)', () => {
     it('iterates a clone of the index stats', () => {
-      const stats = createStats('name', createLog());
+      const stats = createStats('name', createToolingLog());
       stats.archivedIndex('index1');
       stats.archivedIndex('index2');
       stats.deletedIndex('index3');

src/es_archiver/lib/archives/filenames.js

@@ -1,20 +1,9 @@
-import { fromNode } from 'bluebird';
-import { readdir } from 'fs';
 import { basename, extname } from 'path';
 
 export function isGzip(path) {
   return extname(path) === '.gz';
 }
 
-/**
- *  Gead the list of files in an archive.
- *
- *  @return {Promise} [description]
- */
-export async function getArchiveFiles(archiveDir) {
-  return await fromNode(cb => readdir(archiveDir, cb));
-}
-
 /**
  *  Check if a path is for a, potentially gzipped, mapping file
  *  @param  {String} path

src/es_archiver/lib/archives/index.js

@@ -1,6 +1,5 @@
 export {
   isGzip,
-  getArchiveFiles,
   prioritizeMappings,
 } from './filenames';
 

src/es_archiver/lib/archives/parse.js

@@ -12,6 +12,6 @@ export function createParseArchiveStreams({ gzip = false } = {}) {
   return [
     gzip ? createGunzip() : new PassThrough(),
     createSplitStream(RECORD_SEPARATOR),
-    createJsonParseStream()
+    createJsonParseStream(),
   ];
 }

src/es_archiver/lib/directory.js

@@ -0,0 +1,8 @@
+import { readdir } from 'fs';
+
+import { fromNode } from 'bluebird';
+
+export async function readDirectory(path) {
+  const allNames = await fromNode(cb => readdir(path, cb));
+  return allNames.filter(name => !name.startsWith('.'));
+}

src/es_archiver/lib/index.js

@@ -19,12 +19,11 @@ export {
 
 export {
   isGzip,
-  getArchiveFiles,
   prioritizeMappings,
   createParseArchiveStreams,
   createFormatArchiveStreams,
 } from './archives';
 
 export {
-  createLog
-} from './log';
+  readDirectory
+} from './directory';

src/es_archiver/lib/log/index.js

@@ -1 +0,0 @@
-export { createLog } from './log';

src/es_archiver/lib/log/log.js

@@ -1,39 +0,0 @@
-import { format } from 'util';
-import { PassThrough } from 'stream';
-
-import { createLogLevelFlags } from './log_levels';
-import { red, blue, brightBlack } from 'ansicolors';
-
-export function createLog(logLevel = 'silent') {
-  const logLevelFlags = createLogLevelFlags(logLevel);
-
-  function write(stream, ...args) {
-    format(...args).split('\n').forEach((line, i) => {
-      stream.write(`${i === 0 ? '' : '    '}${line}\n`);
-    });
-  }
-
-  class Log extends PassThrough {
-    debug(...args) {
-      if (!logLevelFlags.debug) return;
-      write(this, ' %s ', brightBlack('debg'), format(...args));
-    }
-
-    info(...args) {
-      if (!logLevelFlags.info) return;
-      write(this, ' %s ', blue('info'), format(...args));
-    }
-
-    error(err) {
-      if (!logLevelFlags.error) return;
-
-      if (typeof err !== 'string' && !(err instanceof Error)) {
-        err = new Error(`"${err}" thrown`);
-      }
-
-      write(this, '%s ', red('ERROR'), err.stack || err.message || err);
-    }
-  }
-
-  return new Log();
-}

src/fixtures/es_archives/visualize_source_filters/mappings.json

@@ -0,0 +1,76 @@
+{
+  "type": "index",
+  "value": {
+    "index": ".kibana",
+    "settings": {
+      "index": {
+        "number_of_shards": "1",
+        "number_of_replicas": "0"
+      }
+    },
+    "mappings": {
+      "config": {
+        "properties": {
+          "dateFormat:tz": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "defaultIndex": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          }
+        }
+      },
+      "index-pattern": {
+        "properties": {
+          "fields": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "sourceFilters": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "timeFieldName": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "title": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          }
+        }
+      }
+    }
+  }
+}

src/functional_test_runner/__tests__/fixtures/simple_project/config.js

@@ -0,0 +1,7 @@
+import { resolve } from 'path';
+
+export default () => ({
+  testFiles: [
+    resolve(__dirname, 'tests.js')
+  ]
+});

src/functional_test_runner/__tests__/fixtures/simple_project/tests.js

@@ -0,0 +1,18 @@
+import expect from 'expect.js';
+
+export default () => {
+  describe('app one', () => {
+    before(() => {
+      console.log('$BEFORE$');
+    });
+
+    it('$TESTNAME$', () => {
+      expect(1).to.be(1);
+      console.log('$INTEST$');
+    });
+
+    after(() => {
+      console.log('$AFTER$');
+    });
+  });
+};

src/functional_test_runner/__tests__/fixtures/with_es_archiver/archives/test_archive/mappings.json

@@ -0,0 +1,212 @@
+{
+  "type": "index",
+  "value": {
+    "index": ".kibana",
+    "settings": {
+      "index": {
+        "number_of_shards": "1",
+        "number_of_replicas": "0"
+      }
+    },
+    "mappings": {
+      "server": {
+        "properties": {
+          "uuid": {
+            "type": "keyword"
+          }
+        }
+      },
+      "search": {
+        "properties": {
+          "columns": {
+            "type": "text"
+          },
+          "description": {
+            "type": "text"
+          },
+          "hits": {
+            "type": "integer"
+          },
+          "kibanaSavedObjectMeta": {
+            "properties": {
+              "searchSourceJSON": {
+                "type": "text"
+              }
+            }
+          },
+          "sort": {
+            "type": "text"
+          },
+          "title": {
+            "type": "text"
+          },
+          "version": {
+            "type": "integer"
+          }
+        }
+      },
+      "config": {
+        "properties": {
+          "buildNum": {
+            "type": "keyword"
+          },
+          "dateFormat:tz": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "defaultIndex": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          }
+        }
+      },
+      "dashboard": {
+        "properties": {
+          "description": {
+            "type": "text"
+          },
+          "hits": {
+            "type": "integer"
+          },
+          "kibanaSavedObjectMeta": {
+            "properties": {
+              "searchSourceJSON": {
+                "type": "text"
+              }
+            }
+          },
+          "optionsJSON": {
+            "type": "text"
+          },
+          "panelsJSON": {
+            "type": "text"
+          },
+          "timeFrom": {
+            "type": "text"
+          },
+          "timeRestore": {
+            "type": "boolean"
+          },
+          "timeTo": {
+            "type": "text"
+          },
+          "title": {
+            "type": "text"
+          },
+          "uiStateJSON": {
+            "type": "text"
+          },
+          "version": {
+            "type": "integer"
+          }
+        }
+      },
+      "visualization": {
+        "properties": {
+          "description": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "kibanaSavedObjectMeta": {
+            "properties": {
+              "searchSourceJSON": {
+                "type": "text",
+                "fields": {
+                  "keyword": {
+                    "type": "keyword",
+                    "ignore_above": 256
+                  }
+                }
+              }
+            }
+          },
+          "title": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "uiStateJSON": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "version": {
+            "type": "integer"
+          },
+          "visState": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          }
+        }
+      },
+      "index-pattern": {
+        "properties": {
+          "fields": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "sourceFilters": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "timeFieldName": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          },
+          "title": {
+            "type": "text",
+            "fields": {
+              "keyword": {
+                "type": "keyword",
+                "ignore_above": 256
+              }
+            }
+          }
+        }
+      }
+    }
+  }
+}

src/functional_test_runner/__tests__/fixtures/with_es_archiver/config.js

@@ -0,0 +1,24 @@
+import { resolve } from 'path';
+
+export default () => ({
+  testFiles: [
+    resolve(__dirname, 'tests.js')
+  ],
+
+  esArchiver: {
+    directory: resolve(__dirname, 'archives')
+  },
+
+  servers: {
+    elasticsearch: {
+      protocol: 'http',
+      hostname: 'localhost',
+      port: 5700
+    },
+    kibana: {
+      protocol: 'http',
+      hostname: 'localhost',
+      port: 5701
+    }
+  }
+});

src/functional_test_runner/__tests__/fixtures/with_es_archiver/tests.js

@@ -0,0 +1,23 @@
+export default ({ getService }) => {
+  const esArchiver = getService('esArchiver');
+  const es = getService('es');
+  const log = getService('log');
+
+  describe('tests', () => {
+    before(async () => {
+      log.debug('before load()');
+      await esArchiver.load('test_archive');
+      log.debug('after load()');
+    });
+
+    it('loaded the archive', async () => {
+      log.debug('es aliases', await es.indices.getAlias());
+    });
+
+    after(async () => {
+      log.debug('before unload()');
+      await esArchiver.unload('test_archive');
+      log.debug('after unload()');
+    });
+  });
+};

src/functional_test_runner/__tests__/integration/basic.js

@@ -0,0 +1,20 @@
+import { spawnSync } from 'child_process';
+import { resolve } from 'path';
+
+import expect from 'expect.js';
+
+const SCRIPT = resolve(__dirname, '../../../../scripts/functional_test_runner.js');
+const BASIC_CONFIG = resolve(__dirname, '../fixtures/simple_project/config.js');
+
+describe('basic config file with a single app and test', function () {
+  this.timeout(60 * 1000);
+
+  it('runs and prints expected output', () => {
+    const proc = spawnSync(process.execPath, [SCRIPT, '--config', BASIC_CONFIG]);
+    const stdout = proc.stdout.toString('utf8');
+    expect(stdout).to.contain('$BEFORE$');
+    expect(stdout).to.contain('$TESTNAME$');
+    expect(stdout).to.contain('$INTEST$');
+    expect(stdout).to.contain('$AFTER$');
+  });
+});

src/functional_test_runner/__tests__/integration/with_es_archiver.js

@@ -0,0 +1,71 @@
+import { spawn } from 'child_process';
+import { resolve } from 'path';
+import { format as formatUrl } from 'url';
+
+import { readConfigFile } from '../../lib';
+import { createToolingLog, createReduceStream } from '../../../utils';
+import { startupEs, startupKibana } from '../lib';
+
+const SCRIPT = resolve(__dirname, '../../../../scripts/functional_test_runner.js');
+const CONFIG = resolve(__dirname, '../fixtures/with_es_archiver/config.js');
+
+describe('single test that uses esArchiver', function () {
+  this.timeout(60 * 1000);
+
+  let log;
+  const cleanupWork = [];
+
+  before(async () => {
+    log = createToolingLog('verbose', process.stdout);
+    log.indent(6);
+
+    const config = await readConfigFile(log, CONFIG);
+
+    log.info('starting elasticsearch');
+    log.indent(2);
+    const es = await startupEs({
+      log,
+      port: config.get('servers.elasticsearch.port'),
+      fresh: false
+    });
+    log.indent(-2);
+
+    log.info('starting kibana');
+    log.indent(2);
+    const kibana = await startupKibana({
+      port: config.get('servers.kibana.port'),
+      esUrl: formatUrl(config.get('servers.elasticsearch'))
+    });
+    log.indent(-2);
+
+    cleanupWork.push(() => es.shutdown());
+    cleanupWork.push(() => kibana.close());
+  });
+
+  it('test', async () => {
+    const proc = spawn(process.execPath, [SCRIPT, '--config', CONFIG], {
+      stdio: ['ignore', 'pipe', 'ignore']
+    });
+
+    const concatChunks = (acc, chunk) => `${acc}${chunk}`;
+    const concatStdout = proc.stdout.pipe(createReduceStream(concatChunks));
+
+    const [stdout] = await Promise.all([
+      new Promise((resolve, reject) => {
+        concatStdout.on('error', reject);
+        concatStdout.on('data', resolve); // reduce streams produce a single value, no need to wait for anything else
+      }),
+
+      new Promise((resolve, reject) => {
+        proc.on('error', reject);
+        proc.on('close', resolve);
+      })
+    ]);
+
+    log.debug(stdout.toString('utf8'));
+  });
+
+  after(() => {
+    return Promise.all(cleanupWork.splice(0).map(fn => fn()));
+  });
+});

src/functional_test_runner/__tests__/lib/es.js

@@ -0,0 +1,43 @@
+import { resolve } from 'path';
+
+import { once, merge } from 'lodash';
+import libesvm from 'libesvm';
+
+const VERSION = 'master';
+const DIRECTORY = resolve(__dirname, '../../../../esvm/functional_test_runner_tests');
+
+const createCluster = (options = {}) => {
+  return libesvm.createCluster(merge({
+    directory: DIRECTORY,
+    branch: VERSION,
+  }, options));
+};
+
+const install = once(async (fresh) => {
+  await createCluster({ fresh }).install();
+});
+
+export async function startupEs(opts) {
+  const {
+    port,
+    log,
+    fresh = true
+  } = opts;
+
+  await install({ fresh });
+  const cluster = createCluster({
+    config: {
+      http: {
+        port
+      }
+    }
+  });
+
+  cluster.on('log', (event) => {
+    const method = event.level.toLowerCase() === 'info' ? 'verbose' : 'debug';
+    log[method](`${event.level}: ${event.type} - ${event.message}`);
+  });
+
+  await cluster.start();
+  return cluster;
+}

src/functional_test_runner/__tests__/lib/index.js

@@ -0,0 +1,2 @@
+export { startupEs } from './es';
+export { startupKibana } from './kibana';

src/functional_test_runner/__tests__/lib/kibana.js

@@ -0,0 +1,25 @@
+import { resolve } from 'path';
+
+import { createServer } from '../../../../test/utils/kbn_server';
+
+export async function startupKibana({ port, esUrl }) {
+  const server = createServer({
+    server: {
+      port,
+      autoListen: true,
+    },
+
+    plugins: {
+      scanDirs: [
+        resolve(__dirname, '../../../core_plugins')
+      ],
+    },
+
+    elasticsearch: {
+      url: esUrl
+    }
+  });
+
+  await server.ready();
+  return server;
+}

src/functional_test_runner/cli.js

@@ -0,0 +1,82 @@
+import { resolve } from 'path';
+
+import { Command } from 'commander';
+
+import { createToolingLog } from '../utils';
+import { createFunctionalTestRunner } from './functional_test_runner';
+
+const cmd = new Command('node scripts/functional_test_runner');
+
+const resolveConfigPath = v => resolve(process.cwd(), v);
+const defaultConfigPath = resolveConfigPath('test/functional/config.js');
+
+cmd
+  .option('--config [path]', 'Path to a config file', resolveConfigPath, defaultConfigPath)
+  .option('--bail', 'stop tests after the first failure', false)
+  .option('--grep <pattern>', 'pattern used to select which tests to run')
+  .option('--verbose', 'Log everything', false)
+  .option('--quiet', 'Only log errors', false)
+  .option('--silent', 'Log nothing', false)
+  .option('--debug', 'Run in debug mode', false)
+  .parse(process.argv);
+
+if (!cmd.config) {
+  console.log('');
+  console.log('  --config is a required parameter');
+  console.log('');
+  process.exit(1);
+}
+
+let logLevel = 'info';
+if (cmd.silent) logLevel = 'silent';
+if (cmd.quiet) logLevel = 'error';
+if (cmd.debug) logLevel = 'debug';
+if (cmd.verbose) logLevel = 'verbose';
+
+const log = createToolingLog(logLevel);
+log.pipe(process.stdout);
+
+const functionalTestRunner = createFunctionalTestRunner({
+  log,
+  configFile: cmd.config,
+  configOverrides: {
+    mochaOpts: {
+      bail: cmd.bail,
+      grep: cmd.grep,
+    }
+  }
+});
+
+async function run() {
+  try {
+    const failureCount = await functionalTestRunner.run();
+    process.exitCode = failureCount ? 1 : 0;
+  } catch (err) {
+    await teardown(err);
+  } finally {
+    await teardown();
+  }
+}
+
+let teardownRun = false;
+async function teardown(err) {
+  if (teardownRun) return;
+
+  teardownRun = true;
+  if (err) {
+    log.indent(-log.indent());
+    log.error(err);
+    process.exitCode = 1;
+  }
+
+  try {
+    await functionalTestRunner.close();
+  } finally {
+    process.exit();
+  }
+}
+
+process.on('unhandledRejection', err => teardown(err));
+process.on('SIGTERM', () => teardown());
+process.on('SIGINT', () => teardown());
+run();

src/functional_test_runner/functional_test_runner.js

@@ -0,0 +1,64 @@
+import {
+  createLifecycle,
+  readConfigFile,
+  createProviderCollection,
+  setupMocha,
+  runTests,
+} from './lib';
+
+export function createFunctionalTestRunner({ log, configFile, configOverrides }) {
+  const lifecycle = createLifecycle();
+
+  lifecycle.on('phaseStart', name => {
+    log.verbose('starting %j lifecycle phase', name);
+  });
+
+  lifecycle.on('phaseEnd', name => {
+    log.verbose('ending %j lifecycle phase', name);
+  });
+
+  class FunctionalTestRunner {
+    async run() {
+      let runErrorOccurred = false;
+
+      try {
+        const config = await readConfigFile(log, configFile, configOverrides);
+        log.info('Config loaded');
+
+        const providers = createProviderCollection(lifecycle, log, config);
+        await providers.loadAll();
+
+        const mocha = await setupMocha(lifecycle, log, config, providers);
+        await lifecycle.trigger('beforeTests');
+        log.info('Starting tests');
+        return await runTests(lifecycle, log, mocha);
+
+      } catch (runError) {
+        runErrorOccurred = true;
+        throw runError;
+
+      } finally {
+        try {
+          await this.close();
+
+        } catch (closeError) {
+          if (runErrorOccurred) {
+            log.error('failed to close functional_test_runner');
+            log.error(closeError);
+          } else {
+            throw closeError;
+          }
+        }
+      }
+    }
+
+    async close() {
+      if (this._closed) return;
+
+      this._closed = true;
+      await lifecycle.trigger('cleanup');
+    }
+  }
+
+  return new FunctionalTestRunner();
+}

src/functional_test_runner/index.js

@@ -0,0 +1 @@
+export { createFunctionalTestRunner } from './functional_test_runner';

src/functional_test_runner/lib/config/config.js

@@ -0,0 +1,64 @@
+import { get, has, cloneDeep } from 'lodash';
+import toPath from 'lodash/internal/toPath';
+
+import { schema } from './schema';
+
+const $values = Symbol('values');
+
+export class Config {
+  constructor(settings) {
+    const { error, value } = schema.validate(settings);
+    if (error) throw error;
+    this[$values] = value;
+  }
+
+  has(key) {
+    function recursiveHasCheck(path, values, schema) {
+      if (!schema._inner) return false;
+
+      // normalize child and pattern checks so we can iterate the checks in a single loop
+      const checks = [].concat(
+        // match children first, they have priority
+        (schema._inner.children || []).map(child => ({
+          test: key => child.key === key,
+          schema: child.schema
+        })),
+        // match patterns on any key that doesn't match an explicit child
+        (schema._inner.patterns || []).map(pattern => ({
+          test: key => pattern.regex.test(key) && has(values, key),
+          schema: pattern.rule
+        }))
+      );
+
+      for (const check of checks) {
+        if (!check.test(path[0])) {
+          continue;
+        }
+
+        if (path.length > 1) {
+          return recursiveHasCheck(path.slice(1), get(values, path[0]), check.schema);
+        }
+
+        return true;
+      }
+
+      return false;
+    }
+
+    const path = toPath(key);
+    if (!path.length) return true;
+    return recursiveHasCheck(path, this[$values], schema);
+  }
+
+  get(key, defaultValue) {
+    if (!this.has(key)) {
+      throw new Error(`Unknown config key "${key}"`);
+    }
+
+    return cloneDeep(get(this[$values], key, defaultValue), (v) => {
+      if (typeof v === 'function') {
+        return v;
+      }
+    });
+  }
+}

src/functional_test_runner/lib/config/create_config.js

@@ -0,0 +1,28 @@
+import { defaultsDeep } from 'lodash';
+
+import { Config } from './config';
+
+export async function readConfigFile(log, configFile, settingOverrides) {
+  log.debug('Loading config file from %j', configFile);
+
+  const configModule = require(configFile);
+  const configProvider = configModule.__esModule
+    ? configModule.default
+    : configModule;
+
+  const settings = defaultsDeep(
+    {},
+    settingOverrides,
+    await configProvider({
+      log,
+
+      // give a version of the readConfigFile function to
+      // the config file that already has has the logger bound
+      readConfigFile: async (...args) => (
+        await readConfigFile(log, ...args)
+      )
+    })
+  );
+
+  return new Config(settings);
+}

src/functional_test_runner/lib/config/index.js

@@ -0,0 +1 @@
+export { readConfigFile } from './create_config';

src/functional_test_runner/lib/config/schema.js

@@ -0,0 +1,81 @@
+import Joi from 'joi';
+
+import { ConsoleReporterProvider } from '../reporters';
+
+// valid pattern for ID
+// enforced camel-case identifiers for consistency
+const ID_PATTERN = /^[a-zA-Z0-9_]+$/;
+const INSPECTING = process.execArgv.includes('--inspect');
+
+const urlPartsSchema = () => Joi.object().keys({
+  protocol: Joi.string().valid('http', 'https'),
+  hostname: Joi.string().hostname(),
+  port: Joi.number(),
+  auth: Joi.string().regex(/^[^:]+:.+$/, 'username and password seperated by a colon'),
+  username: Joi.string(),
+  password: Joi.string(),
+  pathname: Joi.string().regex(/^\//, 'start with a /'),
+  hash: Joi.string().regex(/^\//, 'start with a /')
+}).default();
+
+export const schema = Joi.object().keys({
+  testFiles: Joi.array().items(Joi.string()).required(),
+
+  services: Joi.object().pattern(
+    ID_PATTERN,
+    Joi.func().required()
+  ).default(),
+
+  pageObjects: Joi.object().pattern(
+    ID_PATTERN,
+    Joi.func().required()
+  ).default(),
+
+  timeouts: Joi.object().keys({
+    find: Joi.number().default(10000),
+    try: Joi.number().default(40000),
+    test: Joi.number().default(INSPECTING ? Infinity : 120000),
+    esRequestTimeout: Joi.number().default(30000),
+    kibanaStabilize: Joi.number().default(15000),
+    navigateStatusPageCheck: Joi.number().default(250),
+  }).default(),
+
+  mochaOpts: Joi.object().keys({
+    bail: Joi.boolean().default(false),
+    grep: Joi.string(),
+    slow: Joi.number().default(30000),
+    timeout: Joi.number().default(60000),
+    ui: Joi.string().default('bdd'),
+    reporterProvider: Joi.func().default(ConsoleReporterProvider),
+  }).default(),
+
+  users: Joi.object().pattern(
+    ID_PATTERN,
+    Joi.object().keys({
+      username: Joi.string().required(),
+      password: Joi.string().required(),
+    }).required()
+  ),
+
+  servers: Joi.object().keys({
+    webdriver: urlPartsSchema(),
+    kibana: urlPartsSchema(),
+    elasticsearch: urlPartsSchema(),
+  }).default(),
+
+  // definition of apps that work with `common.navigateToApp()`
+  apps: Joi.object().pattern(
+    ID_PATTERN,
+    urlPartsSchema()
+  ).default(),
+
+  // settings for the esArchiver module
+  esArchiver: Joi.object().keys({
+    directory: Joi.string().required()
+  }),
+
+  // settings for the screenshots module
+  screenshots: Joi.object().keys({
+    directory: Joi.string().required()
+  }),
+}).default();

src/functional_test_runner/lib/create_provider_collection.js

@@ -0,0 +1,31 @@
+import {
+  ProviderCollection,
+  readProviderSpec
+} from './providers';
+
+/**
+ *  Create a ProviderCollection that includes the Service
+ *  providers and PageObject providers from config, as well
+ *  providers for the default services, lifecycle, log, and
+ *  config
+ *
+ *  @param  {Lifecycle} lifecycle
+ *  @param  {ToolingLog} log
+ *  @param  {Config} config    [description]
+ *  @return {ProviderCollection}
+ */
+export function createProviderCollection(lifecycle, log, config) {
+  return new ProviderCollection([
+    ...readProviderSpec('Service', {
+      // base level services that functional_test_runner exposes
+      lifecycle: () => lifecycle,
+      log: () => log,
+      config: () => config,
+
+      ...config.get('services'),
+    }),
+    ...readProviderSpec('PageObject', {
+      ...config.get('pageObjects')
+    })
+  ]);
+}

src/functional_test_runner/lib/describe_nesting_validator.js

@@ -0,0 +1,74 @@
+/**
+ *  Creates an object that enables us to intercept all calls to mocha
+ *  interface functions `describe()`, `before()`, etc. and ensure that:
+ *
+ *    - all calls are made within a `describe()`
+ *    - there is only one top-level `describe()`
+ *
+ *  To do this we create a proxy to another object, `context`. Mocha
+ *  interfaces will assign all of their exposed methods on this Proxy
+ *  which will wrap all functions with checks for the above rules.
+ *
+ *  @return {any} the context that mocha-ui interfaces will assign to
+ */
+export function createDescribeNestingValidator(context) {
+  let describeCount = 0;
+  let describeLevel = 0;
+
+  function createContextProxy() {
+    return new Proxy(context, {
+      set(target, property, value) {
+        return Reflect.set(target, property, wrapContextAssignmentValue(property, value));
+      }
+    });
+  }
+
+  function wrapContextAssignmentValue(name, value) {
+    if (typeof value !== 'function') {
+      return value;
+    }
+
+    if (name === 'describe') {
+      return createDescribeProxy(value);
+    }
+
+    return createNonDescribeProxy(name, value);
+  }
+
+  function createDescribeProxy(describe) {
+    return new Proxy(describe, {
+      apply(target, thisArg, args) {
+        try {
+          if (describeCount > 0 && describeLevel === 0) {
+            throw new Error(`
+              Test files must only define a single top-level suite. Please ensure that
+              all calls to \`describe()\` are within a single \`describe()\` call in this file.
+            `);
+          }
+
+          describeCount += 1;
+          describeLevel += 1;
+          return Reflect.apply(describe, thisArg, args);
+        } finally {
+          describeLevel -= 1;
+        }
+      }
+    });
+  }
+
+  function createNonDescribeProxy(name, nonDescribe) {
+    return new Proxy(nonDescribe, {
+      apply(target, thisArg, args) {
+        if (describeCount === 0) {
+          throw new Error(`
+            All ${name}() calls in test files must be within a describe() call.
+          `);
+        }
+
+        return Reflect.apply(nonDescribe, thisArg, args);
+      }
+    });
+  }
+
+  return createContextProxy();
+}

src/functional_test_runner/lib/index.js

@@ -0,0 +1,5 @@
+export { createLifecycle } from './lifecycle';
+export { readConfigFile } from './config';
+export { createProviderCollection } from './create_provider_collection';
+export { setupMocha } from './setup_mocha';
+export { runTests } from './run_tests';

src/functional_test_runner/lib/lifecycle.js

@@ -0,0 +1,42 @@
+export function createLifecycle() {
+  const listeners = {
+    beforeLoadTests: [],
+    beforeTests: [],
+    beforeEachTest: [],
+    cleanup: [],
+    phaseStart: [],
+    phaseEnd: [],
+  };
+
+  class Lifecycle {
+    on(name, fn) {
+      if (!listeners[name]) {
+        throw new TypeError(`invalid lifecycle event "${name}"`);
+      }
+
+      listeners[name].push(fn);
+    }
+
+    async trigger(name, ...args) {
+      if (!listeners[name]) {
+        throw new TypeError(`invalid lifecycle event "${name}"`);
+      }
+
+      try {
+        if (name !== 'phaseStart' && name !== 'phaseEnd') {
+          await this.trigger('phaseStart', name);
+        }
+
+        await Promise.all(listeners[name].map(
+          async fn => await fn(...args)
+        ));
+      } finally {
+        if (name !== 'phaseStart' && name !== 'phaseEnd') {
+          await this.trigger('phaseEnd', name);
+        }
+      }
+    }
+  }
+
+  return new Lifecycle();
+}

src/functional_test_runner/lib/load_test_files.js

@@ -0,0 +1,60 @@
+import { isAbsolute } from 'path';
+
+import { loadTracer } from './load_tracer';
+import { createDescribeNestingValidator } from './describe_nesting_validator';
+
+/**
+ *  Load an array of test files into a mocha instance
+ *
+ *  @param  {Mocha} mocha
+ *  @param  {ToolingLog} log
+ *  @param  {ProviderCollection} providers
+ *  @param  {String} path
+ *  @return {undefined} - mutates mocha, no return value
+ */
+export const loadTestFiles = (mocha, log, providers, paths) => {
+  const innerLoadTestFile = (path) => {
+    if (typeof path !== 'string' || !isAbsolute(path)) {
+      throw new TypeError('loadTestFile() only accepts absolute paths');
+    }
+
+    loadTracer(path, `testFile[${path}]`, () => {
+      log.verbose('Loading test file %s', path);
+
+      const testModule = require(path);
+      const testProvider = testModule.__esModule
+        ? testModule.default
+        : testModule;
+
+      runTestProvider(testProvider, path); // eslint-disable-line
+    });
+  };
+
+  const runTestProvider = (provider, path) => {
+    if (typeof provider !== 'function') {
+      throw new Error(`Default export of test files must be a function, got ${provider}`);
+    }
+
+    loadTracer(provider, `testProvider[${path}]`, () => {
+      // mocha.suite hocus-pocus comes from: https://git.io/vDnXO
+
+      mocha.suite.emit('pre-require', createDescribeNestingValidator(global), path, mocha);
+
+      const returnVal = provider({
+        loadTestFile: innerLoadTestFile,
+        getService: providers.getService,
+        getPageObject: providers.getPageObject,
+        getPageObjects: providers.getPageObjects,
+      });
+
+      if (returnVal && typeof returnVal.then === 'function') {
+        throw new TypeError('Default export of test files must not be an async function');
+      }
+
+      mocha.suite.emit('require', returnVal, path, mocha);
+      mocha.suite.emit('post-require', global, path, mocha);
+    });
+  };
+
+  paths.forEach(innerLoadTestFile);
+};

src/functional_test_runner/lib/load_tracer.js

@@ -0,0 +1,46 @@
+const globalLoadPath = [];
+function getPath(startAt = 0) {
+  return globalLoadPath
+    .slice(startAt)
+    .map(step => step.descrption)
+    .join(' -> ');
+}
+
+function addPathToMessage(message, startAt) {
+  const path = getPath(startAt);
+  if (!path) return message;
+  return `${message} -- from ${path}`;
+}
+
+/**
+ *  Trace the path followed as dependencies are loaded and
+ *  check for circular dependencies at each step
+ *
+ *  @param  {Any} ident identity of this load step, === compaired
+ *                         to identities of previous steps to find circles
+ *  @param  {String} descrption description of this step
+ *  @param  {Function} load function that executes this step
+ *  @return {Any} the value produced by load()
+ */
+export function loadTracer(ident, descrption, load) {
+  const isCircular = globalLoadPath.find(step => step.ident === ident);
+  if (isCircular) {
+    throw new Error(addPathToMessage(`Circular reference to "${descrption}"`));
+  }
+
+  try {
+    globalLoadPath.unshift({ ident, descrption });
+    return load();
+  } catch (err) {
+    if (err.__fromLoadTracer) {
+      throw err;
+    }
+
+    const wrapped = new Error(addPathToMessage(`Failure to load ${descrption}`, 1));
+    wrapped.stack = `${wrapped.message}\n\n  Original Error: ${err.stack}`;
+    wrapped.__fromLoadTracer = true;
+    throw wrapped;
+  } finally {
+    globalLoadPath.shift();
+  }
+}

src/functional_test_runner/lib/providers/async_instance.js

@@ -0,0 +1,99 @@
+const createdInstanceProxies = new WeakSet();
+
+export const isAsyncInstance = val =>(
+  createdInstanceProxies.has(val)
+);
+
+export const createAsyncInstance = (type, name, promiseForValue) => {
+  let finalValue;
+
+  const initPromise = promiseForValue.then(v => finalValue = v);
+  const initFn = () => initPromise;
+
+  const assertReady = desc => {
+    if (!finalValue) {
+      throw new Error(`
+        ${type} \`${desc}\` is loaded asynchronously but isn't available yet. Either await the
+        promise returned from ${name}.init(), or move this access into a test hook
+        like \`before()\` or \`beforeEach()\`.
+      `);
+    }
+  };
+
+  const proxy = new Proxy({}, {
+    apply(target, context, args) {
+      assertReady(`${name}()`);
+      return Reflect.apply(finalValue, context, args);
+    },
+
+    construct(target, args, newTarget) {
+      assertReady(`new ${name}()`);
+      return Reflect.construct(finalValue, args, newTarget);
+    },
+
+    defineProperty(target, prop, descriptor) {
+      assertReady(`${name}.${prop}`);
+      return Reflect.defineProperty(finalValue, prop, descriptor);
+    },
+
+    deleteProperty(target, prop) {
+      assertReady(`${name}.${prop}`);
+      return Reflect.deleteProperty(finalValue, prop);
+    },
+
+    get(target, prop, receiver) {
+      if (prop === 'init') return initFn;
+
+      assertReady(`${name}.${prop}`);
+      return Reflect.get(finalValue, prop, receiver);
+    },
+
+    getOwnPropertyDescriptor(target, prop) {
+      assertReady(`${name}.${prop}`);
+      return Reflect.getOwnPropertyDescriptor(finalValue, prop);
+    },
+
+    getPrototypeOf() {
+      assertReady(`${name}`);
+      return Reflect.getPrototypeOf(finalValue);
+    },
+
+    has(target, prop) {
+      if (prop === 'init') return true;
+
+      assertReady(`${name}.${prop}`);
+      return Reflect.has(finalValue, prop);
+    },
+
+    isExtensible() {
+      assertReady(`${name}`);
+      return Reflect.isExtensible(finalValue);
+    },
+
+    ownKeys() {
+      assertReady(`${name}`);
+      return Reflect.ownKeys(finalValue);
+    },
+
+    preventExtensions() {
+      assertReady(`${name}`);
+      return Reflect.preventExtensions(finalValue);
+    },
+
+    set(target, prop, value, receiver) {
+      assertReady(`${name}.${prop}`);
+      return Reflect.set(finalValue, prop, value, receiver);
+    },
+
+    setPrototypeOf(target, prototype) {
+      assertReady(`${name}`);
+      return Reflect.setPrototypeOf(finalValue, prototype);
+    }
+  });
+
+  // add the created provider to the WeakMap so we can
+  // check for it later in `isAsyncProvider()`
+  createdInstanceProxies.add(proxy);
+
+  return proxy;
+};

src/functional_test_runner/lib/providers/index.js

@@ -0,0 +1,2 @@
+export { ProviderCollection } from './provider_collection';
+export { readProviderSpec } from './read_provider_spec';

src/functional_test_runner/lib/providers/provider_collection.js

@@ -0,0 +1,84 @@
+import { loadTracer } from '../load_tracer';
+import { createAsyncInstance, isAsyncInstance } from './async_instance';
+
+export class ProviderCollection {
+  constructor(providers) {
+    this._instances = new Map();
+    this._providers = providers;
+  }
+
+  getService = name => (
+    this._getInstance('Service', name)
+  )
+
+  getPageObject = name => (
+    this._getInstance('PageObject', name)
+  )
+
+  getPageObjects = names => {
+    const pageObjects = {};
+    names.forEach(name => pageObjects[name] = this.getPageObject(name));
+    return pageObjects;
+  }
+
+  loadExternalService(name, provider) {
+    return this._getInstance('Service', name, provider);
+  }
+
+  async loadAll() {
+    const asyncInitErrors = [];
+    await Promise.all(
+      this._providers.map(async ({ type, name }) => {
+        try {
+          const instance = this._getInstance(type, name);
+          if (isAsyncInstance(instance)) {
+            await instance.init();
+          }
+        } catch (err) {
+          asyncInitErrors.push(err);
+        }
+      })
+    );
+
+    if (asyncInitErrors.length) {
+      // just throw the first, it probably caused the others and if not they
+      // will show up once we fix the first, but creating an AggregateError or
+      // something seems like overkill
+      throw asyncInitErrors[0];
+    }
+  }
+
+  _getProvider(type, name) {
+    const providerDef = this._providers.find(p => p.type === type && p.name === name);
+    if (!providerDef) {
+      throw new Error(`Unknown ${type} "${name}"`);
+    }
+    return providerDef.fn;
+  }
+
+  _getInstance(type, name, provider = this._getProvider(type, name)) {
+    const instances = this._instances;
+
+    return loadTracer(provider, `${type}(${name})`, () => {
+      if (!provider) {
+        throw new Error(`Unknown ${type} "${name}"`);
+      }
+
+      if (!instances.has(provider)) {
+        let instance = provider({
+          getService: this.getService,
+          getPageObject: this.getPageObject,
+          getPageObjects: this.getPageObjects,
+        });
+
+        if (instance && typeof instance.then === 'function') {
+          instance = createAsyncInstance(type, name, instance);
+        }
+
+        instances.set(provider, instance);
+      }
+
+      return instances.get(provider);
+    });
+  }
+}

src/functional_test_runner/lib/providers/read_provider_spec.js

@@ -0,0 +1,9 @@
+export function readProviderSpec(type, providers) {
+  return Object.keys(providers).map(name => {
+    return {
+      type,
+      name,
+      fn: providers[name],
+    };
+  });
+}

src/functional_test_runner/lib/reporters/console_reporter/colors.js

@@ -0,0 +1,19 @@
+import { brightBlack, green, yellow, red, brightWhite, brightCyan } from 'ansicolors';
+
+export const suite = brightWhite;
+export const pending = brightCyan;
+export const pass = green;
+export const fail = red;
+
+export function speed(name, txt) {
+  switch (name) {
+    case 'fast':
+      return green(txt);
+    case 'medium':
+      return yellow(txt);
+    case 'slow':
+      return red(txt);
+    default:
+      return brightBlack(txt);
+  }
+}

src/functional_test_runner/lib/reporters/console_reporter/console_reporter.js

@@ -0,0 +1,122 @@
+import { format } from 'util';
+
+import Mocha from 'mocha';
+
+import * as colors from './colors';
+import * as symbols from './symbols';
+import { ms } from './ms';
+import { writeEpilogue } from './write_epilogue';
+
+export function ConsoleReporterProvider({ getService }) {
+  const log = getService('log');
+
+  return class MochaReporter extends Mocha.reporters.Base {
+    constructor(runner) {
+      super(runner);
+      runner.on('start', this.onStart);
+      runner.on('hook', this.onHookStart);
+      runner.on('hook end', this.onHookEnd);
+      runner.on('test', this.onTestStart);
+      runner.on('suite', this.onSuiteStart);
+      runner.on('pending', this.onPending);
+      runner.on('pass', this.onPass);
+      runner.on('fail', this.onFail);
+      runner.on('test end', this.onTestEnd);
+      runner.on('suite end', this.onSuiteEnd);
+      runner.on('end', this.onEnd);
+    }
+
+    onStart = () => {
+      log.write('');
+    }
+
+    onHookStart = hook => {
+      log.write('-> ' + colors.suite(hook.title));
+      log.indent(2);
+    }
+
+    onHookEnd = () => {
+      log.indent(-2);
+    }
+
+    onSuiteStart = suite => {
+      if (!suite.root) {
+        log.write('-: ' + colors.suite(suite.title));
+      }
+
+      log.indent(2);
+    }
+
+    onSuiteEnd = () => {
+      if (log.indent(-2) === '') {
+        log.write();
+      }
+    }
+
+    onTestStart = test => {
+      log.write(`-> ${test.title}`);
+      log.indent(2);
+    }
+
+    onTestEnd = () => {
+      log.indent(-2);
+    }
+
+    onPending = () => {
+      log.write('-> ' + colors.pending(test.title));
+    }
+
+    onPass = test => {
+
+      let time = '';
+      if (test.speed !== 'fast') {
+        time = colors.speed(test.speed, ` (${ms(test.duration)})`);
+      }
+
+      const pass = colors.pass(`${symbols.ok} pass`);
+      log.write(`- ${pass} ${time}`);
+    }
+
+    onFail = test => {
+      // NOTE: this is super gross
+      //
+      //  - I started by trying to extract the Base.list() logic from mocha
+      //    but it's a lot more complicated than this is horrible.
+      //  - In order to fix the numbering and indentation we monkey-patch
+      //    console.log and parse the logged output.
+      //
+      let output = '';
+      const realLog = console.log;
+      console.log = (...args) => output += `${format(...args)}\n`;
+      try {
+        Mocha.reporters.Base.list([test]);
+      } finally {
+        console.log = realLog;
+      }
+
+      log.indent(-2);
+      log.write(
+        `- ${symbols.err} ` +
+        colors.fail(`fail: "${test.fullTitle()}"`) +
+        '\n' +
+        output
+          .split('\n')
+          .slice(2) // drop the first two lines, (empty + test title)
+          .map(line => {
+            // move leading colors behind leading spaces
+            return line.replace(/^((?:\[.+m)+)(\s+)/, '$2$1');
+          })
+          .map(line => {
+            // shrink mocha's indentation
+            return line.replace(/^\s{5,5}/, ' ');
+          })
+          .join('\n')
+      );
+      log.indent(2);
+    }
+
+    onEnd = () => {
+      writeEpilogue(log, this.stats);
+    }
+  };
+}

src/functional_test_runner/lib/reporters/console_reporter/index.js

@@ -0,0 +1 @@
+export { ConsoleReporterProvider } from './console_reporter';

src/functional_test_runner/lib/reporters/console_reporter/ms.js

@@ -0,0 +1,28 @@
+import moment from 'moment';
+
+/**
+ * Format a milliseconds value as a string
+ *
+ * @param {number} val
+ * @return {string}
+ */
+export function ms(val) {
+  const duration = moment.duration(val);
+  if (duration.days() >= 1) {
+    return duration.days().toFixed(1) + 'd';
+  }
+
+  if (duration.hours() >= 1) {
+    return duration.hours().toFixed(1) + 'h';
+  }
+
+  if (duration.minutes() >= 1) {
+    return duration.minutes().toFixed(1) + 'm';
+  }
+
+  if (duration.seconds() >= 1) {
+    return duration.as('seconds').toFixed(1) + 's';
+  }
+
+  return val + 'ms';
+}

src/functional_test_runner/lib/reporters/console_reporter/symbols.js

@@ -0,0 +1,9 @@
+// originally extracted from mocha https://git.io/v1PGh
+
+export const ok = process.platform === 'win32'
+  ? '\u221A'
+  : '✓';
+
+export const err = process.platform === 'win32'
+  ? '\u00D7'
+  : '✖';

src/functional_test_runner/lib/reporters/console_reporter/write_epilogue.js

@@ -0,0 +1,33 @@
+import * as colors from './colors';
+import { ms } from './ms';
+
+export function writeEpilogue(log, stats) {
+  // header
+  log.write();
+
+  // passes
+  log.write(
+    `${colors.pass('%d passing')} (%s)`,
+    stats.passes || 0,
+    ms(stats.duration)
+  );
+
+  // pending
+  if (stats.pending) {
+    log.write(
+      colors.pending('%d pending'),
+      stats.pending
+    );
+  }
+
+  // failures
+  if (stats.failures) {
+    log.write(
+      '%d failing',
+      stats.failures
+    );
+  }
+
+  // footer
+  log.write();
+}

src/functional_test_runner/lib/reporters/index.js

@@ -0,0 +1 @@
+export { ConsoleReporterProvider } from './console_reporter';

src/functional_test_runner/lib/run_tests.js

@@ -0,0 +1,31 @@
+/**
+ *  Run the tests that have already been loaded into
+ *  mocha. aborts tests on 'cleanup' lifecycle runs
+ *
+ *  @param  {Lifecycle} lifecycle
+ *  @param  {ToolingLog} log
+ *  @param  {Mocha} mocha
+ *  @return {Promise<Number>} resolves to the number of test failures
+ */
+export async function runTests(lifecycle, log, mocha) {
+  let runComplete = false;
+  const runner = mocha.run(() => {
+    runComplete = true;
+  });
+
+  lifecycle.on('cleanup', () => {
+    if (!runComplete) runner.abort();
+  });
+
+  return new Promise((resolve) => {
+    const respond = () => resolve(runner.failures);
+
+    // if there are no tests, mocha.run() is sync
+    // and the 'end' event can't be listened to
+    if (runComplete) {
+      respond();
+    } else {
+      runner.on('end', respond);
+    }
+  });
+}

src/functional_test_runner/lib/setup_mocha.js

@@ -0,0 +1,32 @@
+import Mocha from 'mocha';
+
+import { loadTestFiles } from './load_test_files';
+
+/**
+ *  Instansiate mocha and load testfiles into it
+ *
+ *  @param  {Lifecycle} lifecycle
+ *  @param  {ToolingLog} log
+ *  @param  {Config} config
+ *  @param  {ProviderCollection} providers
+ *  @return {Promise<Mocha>}
+ */
+export async function setupMocha(lifecycle, log, config, providers) {
+  // configure mocha
+  const mocha = new Mocha({
+    timeout: config.get('timeouts.test'),
+    ...config.get('mochaOpts'),
+    reporter: await providers.loadExternalService(
+      'configured mocha reporter',
+      config.get('mochaOpts.reporterProvider')
+    )
+  });
+
+  // global beforeEach hook in root suite triggers before all others
+  mocha.suite.beforeEach('global before each', async () => {
+    await lifecycle.trigger('beforeEachTest');
+  });
+
+  loadTestFiles(mocha, log, providers, config.get('testFiles'));
+  return mocha;
+}

src/ui/public/url/modify_url.js

@@ -1,61 +1,2 @@
-import { parse as parseUrl, format as formatUrl } from 'url';
-
-/**
- *  Takes a URL and a function that takes the meaningful parts
- *  of the URL as a key-value object, modifies some or all of
- *  the parts, and returns the modified parts formatted again
- *  as a url.
- *
- *  Url Parts sent:
- *    - protocol
- *    - slashes (does the url have the //)
- *    - auth
- *    - hostname (just the name of the host, no port or auth information)
- *    - port
- *    - pathmame (the path after the hostname, no query or hash, starts
- *        with a slash if there was a path)
- *    - query (always an object, even when no query on original url)
- *    - hash
- *
- *  Why?
- *    - The default url library in node produces several conflicting
- *      properties on the "parsed" output. Modifying any of these might
- *      lead to the modifications being ignored (depending on which
- *      property was modified)
- *    - It's not always clear wither to use path/pathname, host/hostname,
- *      so this trys to add helpful constraints
- *
- *  @param  {String} url - the url to parse
- *  @param  {Function<Object|undefined>} block - a function that will modify the parsed url, or return a new one
- *  @return {String} the modified and reformatted url
- */
-export function modifyUrl(url, block) {
-  if (typeof block !== 'function') {
-    throw new TypeError('You must pass a block to define the modifications desired');
-  }
-
-  const parsed = parseUrl(url, true);
-
-  // copy over the most specific version of each
-  // property. By default, the parsed url includes
-  // several conflicting properties (like path and
-  // pathname + search, or search and query) and keeping
-  // track of which property is actually used when they
-  // are formatted is harder than necessary
-  const meaningfulParts = {
-    protocol: parsed.protocol,
-    slashes: parsed.slashes,
-    auth: parsed.auth,
-    hostname: parsed.hostname,
-    port: parsed.port,
-    pathname: parsed.pathname,
-    query: parsed.query || {},
-    hash: parsed.hash,
-  };
-
-  // the block modifies the meaningfulParts object, or returns a new one
-  const modifications = block(meaningfulParts) || meaningfulParts;
-
-  // format the modified/replaced meaningfulParts back into a url
-  return formatUrl(modifications);
-}
+// we select the modify_url directly so the other utils, which are not browser compatible, are not included
+export { modifyUrl } from '../../../utils/modify_url';

src/utils/index.js

@@ -17,3 +17,6 @@ export {
   createReduceStream,
   createSplitStream,
 } from './streams';
+
+export { modifyUrl } from './modify_url';
+export { createToolingLog } from './tooling_log';

src/utils/modify_url.js

@@ -0,0 +1,61 @@
+import { parse as parseUrl, format as formatUrl } from 'url';
+
+/**
+ *  Takes a URL and a function that takes the meaningful parts
+ *  of the URL as a key-value object, modifies some or all of
+ *  the parts, and returns the modified parts formatted again
+ *  as a url.
+ *
+ *  Url Parts sent:
+ *    - protocol
+ *    - slashes (does the url have the //)
+ *    - auth
+ *    - hostname (just the name of the host, no port or auth information)
+ *    - port
+ *    - pathmame (the path after the hostname, no query or hash, starts
+ *        with a slash if there was a path)
+ *    - query (always an object, even when no query on original url)
+ *    - hash
+ *
+ *  Why?
+ *    - The default url library in node produces several conflicting
+ *      properties on the "parsed" output. Modifying any of these might
+ *      lead to the modifications being ignored (depending on which
+ *      property was modified)
+ *    - It's not always clear wither to use path/pathname, host/hostname,
+ *      so this trys to add helpful constraints
+ *
+ *  @param  {String} url - the url to parse
+ *  @param  {Function<Object|undefined>} block - a function that will modify the parsed url, or return a new one
+ *  @return {String} the modified and reformatted url
+ */
+export function modifyUrl(url, block) {
+  if (typeof block !== 'function') {
+    throw new TypeError('You must pass a block to define the modifications desired');
+  }
+
+  const parsed = parseUrl(url, true);
+
+  // copy over the most specific version of each
+  // property. By default, the parsed url includes
+  // several conflicting properties (like path and
+  // pathname + search, or search and query) and keeping
+  // track of which property is actually used when they
+  // are formatted is harder than necessary
+  const meaningfulParts = {
+    protocol: parsed.protocol,
+    slashes: parsed.slashes,
+    auth: parsed.auth,
+    hostname: parsed.hostname,
+    port: parsed.port,
+    pathname: parsed.pathname,
+    query: parsed.query || {},
+    hash: parsed.hash,
+  };
+
+  // the block modifies the meaningfulParts object, or returns a new one
+  const modifications = block(meaningfulParts) || meaningfulParts;
+
+  // format the modified/replaced meaningfulParts back into a url
+  return formatUrl(modifications);
+}

src/utils/tooling_log/__tests__/log.js

@@ -4,13 +4,13 @@ import Chance from 'chance';
 import {
   createConcatStream,
   createPromiseFromStreams
-} from '../../../../utils';
+} from '../../streams';
 
-import { createLog } from '../';
+import { createToolingLog } from '../';
 
 const chance = new Chance();
 const capture = (level, block) => {
-  const log = createLog(level);
+  const log = createToolingLog(level);
   block(log);
   log.end();
   return createPromiseFromStreams([ log, createConcatStream('') ]);
@@ -34,9 +34,9 @@ const somethingTest = (logLevel, method) => {
   });
 };
 
-describe('esArchiver: createLog(logLevel, output)', () => {
+describe('utils: createToolingLog(logLevel, output)', () => {
   it('is a readable stream', async () => {
-    const log = createLog('debug');
+    const log = createToolingLog('debug');
     log.info('Foo');
     log.info('Bar');
     log.info('Baz');
@@ -79,7 +79,7 @@ describe('esArchiver: createLog(logLevel, output)', () => {
         // by specifying a long length
         const level = chance.word({ length: 10 });
 
-        expect(() => createLog(level))
+        expect(() => createToolingLog(level))
           .to.throwError(level);
       });
     });

src/utils/tooling_log/__tests__/log_levels.js

@@ -10,8 +10,10 @@ describe('createLogLevelFlags()', () => {
       expect(createLogLevelFlags('silent')).to.eql({
         silent: true,
         error: false,
+        warning: false,
         info: false,
         debug: false,
+        verbose: false,
       });
     });
   });
@@ -21,8 +23,23 @@ describe('createLogLevelFlags()', () => {
       expect(createLogLevelFlags('error')).to.eql({
         silent: true,
         error: true,
+        warning: false,
         info: false,
         debug: false,
+        verbose: false,
+      });
+    });
+  });
+
+  describe('logLevel=warning', () => {
+    it('produces correct map', () => {
+      expect(createLogLevelFlags('warning')).to.eql({
+        silent: true,
+        error: true,
+        warning: true,
+        info: false,
+        debug: false,
+        verbose: false,
       });
     });
   });
@@ -32,8 +49,10 @@ describe('createLogLevelFlags()', () => {
       expect(createLogLevelFlags('info')).to.eql({
         silent: true,
         error: true,
+        warning: true,
         info: true,
         debug: false,
+        verbose: false,
       });
     });
   });
@@ -43,8 +62,23 @@ describe('createLogLevelFlags()', () => {
       expect(createLogLevelFlags('debug')).to.eql({
         silent: true,
         error: true,
+        warning: true,
         info: true,
         debug: true,
+        verbose: false,
+      });
+    });
+  });
+
+  describe('logLevel=verbose', () => {
+    it('produces correct map', () => {
+      expect(createLogLevelFlags('verbose')).to.eql({
+        silent: true,
+        error: true,
+        warning: true,
+        info: true,
+        debug: true,
+        verbose: true,
       });
     });
   });

src/utils/tooling_log/index.js

@@ -0,0 +1 @@
+export { createToolingLog } from './tooling_log';

src/utils/tooling_log/log_levels.js

@@ -2,8 +2,10 @@
 const LEVELS = [
   'silent',
   'error',
+  'warning',
   'info',
   'debug',
+  'verbose',
 ];
 
 export function createLogLevelFlags(levelLimit) {

src/utils/tooling_log/tooling_log.js

@@ -0,0 +1,59 @@
+import { format } from 'util';
+import { PassThrough } from 'stream';
+
+import { createLogLevelFlags } from './log_levels';
+import { magenta, yellow, red, blue, brightBlack } from 'ansicolors';
+
+export function createToolingLog(logLevel = 'silent') {
+  const logLevelFlags = createLogLevelFlags(logLevel);
+
+  let indentString = '';
+
+  class ToolingLog extends PassThrough {
+    verbose(...args) {
+      if (!logLevelFlags.verbose) return;
+      this.write(' %s ', magenta('sill'), format(...args));
+    }
+
+    debug(...args) {
+      if (!logLevelFlags.debug) return;
+      this.write(' %s ', brightBlack('debg'), format(...args));
+    }
+
+    info(...args) {
+      if (!logLevelFlags.info) return;
+      this.write(' %s ', blue('info'), format(...args));
+    }
+
+    warning(...args) {
+      if (!logLevelFlags.warning) return;
+      this.write(' %s ', yellow('warn'), format(...args));
+    }
+
+    error(err) {
+      if (!logLevelFlags.error) return;
+
+      if (typeof err !== 'string' && !(err instanceof Error)) {
+        err = new Error(`"${err}" thrown`);
+      }
+
+      this.write('%s ', red('ERROR'), err.stack || err.message || err);
+    }
+
+    indent(delta = 0) {
+      const width = Math.max(0, indentString.length + delta);
+      indentString = ' '.repeat(width);
+      return indentString.length;
+    }
+
+    write(...args) {
+      format(...args).split('\n').forEach((line, i) => {
+        const subLineIndent = i === 0 ? '' : '       ';
+        const indent = !indentString ? '' : indentString.slice(0, -1) + (i === 0 && line[0] === '-' ? '└' : '│');
+        super.write(`${indent}${subLineIndent}${line}\n`);
+      });
+    }
+  }
+
+  return new ToolingLog();
+}

tasks/config/copy.js

@@ -13,6 +13,7 @@ module.exports = function () {
         '!src/cli/cluster/**',
         '!src/ui_framework/doc_site/**',
         '!src/es_archiver/**',
+        '!src/functional_test_runner/**',
         'bin/**',
         'ui_framework/components/**',
         'ui_framework/services/**',

tasks/config/intern.js

@@ -1,26 +0,0 @@
-module.exports = function (grunt) {
-  return {
-    options: {
-      runType: 'runner',
-      config: 'test/intern',
-      bail: true,
-      reporters: ['Console'],
-      grep: grunt.option('grep'),
-      functionalSuites: grunt.option('functionalSuites'),
-      appSuites: grunt.option('appSuites')
-    },
-    dev: {},
-    api: {
-      options: {
-        runType: 'client',
-        config: 'test/intern_api'
-      }
-    },
-    visualRegression: {
-      options: {
-        runType: 'runner',
-        config: 'test/intern_visual_regression'
-      }
-    }
-  };
-};

tasks/config/simplemocha.js

@@ -1,5 +1,3 @@
-require('../../test/mocha_setup');
-
 module.exports = {
   options: {
     timeout: 10000,
@@ -10,11 +8,18 @@ module.exports = {
   },
   all: {
     src: [
+      'test/mocha_setup.js',
       'test/**/__tests__/**/*.js',
       'src/**/__tests__/**/*.js',
       'test/fixtures/__tests__/*.js',
       '!src/**/public/**',
       '!**/_*.js'
     ]
+  },
+  api: {
+    src: [
+      'test/mocha_setup.js',
+      'test/unit/**/*.js'
+    ]
   }
 };

tasks/functional_test_runner.js

@@ -0,0 +1,31 @@
+import { resolve } from 'path';
+
+import { createFunctionalTestRunner } from '../src/functional_test_runner';
+import { createToolingLog } from '../src/utils';
+
+export default function (grunt) {
+  grunt.registerTask('functionalTestRunner', function () {
+    const log = createToolingLog('debug');
+    log.pipe(process.stdout);
+
+    const functionalTestRunner = createFunctionalTestRunner({
+      log,
+      configFile: resolve(__dirname, '../test/functional/config.js'),
+    });
+
+    const callback = this.async();
+    functionalTestRunner.run()
+      .then(failureCount => {
+        if (failureCount) {
+          grunt.fail.error(`${failureCount} test failures`);
+          return;
+        }
+
+        callback();
+      })
+      .catch(err => {
+        grunt.log.error(err.stack);
+        callback(err);
+      });
+  });
+}

tasks/test.js

@@ -3,16 +3,6 @@ import _, { keys } from 'lodash';
 import visualRegression from '../utilities/visual_regression';
 
 module.exports = function (grunt) {
-  grunt.registerTask('test:visualRegression', [
-    'intern:visualRegression:takeScreenshots',
-    'test:visualRegression:buildGallery'
-  ]);
-
-  grunt.registerTask('test:visualRegression:takeScreenshots', [
-    'clean:screenshots',
-    'intern:visualRegression'
-  ]);
-
   grunt.registerTask(
     'test:visualRegression:buildGallery',
     'Compare screenshots and generate diff images.',
@@ -68,11 +58,9 @@ module.exports = function (grunt) {
     'checkPlugins',
     'esvm:ui',
     'run:testUIServer',
-    'run:chromeDriver',
     'clean:screenshots',
-    'intern:dev',
+    'functionalTestRunner',
     'esvm_shutdown:ui',
-    'stop:chromeDriver',
     'stop:testUIServer'
   ]);
 
@@ -85,14 +73,13 @@ module.exports = function (grunt) {
   grunt.registerTask('test:ui:runner', [
     'checkPlugins',
     'clean:screenshots',
-    'run:devChromeDriver',
-    'intern:dev'
+    'functionalTestRunner'
   ]);
 
   grunt.registerTask('test:api', [
     'esvm:ui',
     'run:apiTestServer',
-    'intern:api',
+    'simplemocha:api',
     'esvm_shutdown:ui',
     'stop:apiTestServer'
   ]);
@@ -103,7 +90,7 @@ module.exports = function (grunt) {
   ]);
 
   grunt.registerTask('test:api:runner', [
-    'intern:api'
+    'simplemocha:api'
   ]);
 
   grunt.registerTask('test', subTask => {

test/functional/apps/console/_console.js

@@ -1,9 +1,5 @@
-
 import expect from 'expect.js';
 
-import PageObjects from '../../../support/page_objects';
-import { bdd } from '../../../support';
-
 const DEFAULT_REQUEST = `
 
 GET _search
@@ -15,54 +11,60 @@ GET _search
 
 `.trim();
 
-bdd.describe('console app', function describeIndexTests() {
-  bdd.before(function () {
-    PageObjects.common.debug('navigateTo console');
-    return PageObjects.common.navigateToApp('console');
-  });
+export default function ({ getService, getPageObjects }) {
+  const retry = getService('retry');
+  const log = getService('log');
+  const PageObjects = getPageObjects(['common', 'console']);
 
-  bdd.it('should show the default request', function () {
-    PageObjects.common.saveScreenshot('Console-help-expanded');
-    // collapse the help pane because we only get the VISIBLE TEXT, not the part that is scrolled
-    return PageObjects.console.collapseHelp()
-    .then(function () {
-      PageObjects.common.saveScreenshot('Console-help-collapsed');
-      return PageObjects.common.try(function () {
-        return PageObjects.console.getRequest()
-        .then(function (actualRequest) {
-          expect(actualRequest.trim()).to.eql(DEFAULT_REQUEST);
+  describe('console app', function describeIndexTests() {
+    before(function () {
+      log.debug('navigateTo console');
+      return PageObjects.common.navigateToApp('console');
+    });
+
+    it('should show the default request', function () {
+      PageObjects.common.saveScreenshot('Console-help-expanded');
+      // collapse the help pane because we only get the VISIBLE TEXT, not the part that is scrolled
+      return PageObjects.console.collapseHelp()
+      .then(function () {
+        PageObjects.common.saveScreenshot('Console-help-collapsed');
+        return retry.try(function () {
+          return PageObjects.console.getRequest()
+          .then(function (actualRequest) {
+            expect(actualRequest.trim()).to.eql(DEFAULT_REQUEST);
+          });
         });
       });
     });
-  });
 
-  bdd.it('default request response should contain .kibana' , function () {
-    const expectedResponseContains = '"_index": ".kibana",';
+    it('default request response should contain .kibana' , function () {
+      const expectedResponseContains = '"_index": ".kibana",';
 
-    return PageObjects.console.clickPlay()
-    .then(function () {
-      PageObjects.common.saveScreenshot('Console-default-request');
-      return PageObjects.common.try(function () {
-        return PageObjects.console.getResponse()
-        .then(function (actualResponse) {
-          PageObjects.common.debug(actualResponse);
-          expect(actualResponse).to.contain(expectedResponseContains);
+      return PageObjects.console.clickPlay()
+      .then(function () {
+        PageObjects.common.saveScreenshot('Console-default-request');
+        return retry.try(function () {
+          return PageObjects.console.getResponse()
+          .then(function (actualResponse) {
+            log.debug(actualResponse);
+            expect(actualResponse).to.contain(expectedResponseContains);
+          });
         });
       });
     });
-  });
 
-  bdd.it('settings should allow changing the text size', async function () {
-    await PageObjects.console.setFontSizeSetting(20);
-    await PageObjects.common.try(async () => {
-      // the settings are not applied synchronously, so we retry for a time
-      expect(await PageObjects.console.getRequestFontSize()).to.be('20px');
-    });
+    it('settings should allow changing the text size', async function () {
+      await PageObjects.console.setFontSizeSetting(20);
+      await retry.try(async () => {
+        // the settings are not applied synchronously, so we retry for a time
+        expect(await PageObjects.console.getRequestFontSize()).to.be('20px');
+      });
 
-    await PageObjects.console.setFontSizeSetting(24);
-    await PageObjects.common.try(async () => {
-      // the settings are not applied synchronously, so we retry for a time
-      expect(await PageObjects.console.getRequestFontSize()).to.be('24px');
+      await PageObjects.console.setFontSizeSetting(24);
+      await retry.try(async () => {
+        // the settings are not applied synchronously, so we retry for a time
+        expect(await PageObjects.console.getRequestFontSize()).to.be('24px');
+      });
     });
   });
-});
+}

test/functional/apps/console/index.js

@@ -1,16 +1,14 @@
+export default function ({ getService, loadTestFile }) {
+  const config = getService('config');
+  const remote = getService('remote');
 
-import {
-  bdd,
-  remote,
-  defaultTimeout,
-} from '../../../support';
+  describe('console app', function () {
+    this.timeout(config.get('timeouts.find'));
 
-bdd.describe('console app', function () {
-  this.timeout = defaultTimeout;
+    before(async function () {
+      await remote.setWindowSize(1200,800);
+    });
 
-  bdd.before(function () {
-    return remote.setWindowSize(1200,800);
+    loadTestFile(require.resolve('./_console'));
   });
-
-  require('./_console');
-});
+}

test/functional/apps/context/_discover_navigation.js

@@ -1,54 +1,57 @@
 import expect from 'expect.js';
 
-import { bdd } from '../../../support';
-import PageObjects from '../../../support/page_objects';
-
-
 const TEST_DISCOVER_START_TIME = '2015-09-19 06:31:44.000';
 const TEST_DISCOVER_END_TIME = '2015-09-23 18:31:44.000';
 const TEST_COLUMN_NAMES = ['@message'];
 
-bdd.describe('context link in discover', function contextSize() {
-  bdd.before(async function() {
-    await PageObjects.common.navigateToApp('discover');
-    await PageObjects.header.setAbsoluteRange(TEST_DISCOVER_START_TIME, TEST_DISCOVER_END_TIME);
-    await Promise.all(TEST_COLUMN_NAMES.map((columnName) => (
-      PageObjects.discover.clickFieldListItemAdd(columnName)
-    )));
-  });
+export default function ({ getService, getPageObjects }) {
+  const retry = getService('retry');
+  const docTable = getService('docTable');
+  const PageObjects = getPageObjects(['common', 'header', 'discover']);
 
-  bdd.it('should open the context view with the selected document as anchor', async function () {
-    const discoverDocTable = await PageObjects.docTable.getTable();
-    const firstRow = (await PageObjects.docTable.getBodyRows(discoverDocTable))[0];
-    const firstTimestamp = await (await PageObjects.docTable.getFields(firstRow))[0]
-      .getVisibleText();
-
-    // add a column in Discover
-    await (await PageObjects.docTable.getRowExpandToggle(firstRow)).click();
-    const firstDetailsRow = (await PageObjects.docTable.getDetailsRows(discoverDocTable))[0];
-    await (await PageObjects.docTable.getRowActions(firstDetailsRow))[0].click();
-
-    // check the column in the Context View
-    await PageObjects.common.try(async () => {
-      const contextDocTable = await PageObjects.docTable.getTable();
-      const anchorRow = await PageObjects.docTable.getAnchorRow(contextDocTable);
-      const anchorTimestamp = await (await PageObjects.docTable.getFields(anchorRow))[0]
-        .getVisibleText();
-      expect(anchorTimestamp).to.equal(firstTimestamp);
-    });
-  });
-
-  bdd.it('should open the context view with the same columns', async function () {
-    const docTable = await PageObjects.docTable.getTable();
-    await PageObjects.common.try(async () => {
-      const headerFields = await PageObjects.docTable.getHeaderFields(docTable);
-      const columnNames = await Promise.all(headerFields.map((headerField) => (
-        headerField.getVisibleText()
+  describe('context link in discover', function contextSize() {
+    before(async function() {
+      await PageObjects.common.navigateToApp('discover');
+      await PageObjects.header.setAbsoluteRange(TEST_DISCOVER_START_TIME, TEST_DISCOVER_END_TIME);
+      await Promise.all(TEST_COLUMN_NAMES.map((columnName) => (
+        PageObjects.discover.clickFieldListItemAdd(columnName)
       )));
-      expect(columnNames).to.eql([
-        'Time',
-        ...TEST_COLUMN_NAMES,
-      ]);
+    });
+
+    it('should open the context view with the selected document as anchor', async function () {
+      const discoverDocTable = await docTable.getTable();
+      const firstRow = (await docTable.getBodyRows(discoverDocTable))[0];
+      const firstTimestamp = await (await docTable.getFields(firstRow))[0]
+        .getVisibleText();
+
+      // add a column in Discover
+      await (await docTable.getRowExpandToggle(firstRow)).click();
+      const firstDetailsRow = (await docTable.getDetailsRows(discoverDocTable))[0];
+      await (await docTable.getRowActions(firstDetailsRow))[0].click();
+
+      // check the column in the Context View
+      await retry.try(async () => {
+        const contextDocTable = await docTable.getTable();
+        const anchorRow = await docTable.getAnchorRow(contextDocTable);
+        const anchorTimestamp = await (await docTable.getFields(anchorRow))[0]
+          .getVisibleText();
+        expect(anchorTimestamp).to.equal(firstTimestamp);
+      });
+    });
+
+    it('should open the context view with the same columns', async function () {
+      const table = await docTable.getTable();
+      await retry.try(async () => {
+        const headerFields = await docTable.getHeaderFields(table);
+        const columnNames = await Promise.all(headerFields.map((headerField) => (
+          headerField.getVisibleText()
+        )));
+        expect(columnNames).to.eql([
+          'Time',
+          ...TEST_COLUMN_NAMES,
+        ]);
+      });
     });
   });
-});
+
+}

test/functional/apps/context/_size.js

@@ -1,63 +1,68 @@
 import expect from 'expect.js';
 
-import { bdd, esClient } from '../../../support';
-import PageObjects from '../../../support/page_objects';
-
-
 const TEST_INDEX_PATTERN = 'logstash-*';
 const TEST_ANCHOR_TYPE = 'apache';
 const TEST_ANCHOR_ID = 'AU_x3_BrGFA8no6QjjaI';
 const TEST_DEFAULT_CONTEXT_SIZE = 7;
 const TEST_STEP_SIZE = 3;
 
-bdd.describe('context size', function contextSize() {
-  bdd.before(async function() {
-    await esClient.updateConfigDoc({
-      'context:defaultSize': `${TEST_DEFAULT_CONTEXT_SIZE}`,
-      'context:step': `${TEST_STEP_SIZE}`,
+export default function ({ getService, getPageObjects }) {
+  const remote = getService('remote');
+  const kibanaServer = getService('kibanaServer');
+  const retry = getService('retry');
+  const docTable = getService('docTable');
+  const PageObjects = getPageObjects(['context']);
+
+  describe('context size', function contextSize() {
+    before(async function() {
+      await kibanaServer.uiSettings.update({
+        'context:defaultSize': `${TEST_DEFAULT_CONTEXT_SIZE}`,
+        'context:step': `${TEST_STEP_SIZE}`,
+      });
+    });
+
+    it('should default to the `context:defaultSize` setting', async function () {
+      await PageObjects.context.navigateTo(TEST_INDEX_PATTERN, TEST_ANCHOR_TYPE, TEST_ANCHOR_ID);
+
+      const table = await docTable.getTable();
+      await retry.try(async function () {
+        expect(await docTable.getBodyRows(table)).to.have.length(2 * TEST_DEFAULT_CONTEXT_SIZE + 1);
+      });
+      await retry.try(async function() {
+        const predecessorCountPicker = await PageObjects.context.getPredecessorCountPicker();
+        expect(await predecessorCountPicker.getProperty('value')).to.equal(`${TEST_DEFAULT_CONTEXT_SIZE}`);
+      });
+      await retry.try(async function() {
+        const successorCountPicker = await PageObjects.context.getSuccessorCountPicker();
+        expect(await successorCountPicker.getProperty('value')).to.equal(`${TEST_DEFAULT_CONTEXT_SIZE}`);
+      });
+    });
+
+    it('should increase according to the `context:step` setting when clicking the `load newer` button', async function() {
+      await PageObjects.context.navigateTo(TEST_INDEX_PATTERN, TEST_ANCHOR_TYPE, TEST_ANCHOR_ID);
+
+      const table = await docTable.getTable();
+      await (await PageObjects.context.getPredecessorLoadMoreButton()).click();
+      await retry.try(async function () {
+        expect(await docTable.getBodyRows(table)).to.have.length(
+          2 * TEST_DEFAULT_CONTEXT_SIZE + TEST_STEP_SIZE + 1
+        );
+      });
+    });
+
+    it('should increase according to the `context:step` setting when clicking the `load older` button', async function() {
+      await PageObjects.context.navigateTo(TEST_INDEX_PATTERN, TEST_ANCHOR_TYPE, TEST_ANCHOR_ID);
+
+      const table = await docTable.getTable();
+      const successorLoadMoreButton = await PageObjects.context.getSuccessorLoadMoreButton();
+      await remote.moveMouseTo(successorLoadMoreButton);  // possibly scroll until the button is visible
+      await successorLoadMoreButton.click();
+      await retry.try(async function () {
+        expect(await docTable.getBodyRows(table)).to.have.length(
+          2 * TEST_DEFAULT_CONTEXT_SIZE + TEST_STEP_SIZE + 1
+        );
+      });
     });
   });
 
-  bdd.it('should default to the `context:defaultSize` setting', async function () {
-    await PageObjects.context.navigateTo(TEST_INDEX_PATTERN, TEST_ANCHOR_TYPE, TEST_ANCHOR_ID);
-
-    const docTable = await PageObjects.docTable.getTable();
-    await PageObjects.common.try(async function () {
-      expect(await PageObjects.docTable.getBodyRows(docTable)).to.have.length(2 * TEST_DEFAULT_CONTEXT_SIZE + 1);
-    });
-    await PageObjects.common.try(async function() {
-      const predecessorCountPicker = await PageObjects.context.getPredecessorCountPicker();
-      expect(await predecessorCountPicker.getProperty('value')).to.equal(`${TEST_DEFAULT_CONTEXT_SIZE}`);
-    });
-    await PageObjects.common.try(async function() {
-      const successorCountPicker = await PageObjects.context.getSuccessorCountPicker();
-      expect(await successorCountPicker.getProperty('value')).to.equal(`${TEST_DEFAULT_CONTEXT_SIZE}`);
-    });
-  });
-
-  bdd.it('should increase according to the `context:step` setting when clicking the `load newer` button', async function() {
-    await PageObjects.context.navigateTo(TEST_INDEX_PATTERN, TEST_ANCHOR_TYPE, TEST_ANCHOR_ID);
-
-    const docTable = await PageObjects.docTable.getTable();
-    await (await PageObjects.context.getPredecessorLoadMoreButton()).click();
-    await PageObjects.common.try(async function () {
-      expect(await PageObjects.docTable.getBodyRows(docTable)).to.have.length(
-        2 * TEST_DEFAULT_CONTEXT_SIZE + TEST_STEP_SIZE + 1
-      );
-    });
-  });
-
-  bdd.it('should increase according to the `context:step` setting when clicking the `load older` button', async function() {
-    await PageObjects.context.navigateTo(TEST_INDEX_PATTERN, TEST_ANCHOR_TYPE, TEST_ANCHOR_ID);
-
-    const docTable = await PageObjects.docTable.getTable();
-    const successorLoadMoreButton = await PageObjects.context.getSuccessorLoadMoreButton();
-    await this.remote.moveMouseTo(successorLoadMoreButton);  // possibly scroll until the button is visible
-    await successorLoadMoreButton.click();
-    await PageObjects.common.try(async function () {
-      expect(await PageObjects.docTable.getBodyRows(docTable)).to.have.length(
-        2 * TEST_DEFAULT_CONTEXT_SIZE + TEST_STEP_SIZE + 1
-      );
-    });
-  });
-});
+}

test/functional/apps/context/index.js

@@ -1,26 +1,25 @@
+export default function ({ getService, getPageObjects, loadTestFile }) {
+  const config = getService('config');
+  const remote = getService('remote');
+  const esArchiver = getService('esArchiver');
+  const PageObjects = getPageObjects(['common']);
 
-import {
-  bdd,
-  defaultTimeout,
-  esArchiver,
-} from '../../../support';
+  describe('context app', function () {
+    this.timeout(config.get('timeouts.test'));
 
-import PageObjects from '../../../support/page_objects';
+    before(async function () {
+      await remote.setWindowSize(1200,800);
+      await esArchiver.loadIfNeeded('logstash_functional');
+      await esArchiver.load('visualize');
+      await PageObjects.common.navigateToApp('discover');
+    });
 
-bdd.describe('context app', function () {
-  this.timeout = defaultTimeout;
+    after(function unloadMakelogs() {
+      return esArchiver.unload('logstash_functional');
+    });
 
-  bdd.before(async function () {
-    await PageObjects.remote.setWindowSize(1200,800);
-    await esArchiver.loadIfNeeded('logstash_functional');
-    await esArchiver.load('visualize');
-    await PageObjects.common.navigateToApp('discover');
+    loadTestFile(require.resolve('./_discover_navigation'));
+    loadTestFile(require.resolve('./_size'));
   });
 
-  bdd.after(function unloadMakelogs() {
-    return esArchiver.unload('logstash_functional');
-  });
-
-  require('./_discover_navigation');
-  require('./_size');
-});
+}

test/functional/apps/dashboard/_dashboard.js

@@ -1,141 +1,148 @@
-
 import expect from 'expect.js';
+
 import {
-  DEFAULT_PANEL_WIDTH, DEFAULT_PANEL_HEIGHT
+  DEFAULT_PANEL_WIDTH,
+  DEFAULT_PANEL_HEIGHT,
 } from '../../../../src/core_plugins/kibana/public/dashboard/panel/panel_state';
 
-import { bdd } from '../../../support';
-import PageObjects from '../../../support/page_objects';
+export default function ({ getService, getPageObjects }) {
+  const retry = getService('retry');
+  const log = getService('log');
+  const PageObjects = getPageObjects(['common', 'dashboard', 'header', 'visualize']);
 
-bdd.describe('dashboard tab', function describeIndexTests() {
-  bdd.before(async function () {
-    return PageObjects.dashboard.initTests();
-  });
-
-  bdd.it('should be able to add visualizations to dashboard', async function addVisualizations() {
-    PageObjects.common.saveScreenshot('Dashboard-no-visualizations');
-
-    // This flip between apps fixes the url so state is preserved when switching apps in test mode.
-    // Without this flip the url in test mode looks something like
-    // "http://localhost:5620/app/kibana?_t=1486069030837#/dashboard?_g=...."
-    // after the initial flip, the url will look like this: "http://localhost:5620/app/kibana#/dashboard?_g=...."
-    await PageObjects.header.clickVisualize();
-    await PageObjects.header.clickDashboard();
-
-    await PageObjects.dashboard.clickNewDashboard();
-    await PageObjects.dashboard.addVisualizations(PageObjects.dashboard.getTestVisualizationNames());
-
-    PageObjects.common.debug('done adding visualizations');
-    PageObjects.common.saveScreenshot('Dashboard-add-visualizations');
-  });
-
-  bdd.it('set the timepicker time to that which contains our test data', async function setTimepicker() {
-    await PageObjects.dashboard.setTimepickerInDataRange();
-  });
-
-  bdd.it('should save and load dashboard', async function saveAndLoadDashboard() {
-    const dashboardName = 'Dashboard Test 1';
-    await PageObjects.dashboard.saveDashboard(dashboardName);
-    await PageObjects.dashboard.gotoDashboardLandingPage();
-
-    await PageObjects.common.try(function () {
-      PageObjects.common.debug('now re-load previously saved dashboard');
-      return PageObjects.dashboard.loadSavedDashboard(dashboardName);
+  describe('dashboard tab', function describeIndexTests() {
+    before(async function () {
+      return PageObjects.dashboard.initTests();
     });
-    await PageObjects.common.saveScreenshot('Dashboard-load-saved');
-  });
 
-  bdd.it('should have all the expected visualizations', function checkVisualizations() {
-    return PageObjects.common.tryForTime(10000, function () {
-      return PageObjects.dashboard.getPanelTitles()
+    it('should be able to add visualizations to dashboard', async function addVisualizations() {
+      await PageObjects.common.saveScreenshot('Dashboard-no-visualizations');
+
+      // This flip between apps fixes the url so state is preserved when switching apps in test mode.
+      // Without this flip the url in test mode looks something like
+      // "http://localhost:5620/app/kibana?_t=1486069030837#/dashboard?_g=...."
+      // after the initial flip, the url will look like this: "http://localhost:5620/app/kibana#/dashboard?_g=...."
+      await PageObjects.header.clickVisualize();
+      await PageObjects.header.clickDashboard();
+
+      await PageObjects.dashboard.clickNewDashboard();
+      await PageObjects.dashboard.addVisualizations(PageObjects.dashboard.getTestVisualizationNames());
+
+      log.debug('done adding visualizations');
+      await PageObjects.common.saveScreenshot('Dashboard-add-visualizations');
+    });
+
+    it('set the timepicker time to that which contains our test data', async function setTimepicker() {
+      await PageObjects.dashboard.setTimepickerInDataRange();
+    });
+
+    it('should save and load dashboard', async function saveAndLoadDashboard() {
+      const dashboardName = 'Dashboard Test 1';
+      await PageObjects.dashboard.saveDashboard(dashboardName);
+      await PageObjects.header.clickToastOK();
+      await PageObjects.dashboard.gotoDashboardLandingPage();
+
+      await retry.try(function () {
+        log.debug('now re-load previously saved dashboard');
+        return PageObjects.dashboard.loadSavedDashboard(dashboardName);
+      });
+      await PageObjects.common.saveScreenshot('Dashboard-load-saved');
+    });
+
+    it('should have all the expected visualizations', function checkVisualizations() {
+      return retry.tryForTime(10000, function () {
+        return PageObjects.dashboard.getPanelTitles()
         .then(function (panelTitles) {
-          PageObjects.common.log('visualization titles = ' + panelTitles);
+          log.info('visualization titles = ' + panelTitles);
           expect(panelTitles).to.eql(PageObjects.dashboard.getTestVisualizationNames());
         });
-    })
+      })
       .then(function () {
         PageObjects.common.saveScreenshot('Dashboard-has-visualizations');
       });
-  });
-
-  bdd.it('should have all the expected initial sizes', function checkVisualizationSizes() {
-    const width = DEFAULT_PANEL_WIDTH;
-    const height = DEFAULT_PANEL_HEIGHT;
-    const titles = PageObjects.dashboard.getTestVisualizationNames();
-    const visObjects = [
-      { dataCol: '1', dataRow: '1', dataSizeX: width, dataSizeY: height, title: titles[0] },
-      {  dataCol: width + 1, dataRow: '1', dataSizeX: width, dataSizeY: height, title: titles[1] },
-      { dataCol: '1', dataRow: height + 1, dataSizeX: width, dataSizeY: height, title: titles[2] },
-      { dataCol: width + 1, dataRow: height + 1, dataSizeX: width, dataSizeY: height, title: titles[3] },
-      { dataCol: '1', dataRow: (height * 2) + 1, dataSizeX: width, dataSizeY: height, title: titles[4] },
-      { dataCol: width + 1, dataRow: (height * 2) + 1, dataSizeX: width, dataSizeY: height, title: titles[5] },
-      { dataCol: '1', dataRow: (height * 3) + 1, dataSizeX: width, dataSizeY: height, title: titles[6] }
-    ];
-    return PageObjects.common.tryForTime(10000, function () {
-      return PageObjects.dashboard.getPanelSizeData()
-        .then(function (panelTitles) {
-          PageObjects.common.log('visualization titles = ' + panelTitles);
-          PageObjects.common.saveScreenshot('Dashboard-visualization-sizes');
-          expect(panelTitles).to.eql(visObjects);
-        });
     });
-  });
-});
 
-bdd.describe('filters', async function () {
-  bdd.it('are not selected by default', async function () {
-    const descriptions = await PageObjects.dashboard.getFilterDescriptions(1000);
-    expect(descriptions.length).to.equal(0);
-  });
-
-  bdd.it('are added when a pie chart slice is clicked', async function () {
-    await PageObjects.dashboard.filterOnPieSlice();
-    const descriptions = await PageObjects.dashboard.getFilterDescriptions();
-    expect(descriptions.length).to.equal(1);
-  });
-});
-
-bdd.it('retains dark theme in state', async function () {
-  await PageObjects.dashboard.clickEdit();
-  await PageObjects.dashboard.useDarkTheme(true);
-  await PageObjects.header.clickVisualize();
-  await PageObjects.header.clickDashboard();
-  const isDarkThemeOn = await PageObjects.dashboard.isDarkThemeOn();
-  expect(isDarkThemeOn).to.equal(true);
-});
-
-bdd.it('should have data-shared-items-count set to the number of visualizations', function  checkSavedItemsCount() {
-  const visualizations = PageObjects.dashboard.getTestVisualizations();
-  return PageObjects.common.tryForTime(10000, () => PageObjects.dashboard.getSharedItemsCount())
-    .then(function (count) {
-      PageObjects.common.log('data-shared-items-count = ' + count);
-      expect(count).to.eql(visualizations.length);
-    });
-});
-
-bdd.it('should have panels with expected data-shared-item title and description', function checkTitles() {
-  const visualizations = PageObjects.dashboard.getTestVisualizations();
-  return PageObjects.common.tryForTime(10000, function () {
-    return PageObjects.dashboard.getPanelSharedItemData()
-      .then(function (data) {
-        expect(data.map(item => item.title)).to.eql(visualizations.map(v => v.name));
-        expect(data.map(item => item.description)).to.eql(visualizations.map(v => v.description));
+    it('should have all the expected initial sizes', function checkVisualizationSizes() {
+      const width = DEFAULT_PANEL_WIDTH;
+      const height = DEFAULT_PANEL_HEIGHT;
+      const titles = PageObjects.dashboard.getTestVisualizationNames();
+      const visObjects = [
+        { dataCol: '1', dataRow: '1', dataSizeX: width, dataSizeY: height, title: titles[0] },
+        { dataCol: width + 1, dataRow: '1', dataSizeX: width, dataSizeY: height, title: titles[1] },
+        { dataCol: '1', dataRow: height + 1, dataSizeX: width, dataSizeY: height, title: titles[2] },
+        { dataCol: width + 1, dataRow: height + 1, dataSizeX: width, dataSizeY: height, title: titles[3] },
+        { dataCol: '1', dataRow: (height * 2) + 1, dataSizeX: width, dataSizeY: height, title: titles[4] },
+        { dataCol: width + 1, dataRow: (height * 2) + 1, dataSizeX: width, dataSizeY: height, title: titles[5] },
+        { dataCol: '1', dataRow: (height * 3) + 1, dataSizeX: width, dataSizeY: height, title: titles[6] }
+      ];
+      return retry.tryForTime(10000, function () {
+        return PageObjects.dashboard.getPanelSizeData()
+          .then(function (panelTitles) {
+            log.info('visualization titles = ' + panelTitles);
+            PageObjects.common.saveScreenshot('Dashboard-visualization-sizes');
+            expect(panelTitles).to.eql(visObjects);
+          });
       });
-  });
+    });
 
-  bdd.it('add new visualization link', async function checkTitles() {
-    await PageObjects.dashboard.clickAddVisualization();
-    await PageObjects.dashboard.clickAddNewVisualizationLink();
-    await PageObjects.visualize.clickAreaChart();
-    await PageObjects.visualize.clickNewSearch();
-    await PageObjects.visualize.saveVisualization('visualization from add new link');
+    describe('filters', async function () {
+      it('are not selected by default', async function () {
+        const filters = await PageObjects.dashboard.getFilters(1000);
+        expect(filters.length).to.equal(0);
+      });
 
-    const visualizations = PageObjects.dashboard.getTestVisualizations();
-    return PageObjects.common.tryForTime(10000, async function () {
-      const panelTitles = await PageObjects.dashboard.getPanelSizeData();
-      PageObjects.common.log('visualization titles = ' + panelTitles.map(item => item.title));
-      PageObjects.common.saveScreenshot('Dashboard-visualization-sizes');
-      expect(panelTitles.length).to.eql(visualizations.length + 1);
+      it('are added when a pie chart slice is clicked', async function () {
+        await PageObjects.dashboard.filterOnPieSlice();
+        const filters = await PageObjects.dashboard.getFilters();
+        expect(filters.length).to.equal(1);
+      });
+    });
+
+    it('retains dark theme in state', async function () {
+      await PageObjects.dashboard.clickEdit();
+      await PageObjects.dashboard.useDarkTheme(true);
+      await PageObjects.header.clickVisualize();
+      await PageObjects.header.clickDashboard();
+      const isDarkThemeOn = await PageObjects.dashboard.isDarkThemeOn();
+      expect(isDarkThemeOn).to.equal(true);
+    });
+
+    it('should have data-shared-items-count set to the number of visualizations', function checkSavedItemsCount() {
+      const visualizations = PageObjects.dashboard.getTestVisualizations();
+      return retry.tryForTime(10000, () => PageObjects.dashboard.getSharedItemsCount())
+      .then(function (count) {
+        log.info('data-shared-items-count = ' + count);
+        expect(count).to.eql(visualizations.length);
+      });
+    });
+
+    it('should have panels with expected data-shared-item title and description', function checkTitles() {
+      const visualizations = PageObjects.dashboard.getTestVisualizations();
+      return retry.tryForTime(10000, function () {
+        return PageObjects.dashboard.getPanelSharedItemData()
+        .then(function (data) {
+          expect(data.map(item => item.title)).to.eql(visualizations.map(v => v.name));
+          expect(data.map(item => item.description)).to.eql(visualizations.map(v => v.description));
+        });
+      });
+    });
+
+    it('add new visualization link', async function checkTitles() {
+      await PageObjects.dashboard.clickAddVisualization();
+      await PageObjects.dashboard.clickAddNewVisualizationLink();
+      await PageObjects.visualize.clickAreaChart();
+      await PageObjects.visualize.clickNewSearch();
+      await PageObjects.visualize.saveVisualization('visualization from add new link');
+      await PageObjects.header.clickToastOK();
+      await PageObjects.header.clickToastOK();
+
+      const visualizations = PageObjects.dashboard.getTestVisualizations();
+      return retry.tryForTime(10000, async function () {
+        const panelTitles = await PageObjects.dashboard.getPanelSizeData();
+        log.info('visualization titles = ' + panelTitles.map(item => item.title));
+        PageObjects.common.saveScreenshot('Dashboard-visualization-sizes');
+        expect(panelTitles.length).to.eql(visualizations.length + 1);
+      });
     });
   });
-});
+}

test/functional/apps/dashboard/_dashboard_save.js

@@ -1,89 +1,97 @@
 import expect from 'expect.js';
-import { bdd } from '../../../support';
 
-import PageObjects from '../../../support/page_objects';
+export default function ({ getPageObjects }) {
+  const PageObjects = getPageObjects(['dashboard', 'header', 'common']);
 
-bdd.describe('dashboard save', function describeIndexTests() {
-  const dashboardName = 'Dashboard Save Test';
+  describe('dashboard save', function describeIndexTests() {
+    const dashboardName = 'Dashboard Save Test';
 
-  bdd.before(async function () {
-    return PageObjects.dashboard.initTests();
-  });
+    before(async function () {
+      await PageObjects.dashboard.initTests();
+    });
 
-  bdd.it('warns on duplicate name for new dashboard', async function() {
-    await PageObjects.dashboard.clickNewDashboard();
-    await PageObjects.dashboard.saveDashboard(dashboardName);
+    after(async function () {
+      await PageObjects.dashboard.gotoDashboardLandingPage();
+    });
 
-    let isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
-    expect(isConfirmOpen).to.equal(false);
-
-    await PageObjects.dashboard.gotoDashboardLandingPage();
-    await PageObjects.dashboard.clickNewDashboard();
-    await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName);
-
-    isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
-    expect(isConfirmOpen).to.equal(true);
-  });
-
-  bdd.it('does not save on reject confirmation', async function() {
-    await PageObjects.common.clickCancelOnModal();
-
-    const countOfDashboards = await PageObjects.dashboard.getDashboardCountWithName(dashboardName);
-    expect(countOfDashboards).to.equal(1);
-  });
-
-  bdd.it('Saves on confirm duplicate title warning', async function() {
-    await PageObjects.dashboard.gotoDashboardLandingPage();
-    await PageObjects.dashboard.clickNewDashboard();
-    await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName);
-
-    await PageObjects.common.clickConfirmOnModal();
-
-    // This is important since saving a new dashboard will cause a refresh of the page.  We have to
-    // wait till it finishes reloading or it might reload the url after simulating the
-    // dashboard landing page click.
-    await PageObjects.header.waitUntilLoadingHasFinished();
-
-    const countOfDashboards = await PageObjects.dashboard.getDashboardCountWithName(dashboardName);
-    expect(countOfDashboards).to.equal(2);
-  });
-
-  bdd.it('Does not warn when you save an existing dashboard with the title it already has, and that title is a duplicate',
-    async function() {
-      await PageObjects.dashboard.clickDashboardByLinkText(dashboardName);
-      await PageObjects.header.isGlobalLoadingIndicatorHidden();
-      await PageObjects.dashboard.clickEdit();
+    it('warns on duplicate name for new dashboard', async function() {
+      await PageObjects.dashboard.clickNewDashboard();
       await PageObjects.dashboard.saveDashboard(dashboardName);
+      await PageObjects.header.clickToastOK();
+
+      let isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
+      expect(isConfirmOpen).to.equal(false);
+
+      await PageObjects.dashboard.gotoDashboardLandingPage();
+      await PageObjects.dashboard.clickNewDashboard();
+      await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName);
+
+      isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
+      expect(isConfirmOpen).to.equal(true);
+    });
+
+    it('does not save on reject confirmation', async function() {
+      await PageObjects.common.clickCancelOnModal();
+
+      const countOfDashboards = await PageObjects.dashboard.getDashboardCountWithName(dashboardName);
+      expect(countOfDashboards).to.equal(1);
+    });
+
+    it('Saves on confirm duplicate title warning', async function() {
+      await PageObjects.dashboard.gotoDashboardLandingPage();
+      await PageObjects.dashboard.clickNewDashboard();
+      await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName);
+
+      await PageObjects.common.clickConfirmOnModal();
+      await PageObjects.header.clickToastOK();
+
+      // This is important since saving a new dashboard will cause a refresh of the page. We have to
+      // wait till it finishes reloading or it might reload the url after simulating the
+      // dashboard landing page click.
+      await PageObjects.header.waitUntilLoadingHasFinished();
+
+      const countOfDashboards = await PageObjects.dashboard.getDashboardCountWithName(dashboardName);
+      expect(countOfDashboards).to.equal(2);
+    });
+
+    it('Does not warn when you save an existing dashboard with the title it already has, and that title is a duplicate',
+      async function() {
+        await PageObjects.dashboard.clickDashboardByLinkText(dashboardName);
+        await PageObjects.header.isGlobalLoadingIndicatorHidden();
+        await PageObjects.dashboard.clickEdit();
+        await PageObjects.dashboard.saveDashboard(dashboardName);
+        await PageObjects.header.clickToastOK();
+
+        const isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
+        expect(isConfirmOpen).to.equal(false);
+      }
+    );
+
+    it('Warns you when you Save as New Dashboard, and the title is a duplicate', async function() {
+      await PageObjects.dashboard.clickEdit();
+      await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName, { saveAsNew: true });
+
+      const isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
+      expect(isConfirmOpen).to.equal(true);
+
+      await PageObjects.common.clickCancelOnModal();
+    });
+
+    it('Does not warn when only the prefix matches', async function() {
+      await PageObjects.dashboard.saveDashboard(dashboardName.split(' ')[0]);
 
       const isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
       expect(isConfirmOpen).to.equal(false);
-    }
-  );
+    });
 
-  bdd.it('Warns you when you Save as New Dashboard, and the title is a duplicate', async function() {
-    await PageObjects.dashboard.clickEdit();
-    await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName, { saveAsNew: true });
+    it('Warns when case is different', async function() {
+      await PageObjects.dashboard.clickEdit();
+      await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName.toUpperCase());
 
-    const isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
-    expect(isConfirmOpen).to.equal(true);
+      const isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
+      expect(isConfirmOpen).to.equal(true);
 
-    await PageObjects.common.clickCancelOnModal();
+      await PageObjects.common.clickCancelOnModal();
+    });
   });
-
-  bdd.it('Does not warn when only the prefix matches', async function() {
-    await PageObjects.dashboard.saveDashboard(dashboardName.split(' ')[0]);
-
-    const isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
-    expect(isConfirmOpen).to.equal(false);
-  });
-
-  bdd.it('Warns when case is different', async function() {
-    await PageObjects.dashboard.clickEdit();
-    await PageObjects.dashboard.enterDashboardTitleAndClickSave(dashboardName.toUpperCase());
-
-    const isConfirmOpen = await PageObjects.common.isConfirmModalOpen();
-    expect(isConfirmOpen).to.equal(true);
-
-    await PageObjects.common.clickCancelOnModal();
-  });
-});
+}

test/functional/apps/dashboard/_dashboard_time.js

@@ -1,86 +1,95 @@
 import expect from 'expect.js';
 
-import { bdd } from '../../../support';
-import PageObjects from '../../../support/page_objects';
-
 const dashboardName = 'Dashboard Test Time';
 
 const fromTime = '2015-09-19 06:31:44.000';
 const toTime = '2015-09-23 18:31:44.000';
 
-bdd.describe('dashboard time', function dashboardSaveWithTime() {
-  bdd.before(async function () {
-    await PageObjects.dashboard.initTests();
+export default function ({ getPageObjects }) {
+  const PageObjects = getPageObjects(['dashboard', 'header']);
 
-    // This flip between apps fixes the url so state is preserved when switching apps in test mode.
-    await PageObjects.header.clickVisualize();
-    await PageObjects.header.clickDashboard();
-  });
+  describe('dashboard time', function dashboardSaveWithTime() {
+    before(async function () {
+      await PageObjects.dashboard.initTests();
 
-  bdd.describe('dashboard without stored timed', async function () {
-    bdd.it('is saved', async function () {
-      await PageObjects.dashboard.clickNewDashboard();
-      await PageObjects.dashboard.addVisualizations([PageObjects.dashboard.getTestVisualizationNames()[0]]);
-      await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: false });
+      // This flip between apps fixes the url so state is preserved when switching apps in test mode.
+      await PageObjects.header.clickVisualize();
+      await PageObjects.header.clickDashboard();
     });
 
-    bdd.it('Does not set the time picker on open', async function () {
-      await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-
-      await PageObjects.dashboard.loadSavedDashboard(dashboardName);
-
-      const fromTimeNext = await PageObjects.header.getFromTime();
-      const toTimeNext = await PageObjects.header.getToTime();
-      expect(fromTimeNext).to.equal(fromTime);
-      expect(toTimeNext).to.equal(toTime);
-    });
-  });
-
-  bdd.describe('dashboard with stored timed', async function () {
-    bdd.it('is saved with quick time', async function () {
-      await PageObjects.dashboard.clickEdit();
-      await PageObjects.header.setQuickTime('Today');
-      await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
+    after(async function () {
+      await PageObjects.dashboard.gotoDashboardLandingPage();
     });
 
-    bdd.it('sets quick time on open', async function () {
-      await PageObjects.header.setAbsoluteRange(fromTime, toTime);
+    describe('dashboard without stored timed', async function () {
+      it('is saved', async function () {
+        await PageObjects.dashboard.clickNewDashboard();
+        await PageObjects.dashboard.addVisualizations([PageObjects.dashboard.getTestVisualizationNames()[0]]);
+        await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: false });
+        await PageObjects.header.clickToastOK();
+      });
 
-      await PageObjects.dashboard.loadSavedDashboard(dashboardName);
+      it('Does not set the time picker on open', async function () {
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
 
-      const prettyPrint = await PageObjects.header.getPrettyDuration();
-      expect(prettyPrint).to.equal('Today');
+        await PageObjects.dashboard.loadSavedDashboard(dashboardName);
+
+        const fromTimeNext = await PageObjects.header.getFromTime();
+        const toTimeNext = await PageObjects.header.getToTime();
+        expect(fromTimeNext).to.equal(fromTime);
+        expect(toTimeNext).to.equal(toTime);
+      });
     });
 
-    bdd.it('is saved with absolute time', async function () {
-      await PageObjects.dashboard.clickEdit();
-      await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-      await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
-    });
+    describe('dashboard with stored timed', async function () {
+      it('is saved with quick time', async function () {
+        await PageObjects.dashboard.clickEdit();
+        await PageObjects.header.setQuickTime('Today');
+        await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
+        await PageObjects.header.clickToastOK();
+      });
 
-    bdd.it('sets absolute time on open', async function () {
-      await PageObjects.header.setQuickTime('Today');
+      it('sets quick time on open', async function () {
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
 
-      await PageObjects.dashboard.loadSavedDashboard(dashboardName);
+        await PageObjects.dashboard.loadSavedDashboard(dashboardName);
 
-      const fromTimeNext = await PageObjects.header.getFromTime();
-      const toTimeNext = await PageObjects.header.getToTime();
-      expect(fromTimeNext).to.equal(fromTime);
-      expect(toTimeNext).to.equal(toTime);
+        const prettyPrint = await PageObjects.header.getPrettyDuration();
+        expect(prettyPrint).to.equal('Today');
+      });
+
+      it('is saved with absolute time', async function () {
+        await PageObjects.dashboard.clickEdit();
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
+        await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
+        await PageObjects.header.clickToastOK();
+      });
+
+      it('sets absolute time on open', async function () {
+        await PageObjects.header.setQuickTime('Today');
+
+        await PageObjects.dashboard.loadSavedDashboard(dashboardName);
+
+        const fromTimeNext = await PageObjects.header.getFromTime();
+        const toTimeNext = await PageObjects.header.getToTime();
+        expect(fromTimeNext).to.equal(fromTime);
+        expect(toTimeNext).to.equal(toTime);
+      });
     });
 
     // If the user has time stored with a dashboard, it's supposed to override the current time settings
     // when it's opened. However, if the user then changes the time, navigates to visualize, then navigates
     // back to dashboard, the overridden time should be preserved. The time is *only* reset on open, not
     // during navigation or page refreshes.
-    bdd.it('preserves time changes during navigation', async function () {
-      await PageObjects.header.setQuickTime('Today');
-      await PageObjects.header.clickVisualize();
-      await PageObjects.header.clickDashboard();
+    describe('time changes', function () {
+      it('preserved during navigation', async function () {
+        await PageObjects.header.setQuickTime('Today');
+        await PageObjects.header.clickVisualize();
+        await PageObjects.header.clickDashboard();
 
-      const prettyPrint = await PageObjects.header.getPrettyDuration();
-      expect(prettyPrint).to.equal('Today');
+        const prettyPrint = await PageObjects.header.getPrettyDuration();
+        expect(prettyPrint).to.equal('Today');
+      });
     });
   });
-
-});
+}

test/functional/apps/dashboard/_view_edit.js

@@ -1,283 +1,298 @@
 import expect from 'expect.js';
 
-import { bdd } from '../../../support';
-import PageObjects from '../../../support/page_objects';
+export default function ({ getService, getPageObjects }) {
+  const testSubjects = getService('testSubjects');
+  const PageObjects = getPageObjects(['dashboard', 'header', 'common', 'visualize']);
+  const dashboardName = 'Dashboard View Edit Test';
 
-const dashboardName = 'Dashboard View Edit Test';
+  describe('dashboard view edit mode', function viewEditModeTests() {
+    before(async function () {
+      await PageObjects.dashboard.initTests();
+    });
 
-bdd.describe('dashboard view edit mode', function viewEditModeTests() {
-  bdd.before(async function () {
-    return PageObjects.dashboard.initTests();
-  });
+    after(async function () {
+      await PageObjects.dashboard.gotoDashboardLandingPage();
+    });
 
-  bdd.it('create new dashboard opens in edit mode', async function () {
-    // This flip between apps fixes the url so state is preserved when switching apps in test mode.
-    // Without this flip the url in test mode looks something like
-    // "http://localhost:5620/app/kibana?_t=1486069030837#/dashboard?_g=...."
-    // after the initial flip, the url will look like this: "http://localhost:5620/app/kibana#/dashboard?_g=...."
-    await PageObjects.header.clickVisualize();
-    await PageObjects.header.clickDashboard();
+    it('create new dashboard opens in edit mode', async function () {
+      // This flip between apps fixes the url so state is preserved when switching apps in test mode.
+      // Without this flip the url in test mode looks something like
+      // "http://localhost:5620/app/kibana?_t=1486069030837#/dashboard?_g=...."
+      // after the initial flip, the url will look like this: "http://localhost:5620/app/kibana#/dashboard?_g=...."
+      await PageObjects.header.clickVisualize();
+      await PageObjects.header.clickDashboard();
 
-    await PageObjects.dashboard.clickNewDashboard();
-    await PageObjects.dashboard.clickCancelOutOfEditMode();
-  });
+      await PageObjects.dashboard.clickNewDashboard();
+      await PageObjects.dashboard.clickCancelOutOfEditMode();
+    });
 
-  bdd.it('create test dashboard', async function () {
-    await PageObjects.dashboard.gotoDashboardLandingPage();
-    await PageObjects.dashboard.clickNewDashboard();
-    await PageObjects.dashboard.addVisualizations(PageObjects.dashboard.getTestVisualizationNames());
-    await PageObjects.dashboard.saveDashboard(dashboardName);
-  });
+    it('create test dashboard', async function () {
+      await PageObjects.dashboard.gotoDashboardLandingPage();
+      await PageObjects.dashboard.clickNewDashboard();
+      await PageObjects.dashboard.addVisualizations(PageObjects.dashboard.getTestVisualizationNames());
+      await PageObjects.dashboard.saveDashboard(dashboardName);
+      await PageObjects.header.clickToastOK();
+    });
 
-  bdd.it('existing dashboard opens in view mode', async function () {
-    await PageObjects.dashboard.gotoDashboardLandingPage();
-    await PageObjects.dashboard.clickDashboardByLinkText(dashboardName);
-    const inViewMode = await PageObjects.dashboard.getIsInViewMode();
-
-    expect(inViewMode).to.equal(true);
-  });
-
-  bdd.describe('panel edit controls', function () {
-    bdd.it('are hidden in view mode', async function () {
+    it('existing dashboard opens in view mode', async function () {
       await PageObjects.dashboard.gotoDashboardLandingPage();
       await PageObjects.dashboard.clickDashboardByLinkText(dashboardName);
+      const inViewMode = await PageObjects.dashboard.getIsInViewMode();
 
-      const editLinkExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelEditLink');
-      const moveExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelMoveIcon');
-      const removeExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelRemoveIcon');
-
-      expect(editLinkExists).to.equal(false);
-      expect(moveExists).to.equal(false);
-      expect(removeExists).to.equal(false);
+      expect(inViewMode).to.equal(true);
     });
 
-    bdd.it('are shown in edit mode', async function () {
-      await PageObjects.dashboard.clickEdit();
+    describe('panel edit controls', function () {
+      it('are hidden in view mode', async function () {
+        await PageObjects.dashboard.gotoDashboardLandingPage();
+        await PageObjects.dashboard.clickDashboardByLinkText(dashboardName);
 
-      const editLinkExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelEditLink');
-      const moveExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelMoveIcon');
-      const removeExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelRemoveIcon');
-
-      expect(editLinkExists).to.equal(true);
-      expect(moveExists).to.equal(true);
-      expect(removeExists).to.equal(true);
-    });
-
-    bdd.describe('on an expanded panel', function () {
-      bdd.it('are hidden in view mode', async function () {
-        await PageObjects.dashboard.saveDashboard(dashboardName);
-        await PageObjects.dashboard.toggleExpandPanel();
-
-        const editLinkExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelEditLink');
-        const moveExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelMoveIcon');
-        const removeExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelRemoveIcon');
+        const editLinkExists = await testSubjects.exists('dashboardPanelEditLink');
+        const moveExists = await testSubjects.exists('dashboardPanelMoveIcon');
+        const removeExists = await testSubjects.exists('dashboardPanelRemoveIcon');
 
         expect(editLinkExists).to.equal(false);
         expect(moveExists).to.equal(false);
         expect(removeExists).to.equal(false);
       });
 
-      bdd.it('in edit mode hides move and remove icons ', async function () {
+      it('are shown in edit mode', async function () {
         await PageObjects.dashboard.clickEdit();
 
-        const editLinkExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelEditLink');
-        const moveExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelMoveIcon');
-        const removeExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelRemoveIcon');
+        const editLinkExists = await testSubjects.exists('dashboardPanelEditLink');
+        const moveExists = await testSubjects.exists('dashboardPanelMoveIcon');
+        const removeExists = await testSubjects.exists('dashboardPanelRemoveIcon');
 
         expect(editLinkExists).to.equal(true);
-        expect(moveExists).to.equal(false);
-        expect(removeExists).to.equal(false);
+        expect(moveExists).to.equal(true);
+        expect(removeExists).to.equal(true);
+      });
 
-        await PageObjects.dashboard.toggleExpandPanel();
+      describe('on an expanded panel', function () {
+        it('are hidden in view mode', async function () {
+          await PageObjects.dashboard.saveDashboard(dashboardName);
+          await PageObjects.header.clickToastOK();
+          await PageObjects.dashboard.toggleExpandPanel();
+
+          const editLinkExists = await testSubjects.exists('dashboardPanelEditLink');
+          const moveExists = await testSubjects.exists('dashboardPanelMoveIcon');
+          const removeExists = await testSubjects.exists('dashboardPanelRemoveIcon');
+
+          expect(editLinkExists).to.equal(false);
+          expect(moveExists).to.equal(false);
+          expect(removeExists).to.equal(false);
+        });
+
+        it('in edit mode hides move and remove icons ', async function () {
+          await PageObjects.dashboard.clickEdit();
+
+          const editLinkExists = await testSubjects.exists('dashboardPanelEditLink');
+          const moveExists = await testSubjects.exists('dashboardPanelMoveIcon');
+          const removeExists = await testSubjects.exists('dashboardPanelRemoveIcon');
+
+          expect(editLinkExists).to.equal(true);
+          expect(moveExists).to.equal(false);
+          expect(removeExists).to.equal(false);
+
+          await PageObjects.dashboard.toggleExpandPanel();
+        });
       });
     });
-  });
 
-  // Panel expand should also be shown in view mode, but only on mouse hover.
-  bdd.describe('panel expand control shown in edit mode', async function () {
-    const expandExists = await PageObjects.common.doesTestSubjectExist('dashboardPanelExpandIcon');
-    expect(expandExists).to.equal(true);
-  });
+    // Panel expand should also be shown in view mode, but only on mouse hover.
+    describe('panel expand control', function () {
+      it('shown in edit mode', async function () {
+        await PageObjects.dashboard.gotoDashboardEditMode(dashboardName);
+        const expandExists = await testSubjects.exists('dashboardPanelExpandIcon');
+        expect(expandExists).to.equal(true);
+      });
+    });
 
-  bdd.it('save auto exits out of edit mode', async function () {
-    await PageObjects.dashboard.saveDashboard(dashboardName);
-    const isViewMode = await PageObjects.dashboard.getIsInViewMode();
+    describe('save', function () {
+      it('auto exits out of edit mode', async function () {
+        await PageObjects.dashboard.gotoDashboardEditMode(dashboardName);
+        await PageObjects.dashboard.saveDashboard(dashboardName);
+        await PageObjects.header.clickToastOK();
+        const isViewMode = await PageObjects.dashboard.getIsInViewMode();
+        expect(isViewMode).to.equal(true);
+      });
+    });
 
-    expect(isViewMode).to.equal(true);
-  });
+    describe('shows lose changes warning', async function () {
+      describe('and loses changes on confirmation', function () {
+        beforeEach(async function () {
+          await PageObjects.dashboard.gotoDashboardEditMode(dashboardName);
+        });
 
-  bdd.describe('shows lose changes warning', async function () {
-    bdd.describe('and loses changes on confirmation', function () {
-      bdd.it('when time changed is stored with dashboard', async function () {
-        await PageObjects.dashboard.clickEdit();
-        const originalFromTime = '2015-09-19 06:31:44.000';
-        const originalToTime = '2015-09-19 06:31:44.000';
-        await PageObjects.header.setAbsoluteRange(originalFromTime, originalToTime);
-        await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
+        it('when time changed is stored with dashboard', async function () {
+          const originalFromTime = '2015-09-19 06:31:44.000';
+          const originalToTime = '2015-09-19 06:31:44.000';
+          await PageObjects.header.setAbsoluteRange(originalFromTime, originalToTime);
+          await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
+          await PageObjects.header.clickToastOK();
 
-        await PageObjects.dashboard.clickEdit();
-        await PageObjects.header.setAbsoluteRange('2013-09-19 06:31:44.000', '2013-09-19 06:31:44.000');
-        await PageObjects.dashboard.clickCancelOutOfEditMode();
+          await PageObjects.dashboard.clickEdit();
+          await PageObjects.header.setAbsoluteRange('2013-09-19 06:31:44.000', '2013-09-19 06:31:44.000');
+          await PageObjects.dashboard.clickCancelOutOfEditMode();
 
-        // confirm lose changes
-        await PageObjects.common.clickConfirmOnModal();
+          // confirm lose changes
+          await PageObjects.common.clickConfirmOnModal();
 
-        const newFromTime = await PageObjects.header.getFromTime();
-        const newToTime = await PageObjects.header.getToTime();
+          const newFromTime = await PageObjects.header.getFromTime();
+          const newToTime = await PageObjects.header.getToTime();
 
-        expect(newFromTime).to.equal(originalFromTime);
-        expect(newToTime).to.equal(originalToTime);
+          expect(newFromTime).to.equal(originalFromTime);
+          expect(newToTime).to.equal(originalToTime);
+        });
+
+        it('when the query is edited and applied', async function () {
+          const originalQuery = await PageObjects.dashboard.getQuery();
+          await PageObjects.dashboard.appendQuery('extra stuff');
+          await PageObjects.dashboard.clickFilterButton();
+
+          await PageObjects.dashboard.clickCancelOutOfEditMode();
+
+          // confirm lose changes
+          await PageObjects.common.clickConfirmOnModal();
+
+          const query = await PageObjects.dashboard.getQuery();
+          expect(query).to.equal(originalQuery);
+        });
+
+        it('when a filter is deleted', async function () {
+          await PageObjects.dashboard.setTimepickerInDataRange();
+          await PageObjects.dashboard.filterOnPieSlice();
+          await PageObjects.dashboard.saveDashboard(dashboardName);
+          await PageObjects.header.clickToastOK();
+
+          // This may seem like a pointless line but there was a bug that only arose when the dashboard
+          // was loaded initially
+          await PageObjects.dashboard.loadSavedDashboard(dashboardName);
+          await PageObjects.dashboard.clickEdit();
+
+          const originalFilters = await PageObjects.dashboard.getFilters();
+
+          // Click to cause hover menu to show up, but it will also actually click the filter, which will turn
+          // it off, so we need to click twice to turn it back on.
+          await originalFilters[0].click();
+          await originalFilters[0].click();
+
+          const removeFilterButton = await testSubjects.find('removeFilter-memory');
+          await removeFilterButton.click();
+
+          const noFilters = await PageObjects.dashboard.getFilters(1000);
+          expect(noFilters.length).to.equal(0);
+
+          await PageObjects.dashboard.clickCancelOutOfEditMode();
+
+          // confirm lose changes
+          await PageObjects.common.clickConfirmOnModal();
+
+          const reloadedFilters = await PageObjects.dashboard.getFilters();
+          expect(reloadedFilters.length).to.equal(1);
+        });
+
+        it('when a new vis is added', async function () {
+          await PageObjects.dashboard.clickAddVisualization();
+          await PageObjects.dashboard.clickAddNewVisualizationLink();
+          await PageObjects.visualize.clickAreaChart();
+          await PageObjects.visualize.clickNewSearch();
+          await PageObjects.visualize.saveVisualization('new viz panel');
+          await PageObjects.header.clickToastOK();
+          await PageObjects.header.clickToastOK();
+
+          await PageObjects.dashboard.clickCancelOutOfEditMode();
+
+          // confirm lose changes
+          await PageObjects.common.clickConfirmOnModal();
+
+          const visualizations = PageObjects.dashboard.getTestVisualizations();
+          const panelTitles = await PageObjects.dashboard.getPanelSizeData();
+          expect(panelTitles.length).to.eql(visualizations.length);
+        });
+
+        it('when an existing vis is added', async function () {
+          await PageObjects.dashboard.addVisualization('new viz panel');
+          await PageObjects.dashboard.clickCancelOutOfEditMode();
+
+          // confirm lose changes
+          await PageObjects.common.clickConfirmOnModal();
+
+          const visualizations = PageObjects.dashboard.getTestVisualizations();
+          const panelTitles = await PageObjects.dashboard.getPanelSizeData();
+          expect(panelTitles.length).to.eql(visualizations.length);
+        });
       });
 
-      bdd.it('when the query is edited and applied', async function () {
+      describe('and preserves edits on cancel', function () {
+        it('when time changed is stored with dashboard', async function () {
+          await PageObjects.dashboard.gotoDashboardEditMode(dashboardName);
+          const newFromTime = '2015-09-19 06:31:44.000';
+          const newToTime = '2015-09-19 06:31:44.000';
+          await PageObjects.header.setAbsoluteRange('2013-09-19 06:31:44.000', '2013-09-19 06:31:44.000');
+          await PageObjects.dashboard.saveDashboard(dashboardName, true);
+          await PageObjects.header.clickToastOK();
+          await PageObjects.dashboard.clickEdit();
+          await PageObjects.header.setAbsoluteRange(newToTime, newToTime);
+          await PageObjects.dashboard.clickCancelOutOfEditMode();
+
+          await PageObjects.common.clickCancelOnModal();
+          await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
+          await PageObjects.header.clickToastOK();
+
+          await PageObjects.dashboard.loadSavedDashboard(dashboardName);
+
+          const fromTime = await PageObjects.header.getFromTime();
+          const toTime = await PageObjects.header.getToTime();
+
+          expect(fromTime).to.equal(newFromTime);
+          expect(toTime).to.equal(newToTime);
+        });
+      });
+    });
+
+    describe('Does not show lose changes warning', async function () {
+      it('when time changed is not stored with dashboard', async function () {
+        await PageObjects.dashboard.gotoDashboardEditMode(dashboardName);
+        await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: false });
+        await PageObjects.header.clickToastOK();
         await PageObjects.dashboard.clickEdit();
-
-        const originalQuery = await PageObjects.dashboard.getQuery();
-        await PageObjects.dashboard.appendQuery('extra stuff');
-        await PageObjects.dashboard.clickFilterButton();
-
+        await PageObjects.header.setAbsoluteRange('2013-10-19 06:31:44.000', '2013-12-19 06:31:44.000');
         await PageObjects.dashboard.clickCancelOutOfEditMode();
 
-        // confirm lose changes
-        await PageObjects.common.clickConfirmOnModal();
-
-        const query = await PageObjects.dashboard.getQuery();
-        expect(query).to.equal(originalQuery);
+        const isOpen = await PageObjects.common.isConfirmModalOpen();
+        expect(isOpen).to.be(false);
       });
 
-      bdd.it('when a filter is deleted', async function () {
-        await PageObjects.dashboard.clickEdit();
+      it('when a dashboard has a filter and remains unchanged', async function () {
+        await PageObjects.dashboard.gotoDashboardEditMode(dashboardName);
         await PageObjects.dashboard.setTimepickerInDataRange();
         await PageObjects.dashboard.filterOnPieSlice();
         await PageObjects.dashboard.saveDashboard(dashboardName);
-
-        // This may seem like a pointless line but there was a bug that only arose when the dashboard
-        // was loaded initially
-        await PageObjects.dashboard.loadSavedDashboard(dashboardName);
+        await PageObjects.header.clickToastOK();
         await PageObjects.dashboard.clickEdit();
-
-        const originalFilters = await PageObjects.dashboard.getFilters();
-
-        // Click to cause hover menu to show up, but it will also actually click the filter, which will turn
-        // it off, so we need to click twice to turn it back on.
-        await originalFilters[0].click();
-        await originalFilters[0].click();
-
-        const removeFilterButton = await PageObjects.common.findTestSubject('removeFilter-memory');
-        await removeFilterButton.click();
-
-        const noFilters = await PageObjects.dashboard.getFilters(1000);
-        expect(noFilters.length).to.equal(0);
-
         await PageObjects.dashboard.clickCancelOutOfEditMode();
 
-        // confirm lose changes
-        await PageObjects.common.clickConfirmOnModal();
-
-        const reloadedFilters = await PageObjects.dashboard.getFilters();
-        expect(reloadedFilters.length).to.equal(1);
+        const isOpen = await PageObjects.common.isConfirmModalOpen();
+        expect(isOpen).to.be(false);
       });
 
-      bdd.it('when a new vis is added', async function () {
-        await PageObjects.dashboard.loadSavedDashboard(dashboardName);
-        await PageObjects.dashboard.clickEdit();
+      // See https://github.com/elastic/kibana/issues/10110 - this is intentional.
+      it('when the query is edited but not applied', async function () {
+        await PageObjects.dashboard.gotoDashboardEditMode(dashboardName);
 
-        await PageObjects.dashboard.clickAddVisualization();
-        await PageObjects.dashboard.clickAddNewVisualizationLink();
-        await PageObjects.visualize.clickAreaChart();
-        await PageObjects.visualize.clickNewSearch();
-        await PageObjects.visualize.saveVisualization('new viz panel');
+        const originalQuery = await PageObjects.dashboard.getQuery();
+        await PageObjects.dashboard.appendQuery('extra stuff');
 
         await PageObjects.dashboard.clickCancelOutOfEditMode();
 
-        // confirm lose changes
-        await PageObjects.common.clickConfirmOnModal();
-
-        const visualizations = PageObjects.dashboard.getTestVisualizations();
-        const panelTitles = await PageObjects.dashboard.getPanelSizeData();
-        expect(panelTitles.length).to.eql(visualizations.length);
-      });
-
-      bdd.it('when an existing vis is added', async function () {
-        await PageObjects.dashboard.loadSavedDashboard(dashboardName);
-        await PageObjects.dashboard.clickEdit();
-
-        await PageObjects.dashboard.addVisualization('new viz panel');
-        await PageObjects.dashboard.clickCancelOutOfEditMode();
-
-        // confirm lose changes
-        await PageObjects.common.clickConfirmOnModal();
-
-        const visualizations = PageObjects.dashboard.getTestVisualizations();
-        const panelTitles = await PageObjects.dashboard.getPanelSizeData();
-        expect(panelTitles.length).to.eql(visualizations.length);
-      });
-    });
-
-    bdd.describe('and preserves edits on cancel', function () {
-      bdd.it('when time changed is stored with dashboard', async function () {
-        await PageObjects.dashboard.clickEdit();
-        const newFromTime = '2015-09-19 06:31:44.000';
-        const newToTime = '2015-09-19 06:31:44.000';
-        await PageObjects.header.setAbsoluteRange('2013-09-19 06:31:44.000', '2013-09-19 06:31:44.000');
-        await PageObjects.dashboard.saveDashboard(dashboardName, true);
-        await PageObjects.dashboard.clickEdit();
-        await PageObjects.header.setAbsoluteRange(newToTime, newToTime);
-        await PageObjects.dashboard.clickCancelOutOfEditMode();
-
-        await PageObjects.common.clickCancelOnModal();
-        await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: true });
+        const isOpen = await PageObjects.common.isConfirmModalOpen();
+        expect(isOpen).to.be(false);
 
         await PageObjects.dashboard.loadSavedDashboard(dashboardName);
-
-        const fromTime = await PageObjects.header.getFromTime();
-        const toTime = await PageObjects.header.getToTime();
-
-        expect(fromTime).to.equal(newFromTime);
-        expect(toTime).to.equal(newToTime);
+        const query = await PageObjects.dashboard.getQuery();
+        expect(query).to.equal(originalQuery);
       });
     });
   });
-
-  bdd.describe('Does not show lose changes warning', async function () {
-    bdd.it('when time changed is not stored with dashboard', async function () {
-      await PageObjects.dashboard.clickEdit();
-      await PageObjects.dashboard.saveDashboard(dashboardName, { storeTimeWithDashboard: false });
-      await PageObjects.dashboard.clickEdit();
-      await PageObjects.header.setAbsoluteRange('2013-10-19 06:31:44.000', '2013-12-19 06:31:44.000');
-      await PageObjects.dashboard.clickCancelOutOfEditMode();
-
-      const isOpen = await PageObjects.common.isConfirmModalOpen();
-      expect(isOpen).to.be(false);
-    });
-
-    bdd.it('when a dashboard has a filter and remains unchanged', async function () {
-      await PageObjects.dashboard.clickEdit();
-      await PageObjects.dashboard.setTimepickerInDataRange();
-      await PageObjects.dashboard.filterOnPieSlice();
-      await PageObjects.dashboard.saveDashboard(dashboardName);
-      await PageObjects.dashboard.clickEdit();
-      await PageObjects.dashboard.clickCancelOutOfEditMode();
-
-      const isOpen = await PageObjects.common.isConfirmModalOpen();
-      expect(isOpen).to.be(false);
-    });
-
-    // See https://github.com/elastic/kibana/issues/10110 - this is intentional.
-    bdd.it('when the query is edited but not applied', async function () {
-      await PageObjects.dashboard.clickEdit();
-
-      const originalQuery = await PageObjects.dashboard.getQuery();
-      await PageObjects.dashboard.appendQuery('extra stuff');
-
-      await PageObjects.dashboard.clickCancelOutOfEditMode();
-
-      const isOpen = await PageObjects.common.isConfirmModalOpen();
-      expect(isOpen).to.be(false);
-
-      await PageObjects.dashboard.loadSavedDashboard(dashboardName);
-      const query = await PageObjects.dashboard.getQuery();
-      expect(query).to.equal(originalQuery);
-    });
-  });
-});
+}

test/functional/apps/dashboard/index.js

@@ -1,19 +1,15 @@
+export default function ({ getService, loadTestFile }) {
+  const config = getService('config');
+  const remote = getService('remote');
 
-import {
-  bdd,
-  remote,
-  defaultTimeout,
-} from '../../../support';
+  describe('dashboard app', function () {
+    this.timeout(config.get('timeouts.test'));
 
-bdd.describe('dashboard app', function () {
-  this.timeout = defaultTimeout;
+    before(() => remote.setWindowSize(1200,800));
 
-  bdd.before(function () {
-    return remote.setWindowSize(1200,800);
+    loadTestFile(require.resolve('./_view_edit'));
+    loadTestFile(require.resolve('./_dashboard'));
+    loadTestFile(require.resolve('./_dashboard_save'));
+    loadTestFile(require.resolve('./_dashboard_time'));
   });
-
-  require('./_view_edit');
-  require('./_dashboard');
-  require('./_dashboard_save');
-  require('./_dashboard_time');
-});
+}

test/functional/apps/discover/_collapse_expand.js

@@ -1,72 +1,73 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esArchiver,
-  esClient,
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const log = getService('log');
+  const esArchiver = getService('esArchiver');
+  const kibanaServer = getService('kibanaServer');
+  const PageObjects = getPageObjects(['common', 'discover', 'header']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('discover tab', function describeIndexTests() {
+    before(function () {
+      const fromTime = '2015-09-19 06:31:44.000';
+      const toTime = '2015-09-23 18:31:44.000';
 
-bdd.describe('discover tab', function describeIndexTests() {
-  bdd.before(function () {
-    const fromTime = '2015-09-19 06:31:44.000';
-    const toTime = '2015-09-23 18:31:44.000';
+      // delete .kibana index and update configDoc
+      return kibanaServer.uiSettings.replace({
+        'dateFormat:tz':'UTC',
+        'defaultIndex':'logstash-*'
+      })
+      .then(function loadkibanaIndexPattern() {
+        log.debug('load kibana index with default index pattern');
+        return esArchiver.load('discover');
+      })
+      // and load a set of makelogs data
+      .then(function loadIfEmptyMakelogs() {
+        return esArchiver.loadIfNeeded('logstash_functional');
+      })
+      .then(function () {
+        log.debug('discover');
+        return PageObjects.common.navigateToApp('discover');
+      })
+      .then(function () {
+        log.debug('setAbsoluteRange');
+        return PageObjects.header.setAbsoluteRange(fromTime, toTime);
+      });
+    });
 
-    // delete .kibana index and update configDoc
-    return esClient.deleteAndUpdateConfigDoc({ 'dateFormat:tz':'UTC', 'defaultIndex':'logstash-*' })
-    .then(function loadkibanaIndexPattern() {
-      PageObjects.common.debug('load kibana index with default index pattern');
-      return esArchiver.load('discover');
-    })
-    // and load a set of makelogs data
-    .then(function loadIfEmptyMakelogs() {
-      return esArchiver.loadIfNeeded('logstash_functional');
-    })
-    .then(function () {
-      PageObjects.common.debug('discover');
-      return PageObjects.common.navigateToApp('discover');
-    })
-    .then(function () {
-      PageObjects.common.debug('setAbsoluteRange');
-      return PageObjects.header.setAbsoluteRange(fromTime, toTime);
+    describe('field data', function () {
+      it('should initially be expanded', function () {
+        PageObjects.common.saveScreenshot('Discover-sidebar-expanded');
+        return PageObjects.discover.getSidebarWidth()
+          .then(function (width) {
+            log.debug('expanded sidebar width = ' + width);
+            expect(width > 20).to.be(true);
+          });
+      });
+
+      it('should collapse when clicked', function () {
+        return PageObjects.discover.toggleSidebarCollapse()
+          .then(function () {
+            PageObjects.common.saveScreenshot('Discover-sidebar-collapsed');
+            log.debug('PageObjects.discover.getSidebarWidth()');
+            return PageObjects.discover.getSidebarWidth();
+          })
+          .then(function (width) {
+            log.debug('collapsed sidebar width = ' + width);
+            expect(width < 20).to.be(true);
+          });
+      });
+
+      it('should expand when clicked', function () {
+        return PageObjects.discover.toggleSidebarCollapse()
+          .then(function () {
+            log.debug('PageObjects.discover.getSidebarWidth()');
+            return PageObjects.discover.getSidebarWidth();
+          })
+          .then(function (width) {
+            log.debug('expanded sidebar width = ' + width);
+            expect(width > 20).to.be(true);
+          });
+      });
     });
   });
-
-  bdd.describe('field data', function () {
-    bdd.it('should initially be expanded', function () {
-      PageObjects.common.saveScreenshot('Discover-sidebar-expanded');
-      return PageObjects.discover.getSidebarWidth()
-        .then(function (width) {
-          PageObjects.common.debug('expanded sidebar width = ' + width);
-          expect(width > 20).to.be(true);
-        });
-    });
-
-    bdd.it('should collapse when clicked', function () {
-      return PageObjects.discover.toggleSidebarCollapse()
-        .then(function () {
-          PageObjects.common.saveScreenshot('Discover-sidebar-collapsed');
-          PageObjects.common.debug('PageObjects.discover.getSidebarWidth()');
-          return PageObjects.discover.getSidebarWidth();
-        })
-        .then(function (width) {
-          PageObjects.common.debug('collapsed sidebar width = ' + width);
-          expect(width < 20).to.be(true);
-        });
-    });
-
-    bdd.it('should expand when clicked', function () {
-      return PageObjects.discover.toggleSidebarCollapse()
-        .then(function () {
-          PageObjects.common.debug('PageObjects.discover.getSidebarWidth()');
-          return PageObjects.discover.getSidebarWidth();
-        })
-        .then(function (width) {
-          PageObjects.common.debug('expanded sidebar width = ' + width);
-          expect(width > 20).to.be(true);
-        });
-    });
-  });
-});
+}

test/functional/apps/discover/_discover.js

@@ -1,242 +1,245 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esArchiver,
-  esClient,
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const log = getService('log');
+  const retry = getService('retry');
+  const esArchiver = getService('esArchiver');
+  const remote = getService('remote');
+  const kibanaServer = getService('kibanaServer');
+  const PageObjects = getPageObjects(['common', 'discover', 'header']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('discover app', function describeIndexTests() {
+    const fromTime = '2015-09-19 06:31:44.000';
+    const fromTimeString = 'September 19th 2015, 06:31:44.000';
+    const toTime = '2015-09-23 18:31:44.000';
+    const toTimeString = 'September 23rd 2015, 18:31:44.000';
 
-bdd.describe('discover app', function describeIndexTests() {
-  const fromTime = '2015-09-19 06:31:44.000';
-  const fromTimeString = 'September 19th 2015, 06:31:44.000';
-  const toTime = '2015-09-23 18:31:44.000';
-  const toTimeString = 'September 23rd 2015, 18:31:44.000';
-
-  bdd.before(async function () {
-    // delete .kibana index and update configDoc
-    await esClient.deleteAndUpdateConfigDoc({ 'dateFormat:tz':'UTC', 'defaultIndex':'logstash-*' });
-    PageObjects.common.debug('load kibana index with default index pattern');
-    await esArchiver.load('discover');
-
-    // and load a set of makelogs data
-    await esArchiver.loadIfNeeded('logstash_functional');
-    PageObjects.common.debug('discover');
-    await PageObjects.common.navigateToApp('discover');
-    PageObjects.common.debug('setAbsoluteRange');
-    await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-  });
-
-  bdd.describe('query', function () {
-    const queryName1 = 'Query # 1';
-
-    bdd.it('should show correct time range string by timepicker', async function () {
-      const actualTimeString = await PageObjects.discover.getTimespanText();
-
-      const expectedTimeString = `${fromTimeString} to ${toTimeString}`;
-      expect(actualTimeString).to.be(expectedTimeString);
-    });
-
-    bdd.it('save query should show toast message and display query name', async function () {
-      await PageObjects.discover.saveSearch(queryName1);
-      const toastMessage = await PageObjects.header.getToastMessage();
-
-      const expectedToastMessage = `Discover: Saved Data Source "${queryName1}"`;
-      expect(toastMessage).to.be(expectedToastMessage);
-      await PageObjects.common.saveScreenshot('Discover-save-query-toast');
-
-      await PageObjects.header.waitForToastMessageGone();
-      const actualQueryNameString = await PageObjects.discover.getCurrentQueryName();
-
-      expect(actualQueryNameString).to.be(queryName1);
-    });
-
-    bdd.it('load query should show query name', async function () {
-      await PageObjects.discover.loadSavedSearch(queryName1);
-
-      await PageObjects.common.try(async function() {
-        expect(await PageObjects.discover.getCurrentQueryName()).to.be(queryName1);
+    before(async function () {
+      // delete .kibana index and update configDoc
+      await kibanaServer.uiSettings.replace({
+        'dateFormat:tz':'UTC',
+        'defaultIndex':'logstash-*'
       });
-      await PageObjects.common.saveScreenshot('Discover-load-query');
+
+      log.debug('load kibana index with default index pattern');
+      await esArchiver.load('discover');
+
+      // and load a set of makelogs data
+      await esArchiver.loadIfNeeded('logstash_functional');
+      log.debug('discover');
+      await PageObjects.common.navigateToApp('discover');
+      log.debug('setAbsoluteRange');
+      await PageObjects.header.setAbsoluteRange(fromTime, toTime);
     });
 
-    bdd.it('should show the correct hit count', async function () {
-      const expectedHitCount = '14,004';
-      await PageObjects.common.try(async function() {
-        expect(await PageObjects.discover.getHitCount()).to.be(expectedHitCount);
+    describe('query', function () {
+      const queryName1 = 'Query # 1';
+
+      it('should show correct time range string by timepicker', async function () {
+        const actualTimeString = await PageObjects.discover.getTimespanText();
+
+        const expectedTimeString = `${fromTimeString} to ${toTimeString}`;
+        expect(actualTimeString).to.be(expectedTimeString);
       });
-    });
 
-    bdd.it('should show the correct bar chart', async function () {
-      const expectedBarChartData = [ 35, 189, 694, 1347, 1285, 704, 176, 29, 39, 189, 640,
-        1276, 1327, 663, 166, 25, 30, 164, 663, 1320, 1270, 681, 188, 27 ];
-      await verifyChartData(expectedBarChartData);
-    });
+      it('save query should show toast message and display query name', async function () {
+        await PageObjects.discover.saveSearch(queryName1);
+        const toastMessage = await PageObjects.header.getToastMessage();
 
-    bdd.it('should show correct time range string in chart', async function () {
-      const actualTimeString = await PageObjects.discover.getChartTimespan();
+        const expectedToastMessage = `Discover: Saved Data Source "${queryName1}"`;
+        expect(toastMessage).to.be(expectedToastMessage);
+        await PageObjects.common.saveScreenshot('Discover-save-query-toast');
 
-      const expectedTimeString = `${fromTimeString} - ${toTimeString}`;
-      expect(actualTimeString).to.be(expectedTimeString);
-    });
+        await PageObjects.header.waitForToastMessageGone();
+        const actualQueryNameString = await PageObjects.discover.getCurrentQueryName();
 
-    bdd.it('should show correct initial chart interval of Auto', async function () {
-      const actualInterval = await PageObjects.discover.getChartInterval();
+        expect(actualQueryNameString).to.be(queryName1);
+      });
 
-      const expectedInterval = 'Auto';
-      expect(actualInterval).to.be(expectedInterval);
-    });
+      it('load query should show query name', async function () {
+        await PageObjects.discover.loadSavedSearch(queryName1);
 
-    bdd.it('should show correct data for chart interval Hourly', async function () {
-      await PageObjects.discover.setChartInterval('Hourly');
+        await retry.try(async function() {
+          expect(await PageObjects.discover.getCurrentQueryName()).to.be(queryName1);
+        });
+        await PageObjects.common.saveScreenshot('Discover-load-query');
+      });
 
-      const expectedBarChartData = [ 4, 7, 16, 23, 38, 87, 132, 159, 248, 320, 349, 376, 380,
-        324, 293, 233, 188, 125, 69, 40, 28, 17, 2, 3, 8, 10, 12, 28, 36, 84, 111, 157, 229, 292,
-        324, 373, 378, 345, 306, 223, 167, 124, 72, 35, 22, 11, 7, 1, 6, 5, 12, 25, 27, 76, 111, 175,
-        228, 294, 358, 372, 366, 344, 276, 213, 201, 113, 72, 39, 36, 12, 7, 3 ];
-      await verifyChartData(expectedBarChartData);
-    });
+      it('should show the correct hit count', async function () {
+        const expectedHitCount = '14,004';
+        await retry.try(async function() {
+          expect(await PageObjects.discover.getHitCount()).to.be(expectedHitCount);
+        });
+      });
 
-    bdd.it('should show correct data for chart interval Daily', async function () {
-      const chartInterval = 'Daily';
-      const expectedBarChartData = [ 4757, 4614, 4633 ];
-      await PageObjects.discover.setChartInterval(chartInterval);
-      await PageObjects.common.try(async () => {
+      it('should show the correct bar chart', async function () {
+        const expectedBarChartData = [ 35, 189, 694, 1347, 1285, 704, 176, 29, 39, 189, 640,
+          1276, 1327, 663, 166, 25, 30, 164, 663, 1320, 1270, 681, 188, 27 ];
         await verifyChartData(expectedBarChartData);
       });
-    });
 
-    bdd.it('should show correct data for chart interval Weekly', async function () {
-      const chartInterval = 'Weekly';
-      const expectedBarChartData = [ 4757, 9247 ];
+      it('should show correct time range string in chart', async function () {
+        const actualTimeString = await PageObjects.discover.getChartTimespan();
 
-      await PageObjects.discover.setChartInterval(chartInterval);
-      await PageObjects.common.try(async () => {
+        const expectedTimeString = `${fromTimeString} - ${toTimeString}`;
+        expect(actualTimeString).to.be(expectedTimeString);
+      });
+
+      it('should show correct initial chart interval of Auto', async function () {
+        const actualInterval = await PageObjects.discover.getChartInterval();
+
+        const expectedInterval = 'Auto';
+        expect(actualInterval).to.be(expectedInterval);
+      });
+
+      it('should show correct data for chart interval Hourly', async function () {
+        await PageObjects.discover.setChartInterval('Hourly');
+
+        const expectedBarChartData = [ 4, 7, 16, 23, 38, 87, 132, 159, 248, 320, 349, 376, 380,
+          324, 293, 233, 188, 125, 69, 40, 28, 17, 2, 3, 8, 10, 12, 28, 36, 84, 111, 157, 229, 292,
+          324, 373, 378, 345, 306, 223, 167, 124, 72, 35, 22, 11, 7, 1, 6, 5, 12, 25, 27, 76, 111, 175,
+          228, 294, 358, 372, 366, 344, 276, 213, 201, 113, 72, 39, 36, 12, 7, 3 ];
         await verifyChartData(expectedBarChartData);
       });
-    });
 
-    bdd.it('browser back button should show previous interval Daily', async function () {
-      const expectedChartInterval = 'Daily';
-      const expectedBarChartData = [ 4757, 4614, 4633 ];
+      it('should show correct data for chart interval Daily', async function () {
+        const chartInterval = 'Daily';
+        const expectedBarChartData = [ 4757, 4614, 4633 ];
+        await PageObjects.discover.setChartInterval(chartInterval);
+        await retry.try(async () => {
+          await verifyChartData(expectedBarChartData);
+        });
+      });
+
+      it('should show correct data for chart interval Weekly', async function () {
+        const chartInterval = 'Weekly';
+        const expectedBarChartData = [ 4757, 9247 ];
+
+        await PageObjects.discover.setChartInterval(chartInterval);
+        await retry.try(async () => {
+          await verifyChartData(expectedBarChartData);
+        });
+      });
+
+      it('browser back button should show previous interval Daily', async function () {
+        const expectedChartInterval = 'Daily';
+        const expectedBarChartData = [ 4757, 4614, 4633 ];
+
+        await remote.goBack();
+        await retry.try(async function tryingForTime() {
+          const actualInterval = await PageObjects.discover.getChartInterval();
+          expect(actualInterval).to.be(expectedChartInterval);
+        });
+        await verifyChartData(expectedBarChartData);
+      });
+
+      it('should show correct data for chart interval Monthly', async function () {
+        const chartInterval = 'Monthly';
+        const expectedBarChartData = [ 13129 ];
+
+        await PageObjects.discover.setChartInterval(chartInterval);
+        await verifyChartData(expectedBarChartData);
+      });
+
+      it('should show correct data for chart interval Yearly', async function () {
+        const chartInterval = 'Yearly';
+        const expectedBarChartData = [ 13129 ];
+
+        await PageObjects.discover.setChartInterval(chartInterval);
+        await verifyChartData(expectedBarChartData);
+      });
+
+      it('should show correct data for chart interval Auto', async function () {
+        const chartInterval = 'Auto';
+        const expectedBarChartData = [ 35, 189, 694, 1347, 1285, 704, 176, 29, 39, 189,
+          640, 1276, 1327, 663, 166, 25, 30, 164, 663, 1320, 1270, 681, 188, 27 ];
+
+        await PageObjects.discover.setChartInterval(chartInterval);
+        await verifyChartData(expectedBarChartData);
+      });
+
+      it('should show Auto chart interval', async function () {
+        const expectedChartInterval = 'Auto';
 
-      await this.remote.goBack();
-      await PageObjects.common.try(async function tryingForTime() {
         const actualInterval = await PageObjects.discover.getChartInterval();
         expect(actualInterval).to.be(expectedChartInterval);
       });
-      await verifyChartData(expectedBarChartData);
-    });
 
-    bdd.it('should show correct data for chart interval Monthly', async function () {
-      const chartInterval = 'Monthly';
-      const expectedBarChartData = [ 13129 ];
-
-      await PageObjects.discover.setChartInterval(chartInterval);
-      await verifyChartData(expectedBarChartData);
-    });
-
-    bdd.it('should show correct data for chart interval Yearly', async function () {
-      const chartInterval = 'Yearly';
-      const expectedBarChartData = [ 13129 ];
-
-      await PageObjects.discover.setChartInterval(chartInterval);
-      await verifyChartData(expectedBarChartData);
-    });
-
-    bdd.it('should show correct data for chart interval Auto', async function () {
-      const chartInterval = 'Auto';
-      const expectedBarChartData = [ 35, 189, 694, 1347, 1285, 704, 176, 29, 39, 189,
-        640, 1276, 1327, 663, 166, 25, 30, 164, 663, 1320, 1270, 681, 188, 27 ];
-
-      await PageObjects.discover.setChartInterval(chartInterval);
-      await verifyChartData(expectedBarChartData);
-    });
-
-    bdd.it('should show Auto chart interval', async function () {
-      const expectedChartInterval = 'Auto';
-
-      const actualInterval = await PageObjects.discover.getChartInterval();
-      expect(actualInterval).to.be(expectedChartInterval);
-    });
-
-    bdd.it('should not show "no results"', async () => {
-      const isVisible = await PageObjects.discover.hasNoResults();
-      expect(isVisible).to.be(false);
-    });
-
-    async function verifyChartData(expectedBarChartData) {
-      await PageObjects.common.try(async function tryingForTime() {
-        const paths = await PageObjects.discover.getBarChartData();
-        // the largest bars are over 100 pixels high so this is less than 1% tolerance
-        const barHeightTolerance = 1;
-        let stringResults = '';
-        let hasFailure = false;
-        for (let y = 0; y < expectedBarChartData.length; y++) {
-          stringResults += y + ': expected = ' + expectedBarChartData[y] + ', actual = ' + paths[y] +
-           ', Pass = ' + (Math.abs(expectedBarChartData[y] - paths[y]) < barHeightTolerance) + '\n';
-          if ((Math.abs(expectedBarChartData[y] - paths[y]) > barHeightTolerance)) {
-            hasFailure = true;
-          }
-        }
-        if (hasFailure) {
-          PageObjects.common.log(stringResults);
-          PageObjects.common.log(paths);
-        }
-        for (let x = 0; x < expectedBarChartData.length; x++) {
-          expect(Math.abs(expectedBarChartData[x] - paths[x]) < barHeightTolerance).to.be.ok();
-        }
+      it('should not show "no results"', async () => {
+        const isVisible = await PageObjects.discover.hasNoResults();
+        expect(isVisible).to.be(false);
       });
-    }
-  });
 
-  bdd.describe('query #2, which has an empty time range', function () {
-    const fromTime = '1999-06-11 09:22:11.000';
-    const toTime = '1999-06-12 11:21:04.000';
-
-    bdd.before(() => {
-      PageObjects.common.debug('setAbsoluteRangeForAnotherQuery');
-      return PageObjects.header
-        .setAbsoluteRange(fromTime, toTime);
+      async function verifyChartData(expectedBarChartData) {
+        await retry.try(async function tryingForTime() {
+          const paths = await PageObjects.discover.getBarChartData();
+          // the largest bars are over 100 pixels high so this is less than 1% tolerance
+          const barHeightTolerance = 1;
+          let stringResults = '';
+          let hasFailure = false;
+          for (let y = 0; y < expectedBarChartData.length; y++) {
+            stringResults += y + ': expected = ' + expectedBarChartData[y] + ', actual = ' + paths[y] +
+             ', Pass = ' + (Math.abs(expectedBarChartData[y] - paths[y]) < barHeightTolerance) + '\n';
+            if ((Math.abs(expectedBarChartData[y] - paths[y]) > barHeightTolerance)) {
+              hasFailure = true;
+            }
+          }
+          if (hasFailure) {
+            log.debug(stringResults);
+            log.debug(paths);
+          }
+          for (let x = 0; x < expectedBarChartData.length; x++) {
+            expect(Math.abs(expectedBarChartData[x] - paths[x]) < barHeightTolerance).to.be.ok();
+          }
+        });
+      }
     });
 
-    bdd.it('should show "no results"', async () => {
-      const isVisible = await PageObjects.discover.hasNoResults();
-      expect(isVisible).to.be(true);
-      await PageObjects.common.saveScreenshot('Discover-no-results');
+    describe('query #2, which has an empty time range', function () {
+      const fromTime = '1999-06-11 09:22:11.000';
+      const toTime = '1999-06-12 11:21:04.000';
+
+      before(() => {
+        log.debug('setAbsoluteRangeForAnotherQuery');
+        return PageObjects.header
+          .setAbsoluteRange(fromTime, toTime);
+      });
+
+      it('should show "no results"', async () => {
+        const isVisible = await PageObjects.discover.hasNoResults();
+        expect(isVisible).to.be(true);
+        await PageObjects.common.saveScreenshot('Discover-no-results');
+      });
+
+      it('should suggest a new time range is picked', async () => {
+        const isVisible = await PageObjects.discover.hasNoResultsTimepicker();
+        expect(isVisible).to.be(true);
+      });
+
+      it('should have a link that opens and closes the time picker', async function() {
+        const noResultsTimepickerLink = await PageObjects.discover.getNoResultsTimepicker();
+        expect(await PageObjects.header.isTimepickerOpen()).to.be(false);
+
+        await noResultsTimepickerLink.click();
+        expect(await PageObjects.header.isTimepickerOpen()).to.be(true);
+
+        await noResultsTimepickerLink.click();
+        expect(await PageObjects.header.isTimepickerOpen()).to.be(false);
+      });
     });
 
-    bdd.it('should suggest a new time range is picked', async () => {
-      const isVisible = await PageObjects.discover.hasNoResultsTimepicker();
-      expect(isVisible).to.be(true);
-    });
+    describe('data-shared-item', function () {
+      it('should have correct data-shared-item title and description', async () => {
+        const expected = {
+          title: 'A Saved Search',
+          description: 'A Saved Search Description'
+        };
 
-    bdd.it('should have a link that opens and closes the time picker', async function() {
-      const noResultsTimepickerLink = await PageObjects.discover.getNoResultsTimepicker();
-      expect(await PageObjects.header.isTimepickerOpen()).to.be(false);
-
-      await noResultsTimepickerLink.click();
-      expect(await PageObjects.header.isTimepickerOpen()).to.be(true);
-
-      await noResultsTimepickerLink.click();
-      expect(await PageObjects.header.isTimepickerOpen()).to.be(false);
+        await PageObjects.discover.loadSavedSearch(expected.title);
+        const { title, description } = await PageObjects.common.getSharedItemTitleAndDescription();
+        expect(title).to.eql(expected.title);
+        expect(description).to.eql(expected.description);
+      });
     });
   });
-
-
-  bdd.describe('data-shared-item', function () {
-    bdd.it('should have correct data-shared-item title and description', async () => {
-      const expected = {
-        title: 'A Saved Search',
-        description: 'A Saved Search Description'
-      };
-
-      await PageObjects.discover.loadSavedSearch(expected.title);
-      const { title, description } = await PageObjects.common.getSharedItemTitleAndDescription();
-      expect(title).to.eql(expected.title);
-      expect(description).to.eql(expected.description);
-    });
-  });
-});
+}

test/functional/apps/discover/_field_data.js

@@ -1,242 +1,244 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esArchiver,
-  esClient,
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const log = getService('log');
+  const retry = getService('retry');
+  const esArchiver = getService('esArchiver');
+  const kibanaServer = getService('kibanaServer');
+  const PageObjects = getPageObjects(['common', 'header', 'discover']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('discover app', function describeIndexTests() {
+    before(function () {
+      const fromTime = '2015-09-19 06:31:44.000';
+      const toTime = '2015-09-23 18:31:44.000';
 
-bdd.describe('discover app', function describeIndexTests() {
-  bdd.before(function () {
-    const fromTime = '2015-09-19 06:31:44.000';
-    const toTime = '2015-09-23 18:31:44.000';
-
-    // delete .kibana index and update configDoc
-    return esClient.deleteAndUpdateConfigDoc({ 'dateFormat:tz':'UTC', 'defaultIndex':'logstash-*' })
-    .then(function loadkibanaIndexPattern() {
-      PageObjects.common.debug('load kibana index with default index pattern');
-      return esArchiver.load('discover');
-    })
-    // and load a set of makelogs data
-    .then(function loadIfEmptyMakelogs() {
-      return esArchiver.loadIfNeeded('logstash_functional');
-    })
-    .then(function () {
-      PageObjects.common.debug('discover');
-      return PageObjects.common.navigateToApp('discover');
-    })
-    .then(function () {
-      PageObjects.common.debug('setAbsoluteRange');
-      return PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    });
-  });
-
-
-  bdd.describe('field data', function () {
-    bdd.it('search php should show the correct hit count', function () {
-      const expectedHitCount = '445';
-      return PageObjects.discover.query('php')
+      // delete .kibana index and update configDoc
+      return kibanaServer.uiSettings.replace({
+        'dateFormat:tz': 'UTC',
+        'defaultIndex': 'logstash-*'
+      })
+      .then(function loadkibanaIndexPattern() {
+        log.debug('load kibana index with default index pattern');
+        return esArchiver.load('discover');
+      })
+      // and load a set of makelogs data
+      .then(function loadIfEmptyMakelogs() {
+        return esArchiver.loadIfNeeded('logstash_functional');
+      })
       .then(function () {
-        return PageObjects.common.try(function tryingForTime() {
-          return PageObjects.discover.getHitCount()
-          .then(function compareData(hitCount) {
-            PageObjects.common.saveScreenshot('Discover-field-data');
-            expect(hitCount).to.be(expectedHitCount);
+        log.debug('discover');
+        return PageObjects.common.navigateToApp('discover');
+      })
+      .then(function () {
+        log.debug('setAbsoluteRange');
+        return PageObjects.header.setAbsoluteRange(fromTime, toTime);
+      });
+    });
+
+
+    describe('field data', function () {
+      it('search php should show the correct hit count', function () {
+        const expectedHitCount = '445';
+        return PageObjects.discover.query('php')
+        .then(function () {
+          return retry.try(function tryingForTime() {
+            return PageObjects.discover.getHitCount()
+            .then(function compareData(hitCount) {
+              PageObjects.common.saveScreenshot('Discover-field-data');
+              expect(hitCount).to.be(expectedHitCount);
+            });
           });
         });
       });
-    });
 
-    bdd.it('the search term should be highlighted in the field data', function () {
-      // marks is the style that highlights the text in yellow
-      return PageObjects.discover.getMarks()
-      .then(function (marks) {
-        expect(marks.length).to.be(50);
-        expect(marks.indexOf('php')).to.be(0);
+      it('the search term should be highlighted in the field data', function () {
+        // marks is the style that highlights the text in yellow
+        return PageObjects.discover.getMarks()
+        .then(function (marks) {
+          expect(marks.length).to.be(50);
+          expect(marks.indexOf('php')).to.be(0);
+        });
       });
-    });
 
-    bdd.it('search _type:apache should show the correct hit count', function () {
-      const expectedHitCount = '11,156';
-      return PageObjects.discover.query('_type:apache')
-      .then(function () {
-        return PageObjects.common.try(function tryingForTime() {
-          return PageObjects.discover.getHitCount()
-          .then(function compareData(hitCount) {
-            expect(hitCount).to.be(expectedHitCount);
+      it('search _type:apache should show the correct hit count', function () {
+        const expectedHitCount = '11,156';
+        return PageObjects.discover.query('_type:apache')
+        .then(function () {
+          return retry.try(function tryingForTime() {
+            return PageObjects.discover.getHitCount()
+            .then(function compareData(hitCount) {
+              expect(hitCount).to.be(expectedHitCount);
+            });
           });
         });
       });
-    });
 
-    bdd.it('doc view should show Time and _source columns', function () {
-      const expectedHeader = 'Time _source';
-      return PageObjects.discover.getDocHeader()
-      .then(function (header) {
-        expect(header).to.be(expectedHeader);
+      it('doc view should show Time and _source columns', function () {
+        const expectedHeader = 'Time _source';
+        return PageObjects.discover.getDocHeader()
+        .then(function (header) {
+          expect(header).to.be(expectedHeader);
+        });
       });
-    });
 
-    bdd.it('doc view should show oldest time first', function () {
-      // Note: Could just check the timestamp, but might as well check that the whole doc is as expected.
-      const ExpectedDoc =
-        'September 22nd 2015, 23:50:13.253 index:logstash-2015.09.22 @timestamp:September 22nd 2015, 23:50:13.253'
-        + ' ip:238.171.34.42 extension:jpg response:200 geo.coordinates:{ "lat": 38.66494528, "lon": -88.45299556'
-        + ' } geo.src:FR geo.dest:KH geo.srcdest:FR:KH @tags:success, info utc_time:September 22nd 2015,'
-        + ' 23:50:13.253 referer:http://twitter.com/success/nancy-currie agent:Mozilla/4.0 (compatible; MSIE 6.0;'
-        + ' Windows NT 5.1; SV1; .NET CLR 1.1.4322) clientip:238.171.34.42 bytes:7,124'
-        + ' host:media-for-the-masses.theacademyofperformingartsandscience.org request:/uploads/karl-henize.jpg'
-        + ' url:https://media-for-the-masses.theacademyofperformingartsandscience.org/uploads/karl-henize.jpg'
-        + ' @message:238.171.34.42 - - [2015-09-22T23:50:13.253Z] "GET /uploads/karl-henize.jpg HTTP/1.1" 200 7124'
-        + ' "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" spaces:this is a'
-        + ' thing with lots of spaces wwwwoooooo xss:<script>console.log("xss")</script>'
-        + ' headings:<h3>alexander-viktorenko</h5>, http://nytimes.com/warning/michael-massimino'
-        + ' links:@www.slate.com, http://www.slate.com/security/frederick-w-leslie, www.www.slate.com'
-        + ' relatedContent:{ "url": "http://www.laweekly.com/music/bjork-at-the-nokia-theatre-12-12-2408191",'
-        + ' "og:type": "article", "og:title": "Bjork at the Nokia Theatre, 12/12", "og:description": "Bjork at the'
-        + ' Nokia Theater, December 12 By Randall Roberts Last night&rsquo;s Bjork show at the Dystopia &ndash;'
-        + ' er, I mean Nokia -- Theatre downtown di...", "og:url": "'
-        + 'http://www.laweekly.com/music/bjork-at-the-nokia-theatre-12-12-2408191", "article:published_time":'
-        + ' "2007-12-13T12:19:35-08:00", "article:modified_time": "2014-11-27T08:28:42-08:00", "article:section":'
-        + ' "Music", "og:image": "'
-        + 'http://IMAGES1.laweekly.com/imager/bjork-at-the-nokia-theatre-12-12/u/original/2470701/bjorktn003.jpg",'
-        + ' "og:image:height": "334", "og:image:width": "480", "og:site_name": "LA Weekly", "twitter:title":'
-        + ' "Bjork at the Nokia Theatre, 12/12", "twitter:description": "Bjork at the Nokia Theater, December 12'
-        + ' By Randall Roberts Last night&rsquo;s Bjork show at the Dystopia &ndash; er, I mean Nokia -- Theatre'
-        + ' downtown di...", "twitter:card": "summary", "twitter:image": "'
-        + 'http://IMAGES1.laweekly.com/imager/bjork-at-the-nokia-theatre-12-12/u/original/2470701/bjorktn003.jpg",'
-        + ' "twitter:site": "@laweekly" }, { "url": "'
-        + 'http://www.laweekly.com/music/the-rapture-at-the-mayan-7-25-2401011", "og:type": "article", "og:title":'
-        + ' "The Rapture at the Mayan, 7/25", "og:description": "If you haven&rsquo;t yet experienced the'
-        + ' phenomenon of people walk-dancing, apparently the best place to witness this is at a Rapture show.'
-        + ' Here&rsquo;s...", "og:url": "http://www.laweekly.com/music/the-rapture-at-the-mayan-7-25-2401011",'
-        + ' "article:published_time": "2007-07-26T12:42:30-07:00", "article:modified_time":'
-        + ' "2014-11-27T08:00:51-08:00", "article:section": "Music", "og:image": "'
-        + 'http://IMAGES1.laweekly.com/imager/the-rapture-at-the-mayan-7-25/u/original/2463272/rapturetn05.jpg",'
-        + ' "og:image:height": "321", "og:image:width": "480", "og:site_name": "LA Weekly", "twitter:title": "The'
-        + ' Rapture at the Mayan, 7/25", "twitter:description": "If you haven&rsquo;t yet experienced the'
-        + ' phenomenon of people walk-dancing, apparently the best place to witness this is at a Rapture show.'
-        + ' Here&rsquo;s...", "twitter:card": "summary", "twitter:image": "'
-        + 'http://IMAGES1.laweekly.com/imager/the-rapture-at-the-mayan-7-25/u/original/2463272/rapturetn05.jpg",'
-        + ' "twitter:site": "@laweekly" } machine.os:win 7 machine.ram:7,516,192,768 _id:AU_x3_g4GFA8no6QjkYX'
-        + ' _type:apache _index:logstash-2015.09.22 _score: - relatedContent.article:modified_time:November 27th'
-        + ' 2014, 16:00:51.000, November 27th 2014, 16:28:42.000 relatedContent.article:published_time:July 26th'
-        + ' 2007, 19:42:30.000, December 13th 2007, 20:19:35.000';
-      return PageObjects.discover.getDocTableIndex(1)
-      .then(function (rowData) {
-        expect(rowData).to.be(ExpectedDoc);
+      it('doc view should show oldest time first', function () {
+        // Note: Could just check the timestamp, but might as well check that the whole doc is as expected.
+        const ExpectedDoc =
+          'September 22nd 2015, 23:50:13.253 index:logstash-2015.09.22 @timestamp:September 22nd 2015, 23:50:13.253'
+          + ' ip:238.171.34.42 extension:jpg response:200 geo.coordinates:{ "lat": 38.66494528, "lon": -88.45299556'
+          + ' } geo.src:FR geo.dest:KH geo.srcdest:FR:KH @tags:success, info utc_time:September 22nd 2015,'
+          + ' 23:50:13.253 referer:http://twitter.com/success/nancy-currie agent:Mozilla/4.0 (compatible; MSIE 6.0;'
+          + ' Windows NT 5.1; SV1; .NET CLR 1.1.4322) clientip:238.171.34.42 bytes:7,124'
+          + ' host:media-for-the-masses.theacademyofperformingartsandscience.org request:/uploads/karl-henize.jpg'
+          + ' url:https://media-for-the-masses.theacademyofperformingartsandscience.org/uploads/karl-henize.jpg'
+          + ' @message:238.171.34.42 - - [2015-09-22T23:50:13.253Z] "GET /uploads/karl-henize.jpg HTTP/1.1" 200 7124'
+          + ' "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" spaces:this is a'
+          + ' thing with lots of spaces wwwwoooooo xss:<script>console.log("xss")</script>'
+          + ' headings:<h3>alexander-viktorenko</h5>, http://nytimes.com/warning/michael-massimino'
+          + ' links:@www.slate.com, http://www.slate.com/security/frederick-w-leslie, www.www.slate.com'
+          + ' relatedContent:{ "url": "http://www.laweekly.com/music/bjork-at-the-nokia-theatre-12-12-2408191",'
+          + ' "og:type": "article", "og:title": "Bjork at the Nokia Theatre, 12/12", "og:description": "Bjork at the'
+          + ' Nokia Theater, December 12 By Randall Roberts Last night&rsquo;s Bjork show at the Dystopia &ndash;'
+          + ' er, I mean Nokia -- Theatre downtown di...", "og:url": "'
+          + 'http://www.laweekly.com/music/bjork-at-the-nokia-theatre-12-12-2408191", "article:published_time":'
+          + ' "2007-12-13T12:19:35-08:00", "article:modified_time": "2014-11-27T08:28:42-08:00", "article:section":'
+          + ' "Music", "og:image": "'
+          + 'http://IMAGES1.laweekly.com/imager/bjork-at-the-nokia-theatre-12-12/u/original/2470701/bjorktn003.jpg",'
+          + ' "og:image:height": "334", "og:image:width": "480", "og:site_name": "LA Weekly", "twitter:title":'
+          + ' "Bjork at the Nokia Theatre, 12/12", "twitter:description": "Bjork at the Nokia Theater, December 12'
+          + ' By Randall Roberts Last night&rsquo;s Bjork show at the Dystopia &ndash; er, I mean Nokia -- Theatre'
+          + ' downtown di...", "twitter:card": "summary", "twitter:image": "'
+          + 'http://IMAGES1.laweekly.com/imager/bjork-at-the-nokia-theatre-12-12/u/original/2470701/bjorktn003.jpg",'
+          + ' "twitter:site": "@laweekly" }, { "url": "'
+          + 'http://www.laweekly.com/music/the-rapture-at-the-mayan-7-25-2401011", "og:type": "article", "og:title":'
+          + ' "The Rapture at the Mayan, 7/25", "og:description": "If you haven&rsquo;t yet experienced the'
+          + ' phenomenon of people walk-dancing, apparently the best place to witness this is at a Rapture show.'
+          + ' Here&rsquo;s...", "og:url": "http://www.laweekly.com/music/the-rapture-at-the-mayan-7-25-2401011",'
+          + ' "article:published_time": "2007-07-26T12:42:30-07:00", "article:modified_time":'
+          + ' "2014-11-27T08:00:51-08:00", "article:section": "Music", "og:image": "'
+          + 'http://IMAGES1.laweekly.com/imager/the-rapture-at-the-mayan-7-25/u/original/2463272/rapturetn05.jpg",'
+          + ' "og:image:height": "321", "og:image:width": "480", "og:site_name": "LA Weekly", "twitter:title": "The'
+          + ' Rapture at the Mayan, 7/25", "twitter:description": "If you haven&rsquo;t yet experienced the'
+          + ' phenomenon of people walk-dancing, apparently the best place to witness this is at a Rapture show.'
+          + ' Here&rsquo;s...", "twitter:card": "summary", "twitter:image": "'
+          + 'http://IMAGES1.laweekly.com/imager/the-rapture-at-the-mayan-7-25/u/original/2463272/rapturetn05.jpg",'
+          + ' "twitter:site": "@laweekly" } machine.os:win 7 machine.ram:7,516,192,768 _id:AU_x3_g4GFA8no6QjkYX'
+          + ' _type:apache _index:logstash-2015.09.22 _score: - relatedContent.article:modified_time:November 27th'
+          + ' 2014, 16:00:51.000, November 27th 2014, 16:28:42.000 relatedContent.article:published_time:July 26th'
+          + ' 2007, 19:42:30.000, December 13th 2007, 20:19:35.000';
+        return PageObjects.discover.getDocTableIndex(1)
+        .then(function (rowData) {
+          expect(rowData).to.be(ExpectedDoc);
+        });
       });
-    });
 
-    bdd.it('doc view should sort ascending', function () {
-      // Note: Could just check the timestamp, but might as well check that the whole doc is as expected.
-      const ExpectedDoc =
-        'September 20th 2015, 00:00:00.000 index:logstash-2015.09.20 @timestamp:September 20th 2015, 00:00:00.000'
-        + ' ip:143.84.142.7 extension:jpg response:200 geo.coordinates:{ "lat": 38.68407028, "lon": -120.9871642 }'
-        + ' geo.src:ES geo.dest:US geo.srcdest:ES:US @tags:error, info utc_time:September 20th 2015, 00:00:00.000'
-        + ' referer:http://www.slate.com/success/vladimir-kovalyonok agent:Mozilla/4.0 (compatible; MSIE 6.0;'
-        + ' Windows NT 5.1; SV1; .NET CLR 1.1.4322) clientip:143.84.142.7 bytes:1,623'
-        + ' host:media-for-the-masses.theacademyofperformingartsandscience.org request:/uploads/steven-hawley.jpg'
-        + ' url:https://media-for-the-masses.theacademyofperformingartsandscience.org/uploads/steven-hawley.jpg'
-        + ' @message:143.84.142.7 - - [2015-09-20T00:00:00.000Z] "GET /uploads/steven-hawley.jpg HTTP/1.1" 200'
-        + ' 1623 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" spaces:this is a'
-        + ' thing with lots of spaces wwwwoooooo xss:<script>console.log("xss")</script>'
-        + ' headings:<h3>kimiya-yui</h5>, http://www.slate.com/success/koichi-wakata'
-        + ' links:thomas-marshburn@twitter.com, http://www.slate.com/info/michael-p-anderson, www.twitter.com'
-        + ' relatedContent:{ "url":'
-        + ' "http://www.laweekly.com/music/jay-electronica-much-better-than-his-name-would-suggest-2412364",'
-        + ' "og:type": "article", "og:title": "Jay Electronica: Much Better Than His Name Would Suggest",'
-        + ' "og:description": "You may not know who Jay Electronica is yet, but I&#039;m willing to bet that you'
-        + ' would had he chosen a better name. Jay Electronica does not sound like the ...", "og:url":'
-        + ' "http://www.laweekly.com/music/jay-electronica-much-better-than-his-name-would-suggest-2412364",'
-        + ' "article:published_time": "2008-04-04T16:00:00-07:00", "article:modified_time":'
-        + ' "2014-11-27T08:01:03-08:00", "article:section": "Music", "og:site_name": "LA Weekly", "twitter:title":'
-        + ' "Jay Electronica: Much Better Than His Name Would Suggest", "twitter:description": "You may not know'
-        + ' who Jay Electronica is yet, but I&#039;m willing to bet that you would had he chosen a better name.'
-        + ' Jay Electronica does not sound like the ...", "twitter:card": "summary", "twitter:site": "@laweekly"'
-        + ' }, { "url": "http://www.laweekly.com/news/mandoe-on-gower-near-fountain-2368123", "og:type":'
-        + ' "article", "og:title": "MANDOE On Gower Near Fountain", "og:description": "MANDOE has a stunner on a'
-        + ' wall north of an east-west street crossing Gower around Fountain (but not on Fountain). MADNOE, PROSE'
-        + ' and FUKM are listed on t...", "og:url": "'
-        + 'http://www.laweekly.com/news/mandoe-on-gower-near-fountain-2368123", "article:published_time":'
-        + ' "2008-04-25T07:26:41-07:00", "article:modified_time": "2014-10-28T15:00:08-07:00", "article:section":'
-        + ' "News", "og:image": "'
-        + 'http://images1.laweekly.com/imager/mandoe-on-gower-near-fountain/u/original/2430891/img_6648.jpg",'
-        + ' "og:image:height": "640", "og:image:width": "480", "og:site_name": "LA Weekly", "twitter:title": '
-        + '"MANDOE On Gower Near Fountain", "twitter:description": "MANDOE has a stunner on a wall north of an'
-        + ' east-west street crossing Gower around Fountain (but not on Fountain). MADNOE, PROSE and FUKM are'
-        + ' listed on t...", "twitter:card": "summary", "twitter:image": "'
-        + 'http://images1.laweekly.com/imager/mandoe-on-gower-near-fountain/u/original/2430891/img_6648.jpg", '
-        + '"twitter:site": "@laweekly" }, { "url": "http://www.laweekly.com/arts/meghan-finds-the-love-2373346",'
-        + ' "og:type": "article", "og:title": "Meghan Finds The Love", "og:description": "LA Weekly is the'
-        + ' definitive source of information for news, music, movies, restaurants, reviews, and events in Los'
-        + ' Angeles.", "og:url": "http://www.laweekly.com/arts/meghan-finds-the-love-2373346",'
-        + ' "article:published_time": "2005-10-20T18:10:25-07:00", "article:modified_time":'
-        + ' "2014-11-25T19:52:35-08:00", "article:section": "Arts", "og:site_name": "LA Weekly", "twitter:title":'
-        + ' "Meghan Finds The Love", "twitter:description": "LA Weekly is the definitive source of information for'
-        + ' news, music, movies, restaurants, reviews, and events in Los Angeles.", "twitter:card": "summary",'
-        + ' "twitter:site": "@laweekly" }, { "url": "http://www.laweekly.com/arts/these-clowns-are-with-me-2371051'
-        + '", "og:type": "article", "og:title": "These Clowns Are With Me", "og:description": "&nbsp; &nbsp; I'
-        + ' didn&#039;t mean to blow off all my responsibilities yesterday, but when a schmoozy Hollywood luncheon'
-        + ' turns into a full-on party by 3pm, and...", "og:url": "'
-        + 'http://www.laweekly.com/arts/these-clowns-are-with-me-2371051", "article:published_time": '
-        + '"2006-03-04T17:03:42-08:00", "article:modified_time": "2014-11-25T17:05:47-08:00", "article:section":'
-        + ' "Arts", "og:image": "'
-        + 'http://images1.laweekly.com/imager/these-clowns-are-with-me/u/original/2434556/e4b8scd.jpg",'
-        + ' "og:image:height": "375", "og:image:width": "500", "og:site_name": "LA Weekly", "twitter:title":'
-        + ' "These Clowns Are With Me", "twitter:description": "&nbsp; &nbsp; I didn&#039;t mean to blow off all'
-        + ' my responsibilities yesterday, but when a schmoozy Hollywood luncheon turns into a full-on party by'
-        + ' 3pm, and...", "twitter:card": "summary", "twitter:image": "'
-        + 'http://images1.laweekly.com/imager/these-clowns-are-with-me/u/original/2434556/e4b8scd.jpg",'
-        + ' "twitter:site": "@laweekly" }, { "url": "http://www.laweekly.com/arts/shopping-daze-2373807",'
-        + ' "og:type": "article", "og:title": "Shopping Daze", "og:description": "LA Weekly is the definitive '
-        + 'source of information for news, music, movies, restaurants, reviews, and events in Los Angeles.",'
-        + ' "og:url": "http://www.laweekly.com/arts/shopping-daze-2373807", "article:published_time":'
-        + ' "2006-12-13T12:12:04-08:00", "article:modified_time": "2014-11-25T20:15:21-08:00", "article:section":'
-        + ' "Arts", "og:site_name": "LA Weekly", "twitter:title": "Shopping Daze", "twitter:description": "LA'
-        + ' Weekly is the definitive source of information for news, music, movies, restaurants, reviews, and'
-        + ' events in Los Angeles.", "twitter:card": "summary", "twitter:site": "@laweekly" } machine.os:osx'
-        + ' machine.ram:15,032,385,536 _id:AU_x3_g3GFA8no6QjkFm _type:apache _index:logstash-2015.09.20 _score: -'
-        + ' relatedContent.article:modified_time:October 28th 2014, 22:00:08.000, November 26th 2014,'
-        + ' 01:05:47.000, November 26th 2014, 03:52:35.000, November 26th 2014, 04:15:21.000, November 27th 2014,'
-        + ' 16:01:03.000 relatedContent.article:published_time:October 21st 2005, 01:10:25.000, March 5th 2006,'
-        + ' 01:03:42.000, December 13th 2006, 20:12:04.000, April 4th 2008, 23:00:00.000, April 25th 2008,'
-        + ' 14:26:41.000';
-      return PageObjects.discover.clickDocSortDown()
-      .then(function () {
-        // we don't technically need this sleep here because the tryForTime will retry and the
-        // results will match on the 2nd or 3rd attempt, but that debug output is huge in this
-        // case and it can be avoided with just a few seconds sleep.
-        return PageObjects.common.sleep(2000);
-      })
-      .then(function () {
-        return PageObjects.common.try(function tryingForTime() {
-          return PageObjects.discover.getDocTableIndex(1)
-          .then(function (rowData) {
-            PageObjects.common.saveScreenshot('Discover-sort-down');
-            expect(rowData).to.be(ExpectedDoc);
+      it('doc view should sort ascending', function () {
+        // Note: Could just check the timestamp, but might as well check that the whole doc is as expected.
+        const ExpectedDoc =
+          'September 20th 2015, 00:00:00.000 index:logstash-2015.09.20 @timestamp:September 20th 2015, 00:00:00.000'
+          + ' ip:143.84.142.7 extension:jpg response:200 geo.coordinates:{ "lat": 38.68407028, "lon": -120.9871642 }'
+          + ' geo.src:ES geo.dest:US geo.srcdest:ES:US @tags:error, info utc_time:September 20th 2015, 00:00:00.000'
+          + ' referer:http://www.slate.com/success/vladimir-kovalyonok agent:Mozilla/4.0 (compatible; MSIE 6.0;'
+          + ' Windows NT 5.1; SV1; .NET CLR 1.1.4322) clientip:143.84.142.7 bytes:1,623'
+          + ' host:media-for-the-masses.theacademyofperformingartsandscience.org request:/uploads/steven-hawley.jpg'
+          + ' url:https://media-for-the-masses.theacademyofperformingartsandscience.org/uploads/steven-hawley.jpg'
+          + ' @message:143.84.142.7 - - [2015-09-20T00:00:00.000Z] "GET /uploads/steven-hawley.jpg HTTP/1.1" 200'
+          + ' 1623 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)" spaces:this is a'
+          + ' thing with lots of spaces wwwwoooooo xss:<script>console.log("xss")</script>'
+          + ' headings:<h3>kimiya-yui</h5>, http://www.slate.com/success/koichi-wakata'
+          + ' links:thomas-marshburn@twitter.com, http://www.slate.com/info/michael-p-anderson, www.twitter.com'
+          + ' relatedContent:{ "url":'
+          + ' "http://www.laweekly.com/music/jay-electronica-much-better-than-his-name-would-suggest-2412364",'
+          + ' "og:type": "article", "og:title": "Jay Electronica: Much Better Than His Name Would Suggest",'
+          + ' "og:description": "You may not know who Jay Electronica is yet, but I&#039;m willing to bet that you'
+          + ' would had he chosen a better name. Jay Electronica does not sound like the ...", "og:url":'
+          + ' "http://www.laweekly.com/music/jay-electronica-much-better-than-his-name-would-suggest-2412364",'
+          + ' "article:published_time": "2008-04-04T16:00:00-07:00", "article:modified_time":'
+          + ' "2014-11-27T08:01:03-08:00", "article:section": "Music", "og:site_name": "LA Weekly", "twitter:title":'
+          + ' "Jay Electronica: Much Better Than His Name Would Suggest", "twitter:description": "You may not know'
+          + ' who Jay Electronica is yet, but I&#039;m willing to bet that you would had he chosen a better name.'
+          + ' Jay Electronica does not sound like the ...", "twitter:card": "summary", "twitter:site": "@laweekly"'
+          + ' }, { "url": "http://www.laweekly.com/news/mandoe-on-gower-near-fountain-2368123", "og:type":'
+          + ' "article", "og:title": "MANDOE On Gower Near Fountain", "og:description": "MANDOE has a stunner on a'
+          + ' wall north of an east-west street crossing Gower around Fountain (but not on Fountain). MADNOE, PROSE'
+          + ' and FUKM are listed on t...", "og:url": "'
+          + 'http://www.laweekly.com/news/mandoe-on-gower-near-fountain-2368123", "article:published_time":'
+          + ' "2008-04-25T07:26:41-07:00", "article:modified_time": "2014-10-28T15:00:08-07:00", "article:section":'
+          + ' "News", "og:image": "'
+          + 'http://images1.laweekly.com/imager/mandoe-on-gower-near-fountain/u/original/2430891/img_6648.jpg",'
+          + ' "og:image:height": "640", "og:image:width": "480", "og:site_name": "LA Weekly", "twitter:title": '
+          + '"MANDOE On Gower Near Fountain", "twitter:description": "MANDOE has a stunner on a wall north of an'
+          + ' east-west street crossing Gower around Fountain (but not on Fountain). MADNOE, PROSE and FUKM are'
+          + ' listed on t...", "twitter:card": "summary", "twitter:image": "'
+          + 'http://images1.laweekly.com/imager/mandoe-on-gower-near-fountain/u/original/2430891/img_6648.jpg", '
+          + '"twitter:site": "@laweekly" }, { "url": "http://www.laweekly.com/arts/meghan-finds-the-love-2373346",'
+          + ' "og:type": "article", "og:title": "Meghan Finds The Love", "og:description": "LA Weekly is the'
+          + ' definitive source of information for news, music, movies, restaurants, reviews, and events in Los'
+          + ' Angeles.", "og:url": "http://www.laweekly.com/arts/meghan-finds-the-love-2373346",'
+          + ' "article:published_time": "2005-10-20T18:10:25-07:00", "article:modified_time":'
+          + ' "2014-11-25T19:52:35-08:00", "article:section": "Arts", "og:site_name": "LA Weekly", "twitter:title":'
+          + ' "Meghan Finds The Love", "twitter:description": "LA Weekly is the definitive source of information for'
+          + ' news, music, movies, restaurants, reviews, and events in Los Angeles.", "twitter:card": "summary",'
+          + ' "twitter:site": "@laweekly" }, { "url": "http://www.laweekly.com/arts/these-clowns-are-with-me-2371051'
+          + '", "og:type": "article", "og:title": "These Clowns Are With Me", "og:description": "&nbsp; &nbsp; I'
+          + ' didn&#039;t mean to blow off all my responsibilities yesterday, but when a schmoozy Hollywood luncheon'
+          + ' turns into a full-on party by 3pm, and...", "og:url": "'
+          + 'http://www.laweekly.com/arts/these-clowns-are-with-me-2371051", "article:published_time": '
+          + '"2006-03-04T17:03:42-08:00", "article:modified_time": "2014-11-25T17:05:47-08:00", "article:section":'
+          + ' "Arts", "og:image": "'
+          + 'http://images1.laweekly.com/imager/these-clowns-are-with-me/u/original/2434556/e4b8scd.jpg",'
+          + ' "og:image:height": "375", "og:image:width": "500", "og:site_name": "LA Weekly", "twitter:title":'
+          + ' "These Clowns Are With Me", "twitter:description": "&nbsp; &nbsp; I didn&#039;t mean to blow off all'
+          + ' my responsibilities yesterday, but when a schmoozy Hollywood luncheon turns into a full-on party by'
+          + ' 3pm, and...", "twitter:card": "summary", "twitter:image": "'
+          + 'http://images1.laweekly.com/imager/these-clowns-are-with-me/u/original/2434556/e4b8scd.jpg",'
+          + ' "twitter:site": "@laweekly" }, { "url": "http://www.laweekly.com/arts/shopping-daze-2373807",'
+          + ' "og:type": "article", "og:title": "Shopping Daze", "og:description": "LA Weekly is the definitive '
+          + 'source of information for news, music, movies, restaurants, reviews, and events in Los Angeles.",'
+          + ' "og:url": "http://www.laweekly.com/arts/shopping-daze-2373807", "article:published_time":'
+          + ' "2006-12-13T12:12:04-08:00", "article:modified_time": "2014-11-25T20:15:21-08:00", "article:section":'
+          + ' "Arts", "og:site_name": "LA Weekly", "twitter:title": "Shopping Daze", "twitter:description": "LA'
+          + ' Weekly is the definitive source of information for news, music, movies, restaurants, reviews, and'
+          + ' events in Los Angeles.", "twitter:card": "summary", "twitter:site": "@laweekly" } machine.os:osx'
+          + ' machine.ram:15,032,385,536 _id:AU_x3_g3GFA8no6QjkFm _type:apache _index:logstash-2015.09.20 _score: -'
+          + ' relatedContent.article:modified_time:October 28th 2014, 22:00:08.000, November 26th 2014,'
+          + ' 01:05:47.000, November 26th 2014, 03:52:35.000, November 26th 2014, 04:15:21.000, November 27th 2014,'
+          + ' 16:01:03.000 relatedContent.article:published_time:October 21st 2005, 01:10:25.000, March 5th 2006,'
+          + ' 01:03:42.000, December 13th 2006, 20:12:04.000, April 4th 2008, 23:00:00.000, April 25th 2008,'
+          + ' 14:26:41.000';
+        return PageObjects.discover.clickDocSortDown()
+        .then(function () {
+          // we don't technically need this sleep here because the tryForTime will retry and the
+          // results will match on the 2nd or 3rd attempt, but that debug output is huge in this
+          // case and it can be avoided with just a few seconds sleep.
+          return PageObjects.common.sleep(2000);
+        })
+        .then(function () {
+          return retry.try(function tryingForTime() {
+            return PageObjects.discover.getDocTableIndex(1)
+            .then(function (rowData) {
+              PageObjects.common.saveScreenshot('Discover-sort-down');
+              expect(rowData).to.be(ExpectedDoc);
+            });
           });
         });
       });
-    });
 
 
-    bdd.it('a bad syntax query should show an error message', function () {
-      const expectedError = 'Discover: Failed to parse query [xxx(yyy]';
-      return PageObjects.discover.query('xxx(yyy')
-      .then(function () {
-        return PageObjects.header.getToastMessage();
-      })
-      .then(function (toastMessage) {
-        PageObjects.common.saveScreenshot('Discover-syntax-error-toast');
-        expect(toastMessage).to.be(expectedError);
-      })
-      .then(function () {
-        return PageObjects.header.clickToastOK();
+      it('a bad syntax query should show an error message', function () {
+        const expectedError = 'Discover: Failed to parse query [xxx(yyy]';
+        return PageObjects.discover.query('xxx(yyy')
+        .then(function () {
+          return PageObjects.header.getToastMessage();
+        })
+        .then(function (toastMessage) {
+          PageObjects.common.saveScreenshot('Discover-syntax-error-toast');
+          expect(toastMessage).to.be(expectedError);
+        })
+        .then(function () {
+          return PageObjects.header.clickToastOK();
+        });
       });
     });
   });
-});
+}

test/functional/apps/discover/_shared_links.js

@@ -1,129 +1,131 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esArchiver,
-  esClient,
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const retry = getService('retry');
+  const log = getService('log');
+  const esArchiver = getService('esArchiver');
+  const kibanaServer = getService('kibanaServer');
+  const PageObjects = getPageObjects(['common', 'discover', 'header']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('shared links', function describeIndexTests() {
+    let baseUrl;
+    // The message changes for Firefox < 41 and Firefox >= 41
+    // var expectedToastMessage = 'Share search: URL selected. Press Ctrl+C to copy.';
+    // var expectedToastMessage = 'Share search: URL copied to clipboard.';
+    // Pass either one.
+    const expectedToastMessage = /Share search: URL (selected\. Press Ctrl\+C to copy\.|copied to clipboard\.)/;
 
-bdd.describe('shared links', function describeIndexTests() {
-  let baseUrl;
-  // The message changes for Firefox < 41 and Firefox >= 41
-  // var expectedToastMessage = 'Share search: URL selected. Press Ctrl+C to copy.';
-  // var expectedToastMessage = 'Share search: URL copied to clipboard.';
-  // Pass either one.
-  const expectedToastMessage = /Share search: URL (selected\. Press Ctrl\+C to copy\.|copied to clipboard\.)/;
+    before(function () {
+      baseUrl = PageObjects.common.getHostPort();
+      log.debug('baseUrl = ' + baseUrl);
+      // browsers don't show the ':port' if it's 80 or 443 so we have to
+      // remove that part so we can get a match in the tests.
+      baseUrl = baseUrl.replace(':80','').replace(':443','');
+      log.debug('New baseUrl = ' + baseUrl);
 
-  bdd.before(function () {
-    baseUrl = PageObjects.common.getHostPort();
-    PageObjects.common.debug('baseUrl = ' + baseUrl);
-    // browsers don't show the ':port' if it's 80 or 443 so we have to
-    // remove that part so we can get a match in the tests.
-    baseUrl = baseUrl.replace(':80','').replace(':443','');
-    PageObjects.common.debug('New baseUrl = ' + baseUrl);
+      const fromTime = '2015-09-19 06:31:44.000';
+      const toTime = '2015-09-23 18:31:44.000';
 
-    const fromTime = '2015-09-19 06:31:44.000';
-    const toTime = '2015-09-23 18:31:44.000';
-
-    // delete .kibana index and update configDoc
-    return esClient.deleteAndUpdateConfigDoc({ 'dateFormat:tz':'UTC', 'defaultIndex':'logstash-*' })
-    .then(function loadkibanaIndexPattern() {
-      PageObjects.common.debug('load kibana index with default index pattern');
-      return esArchiver.load('discover');
-    })
-    // and load a set of makelogs data
-    .then(function loadIfEmptyMakelogs() {
-      return esArchiver.loadIfNeeded('logstash_functional');
-    })
-    .then(function () {
-      PageObjects.common.debug('discover');
-      return PageObjects.common.navigateToApp('discover');
-    })
-    .then(function () {
-      PageObjects.common.debug('setAbsoluteRange');
-      return PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    })
-    .then(function () {
-      //After hiding the time picker, we need to wait for
-      //the refresh button to hide before clicking the share button
-      return PageObjects.common.sleep(1000);
-    });
-  });
-
-
-  bdd.describe('shared link', function () {
-    bdd.it('should show "Share a link" caption', function () {
-      const expectedCaption = 'Share saved';
-      return PageObjects.discover.clickShare()
-      .then(function () {
-        PageObjects.common.saveScreenshot('Discover-share-link');
-        return PageObjects.discover.getShareCaption();
+      // delete .kibana index and update configDoc
+      return kibanaServer.uiSettings.replace({
+        'dateFormat:tz': 'UTC',
+        'defaultIndex': 'logstash-*'
       })
-      .then(function (actualCaption) {
-        expect(actualCaption).to.contain(expectedCaption);
+      .then(function loadkibanaIndexPattern() {
+        log.debug('load kibana index with default index pattern');
+        return esArchiver.load('discover');
+      })
+      // and load a set of makelogs data
+      .then(function loadIfEmptyMakelogs() {
+        return esArchiver.loadIfNeeded('logstash_functional');
+      })
+      .then(function () {
+        log.debug('discover');
+        return PageObjects.common.navigateToApp('discover');
+      })
+      .then(function () {
+        log.debug('setAbsoluteRange');
+        return PageObjects.header.setAbsoluteRange(fromTime, toTime);
+      })
+      .then(function () {
+        //After hiding the time picker, we need to wait for
+        //the refresh button to hide before clicking the share button
+        return PageObjects.common.sleep(1000);
       });
     });
 
-    bdd.it('should show the correct formatted URL', function () {
-      const expectedUrl = baseUrl
-        + '/app/kibana?_t=1453775307251#'
-        + '/discover?_g=(refreshInterval:(display:Off,pause:!f,value:0),time'
-        + ':(from:\'2015-09-19T06:31:44.000Z\',mode:absolute,to:\'2015-09'
-        + '-23T18:31:44.000Z\'))&_a=(columns:!(_source),index:\'logstash-'
-        + '*\',interval:auto,query:(query_string:(analyze_wildcard:!t,query'
-        + ':\'*\')),sort:!(\'@timestamp\',desc))';
-      return PageObjects.discover.getSharedUrl()
-      .then(function (actualUrl) {
-        // strip the timestamp out of each URL
-        expect(actualUrl.replace(/_t=\d{13}/,'_t=TIMESTAMP'))
-          .to.be(expectedUrl.replace(/_t=\d{13}/,'_t=TIMESTAMP'));
-      });
-    });
 
-    bdd.it('should show toast message for copy to clipboard', function () {
-      return PageObjects.discover.clickCopyToClipboard()
-      .then(function () {
-        return PageObjects.header.getToastMessage();
-      })
-      .then(function (toastMessage) {
-        PageObjects.common.saveScreenshot('Discover-copy-to-clipboard-toast');
-        expect(toastMessage).to.match(expectedToastMessage);
-      })
-      .then(function () {
-        return PageObjects.header.waitForToastMessageGone();
+    describe('shared link', function () {
+      it('should show "Share a link" caption', function () {
+        const expectedCaption = 'Share saved';
+        return PageObjects.discover.clickShare()
+        .then(function () {
+          PageObjects.common.saveScreenshot('Discover-share-link');
+          return PageObjects.discover.getShareCaption();
+        })
+        .then(function (actualCaption) {
+          expect(actualCaption).to.contain(expectedCaption);
+        });
       });
-    });
 
-    // TODO: verify clipboard contents
-    bdd.it('shorten URL button should produce a short URL', function () {
-      const re = new RegExp(baseUrl + '/goto/[0-9a-f]{32}$');
-      return PageObjects.discover.clickShortenUrl()
-      .then(function () {
-        return PageObjects.common.try(function tryingForTime() {
-          PageObjects.common.saveScreenshot('Discover-shorten-url-button');
-          return PageObjects.discover.getSharedUrl()
-          .then(function (actualUrl) {
-            expect(actualUrl).to.match(re);
+      it('should show the correct formatted URL', function () {
+        const expectedUrl = baseUrl
+          + '/app/kibana?_t=1453775307251#'
+          + '/discover?_g=(refreshInterval:(display:Off,pause:!f,value:0),time'
+          + ':(from:\'2015-09-19T06:31:44.000Z\',mode:absolute,to:\'2015-09'
+          + '-23T18:31:44.000Z\'))&_a=(columns:!(_source),index:\'logstash-'
+          + '*\',interval:auto,query:(query_string:(analyze_wildcard:!t,query'
+          + ':\'*\')),sort:!(\'@timestamp\',desc))';
+        return PageObjects.discover.getSharedUrl()
+        .then(function (actualUrl) {
+          // strip the timestamp out of each URL
+          expect(actualUrl.replace(/_t=\d{13}/,'_t=TIMESTAMP'))
+            .to.be(expectedUrl.replace(/_t=\d{13}/,'_t=TIMESTAMP'));
+        });
+      });
+
+      it('should show toast message for copy to clipboard', function () {
+        return PageObjects.discover.clickCopyToClipboard()
+        .then(function () {
+          return PageObjects.header.getToastMessage();
+        })
+        .then(function (toastMessage) {
+          PageObjects.common.saveScreenshot('Discover-copy-to-clipboard-toast');
+          expect(toastMessage).to.match(expectedToastMessage);
+        })
+        .then(function () {
+          return PageObjects.header.waitForToastMessageGone();
+        });
+      });
+
+      // TODO: verify clipboard contents
+      it('shorten URL button should produce a short URL', function () {
+        const re = new RegExp(baseUrl + '/goto/[0-9a-f]{32}$');
+        return PageObjects.discover.clickShortenUrl()
+        .then(function () {
+          return retry.try(function tryingForTime() {
+            PageObjects.common.saveScreenshot('Discover-shorten-url-button');
+            return PageObjects.discover.getSharedUrl()
+            .then(function (actualUrl) {
+              expect(actualUrl).to.match(re);
+            });
           });
         });
       });
-    });
 
-    // NOTE: This test has to run immediately after the test above
-    bdd.it('should show toast message for copy to clipboard', function () {
-      return PageObjects.discover.clickCopyToClipboard()
-      .then(function () {
-        return PageObjects.header.getToastMessage();
-      })
-      .then(function (toastMessage) {
-        expect(toastMessage).to.match(expectedToastMessage);
-      })
-      .then(function () {
-        return PageObjects.header.waitForToastMessageGone();
+      // NOTE: This test has to run immediately after the test above
+      it('should show toast message for copy to clipboard', function () {
+        return PageObjects.discover.clickCopyToClipboard()
+        .then(function () {
+          return PageObjects.header.getToastMessage();
+        })
+        .then(function (toastMessage) {
+          expect(toastMessage).to.match(expectedToastMessage);
+        })
+        .then(function () {
+          return PageObjects.header.waitForToastMessageGone();
+        });
       });
     });
   });
-});
+}

test/functional/apps/discover/_source_filters.js

@@ -1,50 +1,51 @@
-import {
-  bdd,
-  esArchiver,
-  esClient,
-} from '../../../support';
-
-import PageObjects from '../../../support/page_objects';
-
 import expect from 'expect.js';
 
-bdd.describe('source filters', function describeIndexTests() {
-  bdd.before(function () {
+export default function ({ getService, getPageObjects }) {
+  const log = getService('log');
+  const esArchiver = getService('esArchiver');
+  const kibanaServer = getService('kibanaServer');
+  const PageObjects = getPageObjects(['common', 'header', 'discover']);
 
-    const fromTime = '2015-09-19 06:31:44.000';
-    const toTime = '2015-09-23 18:31:44.000';
+  describe('source filters', function describeIndexTests() {
+    before(function () {
+      const fromTime = '2015-09-19 06:31:44.000';
+      const toTime = '2015-09-23 18:31:44.000';
 
-    // delete .kibana index and update configDoc
-    return esClient.deleteAndUpdateConfigDoc({ 'dateFormat:tz':'UTC', 'defaultIndex':'logstash-*' })
-    .then(function loadkibanaIndexPattern() {
-      PageObjects.common.debug('load kibana index with default index pattern');
-      return esArchiver.load('visualize_source-filters');
-    })
-    // and load a set of makelogs data
-    .then(function loadIfEmptyMakelogs() {
-      return esArchiver.loadIfNeeded('logstash_functional');
-    })
-    .then(function () {
-      PageObjects.common.debug('discover');
-      return PageObjects.common.navigateToApp('discover');
-    })
-    .then(function () {
-      PageObjects.common.debug('setAbsoluteRange');
-      return PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    })
-    .then(function () {
-      //After hiding the time picker, we need to wait for
-      //the refresh button to hide before clicking the share button
-      return PageObjects.common.sleep(1000);
+      // delete .kibana index and update configDoc
+      return kibanaServer.uiSettings.replace({
+        'dateFormat:tz': 'UTC',
+        'defaultIndex':'logstash-*'
+      })
+      .then(function loadkibanaIndexPattern() {
+        log.debug('load kibana index with default index pattern');
+        return esArchiver.load('visualize_source-filters');
+      })
+      // and load a set of makelogs data
+      .then(function loadIfEmptyMakelogs() {
+        return esArchiver.loadIfNeeded('logstash_functional');
+      })
+      .then(function () {
+        log.debug('discover');
+        return PageObjects.common.navigateToApp('discover');
+      })
+      .then(function () {
+        log.debug('setAbsoluteRange');
+        return PageObjects.header.setAbsoluteRange(fromTime, toTime);
+      })
+      .then(function () {
+        //After hiding the time picker, we need to wait for
+        //the refresh button to hide before clicking the share button
+        return PageObjects.common.sleep(1000);
+      });
+    });
+
+    it('should not get the field referer', function () {
+      return PageObjects.discover.getAllFieldNames()
+      .then(function (fieldNames) {
+        expect(fieldNames).to.not.contain('referer');
+        const relatedContentFields = fieldNames.filter((fieldName) => fieldName.indexOf('relatedContent') === 0);
+        expect(relatedContentFields).to.have.length(0);
+      });
     });
   });
-
-  bdd.it('should not get the field referer', function () {
-    return PageObjects.discover.getAllFieldNames()
-    .then(function (fieldNames) {
-      expect(fieldNames).to.not.contain('referer');
-      const relatedContentFields = fieldNames.filter((fieldName) => fieldName.indexOf('relatedContent') === 0);
-      expect(relatedContentFields).to.have.length(0);
-    });
-  });
-});
+}

test/functional/apps/discover/index.js

@@ -1,26 +1,23 @@
+export default function ({ getService, loadTestFile }) {
+  const config = getService('config');
+  const esArchiver = getService('esArchiver');
+  const remote = getService('remote');
 
-import {
-  bdd,
-  esArchiver,
-  defaultTimeout,
-} from '../../../support';
+  describe('discover app', function () {
+    this.timeout(config.get('timeouts.test'));
 
-import PageObjects from '../../../support/page_objects';
+    before(function () {
+      return remote.setWindowSize(1200,800);
+    });
 
-bdd.describe('discover app', function () {
-  this.timeout = defaultTimeout;
+    after(function unloadMakelogs() {
+      return esArchiver.unload('logstash_functional');
+    });
 
-  bdd.before(function () {
-    return PageObjects.remote.setWindowSize(1200,800);
+    loadTestFile(require.resolve('./_discover'));
+    loadTestFile(require.resolve('./_field_data'));
+    loadTestFile(require.resolve('./_shared_links'));
+    loadTestFile(require.resolve('./_collapse_expand'));
+    loadTestFile(require.resolve('./_source_filters'));
   });
-
-  bdd.after(function unloadMakelogs() {
-    return esArchiver.unload('logstash_functional');
-  });
-
-  require('./_discover');
-  require('./_field_data');
-  require('./_shared_links');
-  require('./_collapse_expand');
-  require('./_source_filters');
-});
+}

test/functional/apps/management/_creation_form_changes.js

@@ -1,59 +1,56 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esClient,
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const PageObjects = getPageObjects(['settings', 'common']);
 
-import PageObjects from '../../../support/page_objects';
-
-bdd.describe('user input reactions', function () {
-  bdd.beforeEach(function () {
-    // delete .kibana index and then wait for Kibana to re-create it
-    return esClient.deleteAndUpdateConfigDoc()
-    .then(function () {
-      return PageObjects.settings.navigateTo();
-    })
-    .then(function () {
-      return PageObjects.settings.clickKibanaIndicies();
+  describe('user input reactions', function () {
+    beforeEach(function () {
+      // delete .kibana index and then wait for Kibana to re-create it
+      return kibanaServer.uiSettings.replace({})
+      .then(function () {
+        return PageObjects.settings.navigateTo();
+      })
+      .then(function () {
+        return PageObjects.settings.clickKibanaIndicies();
+      });
     });
-  });
 
-  bdd.it('should hide time-based index pattern when time-based option is unchecked', function () {
-    const self = this;
-    return PageObjects.settings.getTimeBasedEventsCheckbox()
-    .then(function (selected) {
-      // uncheck the 'time-based events' checkbox
-      return selected.click();
-    })
-    // try to find the checkbox (this shouldn fail)
-    .then(function () {
-      const waitTime = 10000;
-      return PageObjects.settings.getTimeBasedIndexPatternCheckbox(waitTime);
-    })
-    .then(function () {
-      PageObjects.common.saveScreenshot('Settings-indices-hide-time-based-index-pattern');
-      // we expect the promise above to fail
-      const handler = PageObjects.common.createErrorHandler(self);
-      const msg = 'Found time based index pattern checkbox';
-      handler(msg);
-    })
-    .catch(function () {
-      // we expect this failure since checkbox should be hidden
-      return;
+    it('should hide time-based index pattern when time-based option is unchecked', function () {
+      const self = this;
+      return PageObjects.settings.getTimeBasedEventsCheckbox()
+      .then(function (selected) {
+        // uncheck the 'time-based events' checkbox
+        return selected.click();
+      })
+      // try to find the checkbox (this shouldn fail)
+      .then(function () {
+        const waitTime = 10000;
+        return PageObjects.settings.getTimeBasedIndexPatternCheckbox(waitTime);
+      })
+      .then(function () {
+        PageObjects.common.saveScreenshot('Settings-indices-hide-time-based-index-pattern');
+        // we expect the promise above to fail
+        const handler = PageObjects.common.createErrorHandler(self);
+        const msg = 'Found time based index pattern checkbox';
+        handler(msg);
+      })
+      .catch(function () {
+        // we expect this failure since checkbox should be hidden
+        return;
+      });
     });
-  });
 
-  bdd.it('should enable creation after selecting time field', function () {
-    // select a time field and check that Create button is enabled
-    return PageObjects.settings.selectTimeFieldOption('@timestamp')
-    .then(function () {
-      return PageObjects.settings.getCreateButton().isEnabled()
-      .then(function (enabled) {
-        PageObjects.common.saveScreenshot('Settings-indices-enable-creation');
-        expect(enabled).to.be.ok();
+    it('should enable creation after selecting time field', function () {
+      // select a time field and check that Create button is enabled
+      return PageObjects.settings.selectTimeFieldOption('@timestamp')
+      .then(function () {
+        return PageObjects.settings.getCreateButton().isEnabled()
+        .then(function (enabled) {
+          PageObjects.common.saveScreenshot('Settings-indices-enable-creation');
+          expect(enabled).to.be.ok();
+        });
       });
     });
   });
-});
+}

test/functional/apps/management/_index_pattern_create_delete.js

@@ -1,102 +1,101 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  remote,
-  esClient
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const remote = getService('remote');
+  const log = getService('log');
+  const retry = getService('retry');
+  const PageObjects = getPageObjects(['settings', 'common']);
 
-import PageObjects from '../../../support/page_objects';
-
-bdd.describe('creating and deleting default index', function describeIndexTests() {
-  bdd.before(function () {
-    // delete .kibana index and then wait for Kibana to re-create it
-    return esClient.deleteAndUpdateConfigDoc()
-    .then(function () {
-      return PageObjects.settings.navigateTo();
-    })
-    .then(function () {
-      return PageObjects.settings.clickKibanaIndicies();
-    });
-  });
-
-  bdd.describe('index pattern creation', function indexPatternCreation() {
-    bdd.before(function () {
-      return PageObjects.settings.createIndexPattern();
-    });
-
-    bdd.it('should have index pattern in page header', function () {
-      return PageObjects.settings.getIndexPageHeading().getVisibleText()
-      .then(function (patternName) {
-        PageObjects.common.saveScreenshot('Settings-indices-new-index-pattern');
-        expect(patternName).to.be('logstash-*');
+  describe('creating and deleting default index', function describeIndexTests() {
+    before(function () {
+      // delete .kibana index and then wait for Kibana to re-create it
+      return kibanaServer.uiSettings.replace({})
+      .then(function () {
+        return PageObjects.settings.navigateTo();
+      })
+      .then(function () {
+        return PageObjects.settings.clickKibanaIndicies();
       });
     });
 
-    bdd.it('should have index pattern in url', function url() {
-      return PageObjects.common.try(function tryingForTime() {
-        return remote.getCurrentUrl()
-        .then(function (currentUrl) {
-          expect(currentUrl).to.contain('logstash-*');
+    describe('index pattern creation', function indexPatternCreation() {
+      before(function () {
+        return PageObjects.settings.createIndexPattern();
+      });
+
+      it('should have index pattern in page header', function () {
+        return PageObjects.settings.getIndexPageHeading().getVisibleText()
+        .then(function (patternName) {
+          PageObjects.common.saveScreenshot('Settings-indices-new-index-pattern');
+          expect(patternName).to.be('logstash-*');
         });
       });
-    });
 
-    bdd.it('should have expected table headers', function checkingHeader() {
-      return PageObjects.settings.getTableHeader()
-      .then(function (headers) {
-        PageObjects.common.debug('header.length = ' + headers.length);
-        const expectedHeaders = [
-          'name',
-          'type',
-          'format',
-          'searchable',
-          'aggregatable',
-          'analyzed',
-          'excluded',
-          'controls'
-        ];
-
-        expect(headers.length).to.be(expectedHeaders.length);
-
-        const comparedHeaders = headers.map(function compareHead(header, i) {
-          return header.getVisibleText()
-          .then(function (text) {
-            expect(text).to.be(expectedHeaders[i]);
+      it('should have index pattern in url', function url() {
+        return retry.try(function tryingForTime() {
+          return remote.getCurrentUrl()
+          .then(function (currentUrl) {
+            expect(currentUrl).to.contain('logstash-*');
           });
         });
-
-        return Promise.all(comparedHeaders);
       });
-    });
-  });
 
-  bdd.describe('index pattern deletion', function indexDelete() {
-    bdd.before(function () {
-      const expectedAlertText = 'Are you sure you want to remove this index pattern?';
-      return PageObjects.settings.removeIndexPattern()
-      .then(function (alertText) {
-        PageObjects.common.saveScreenshot('Settings-indices-confirm-remove-index-pattern');
-        expect(alertText).to.be(expectedAlertText);
+      it('should have expected table headers', function checkingHeader() {
+        return PageObjects.settings.getTableHeader()
+        .then(function (headers) {
+          log.debug('header.length = ' + headers.length);
+          const expectedHeaders = [
+            'name',
+            'type',
+            'format',
+            'searchable',
+            'aggregatable',
+            'analyzed',
+            'excluded',
+            'controls'
+          ];
+
+          expect(headers.length).to.be(expectedHeaders.length);
+
+          const comparedHeaders = headers.map(function compareHead(header, i) {
+            return header.getVisibleText()
+            .then(function (text) {
+              expect(text).to.be(expectedHeaders[i]);
+            });
+          });
+
+          return Promise.all(comparedHeaders);
+        });
       });
     });
 
-    bdd.it('should return to index pattern creation page', function returnToPage() {
-      return PageObjects.common.try(function tryingForTime() {
-        return PageObjects.settings.getCreateButton();
+    describe('index pattern deletion', function indexDelete() {
+      before(function () {
+        const expectedAlertText = 'Are you sure you want to remove this index pattern?';
+        return PageObjects.settings.removeIndexPattern()
+        .then(function (alertText) {
+          PageObjects.common.saveScreenshot('Settings-indices-confirm-remove-index-pattern');
+          expect(alertText).to.be(expectedAlertText);
+        });
       });
-    });
 
-    bdd.it('should remove index pattern from url', function indexNotInUrl() {
-      // give the url time to settle
-      return PageObjects.common.try(function tryingForTime() {
-        return remote.getCurrentUrl()
-        .then(function (currentUrl) {
-          PageObjects.common.debug('currentUrl = ' + currentUrl);
-          expect(currentUrl).to.not.contain('logstash-*');
+      it('should return to index pattern creation page', function returnToPage() {
+        return retry.try(function tryingForTime() {
+          return PageObjects.settings.getCreateButton();
+        });
+      });
+
+      it('should remove index pattern from url', function indexNotInUrl() {
+        // give the url time to settle
+        return retry.try(function tryingForTime() {
+          return remote.getCurrentUrl()
+          .then(function (currentUrl) {
+            log.debug('currentUrl = ' + currentUrl);
+            expect(currentUrl).to.not.contain('logstash-*');
+          });
         });
       });
     });
   });
-});
+}

test/functional/apps/management/_index_pattern_filter.js

@@ -1,57 +1,54 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esClient
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const retry = getService('retry');
+  const PageObjects = getPageObjects(['settings']);
 
-import PageObjects from '../../../support/page_objects';
-
-bdd.describe('index pattern filter', function describeIndexTests() {
-  bdd.before(function () {
-    // delete .kibana index and then wait for Kibana to re-create it
-    return esClient.deleteAndUpdateConfigDoc()
-    .then(function () {
-      return PageObjects.settings.navigateTo();
-    })
-    .then(function () {
-      return PageObjects.settings.clickKibanaIndicies();
-    });
-  });
-
-  bdd.beforeEach(function () {
-    return PageObjects.settings.createIndexPattern();
-  });
-
-  bdd.afterEach(function () {
-    return PageObjects.settings.removeIndexPattern();
-  });
-
-  bdd.it('should filter indexed fields', async function () {
-    await PageObjects.settings.navigateTo();
-    await PageObjects.settings.clickKibanaIndicies();
-    await PageObjects.settings.getFieldTypes();
-
-
-    await PageObjects.settings.setFieldTypeFilter('string');
-
-    await PageObjects.common.try(async function() {
-      const fieldTypes = await PageObjects.settings.getFieldTypes();
-      expect(fieldTypes.length).to.be.above(0);
-      for (const fieldType of fieldTypes) {
-        expect(fieldType).to.be('string');
-      }
+  describe('index pattern filter', function describeIndexTests() {
+    before(function () {
+      // delete .kibana index and then wait for Kibana to re-create it
+      return kibanaServer.uiSettings.replace({})
+      .then(function () {
+        return PageObjects.settings.navigateTo();
+      })
+      .then(function () {
+        return PageObjects.settings.clickKibanaIndicies();
+      });
     });
 
-    await PageObjects.settings.setFieldTypeFilter('number');
+    beforeEach(function () {
+      return PageObjects.settings.createIndexPattern();
+    });
 
-    await PageObjects.common.try(async function() {
-      const fieldTypes = await PageObjects.settings.getFieldTypes();
-      expect(fieldTypes.length).to.be.above(0);
-      for (const fieldType of fieldTypes) {
-        expect(fieldType).to.be('number');
-      }
+    afterEach(function () {
+      return PageObjects.settings.removeIndexPattern();
+    });
+
+    it('should filter indexed fields', async function () {
+      await PageObjects.settings.navigateTo();
+      await PageObjects.settings.clickKibanaIndicies();
+      await PageObjects.settings.getFieldTypes();
+
+      await PageObjects.settings.setFieldTypeFilter('string');
+
+      await retry.try(async function() {
+        const fieldTypes = await PageObjects.settings.getFieldTypes();
+        expect(fieldTypes.length).to.be.above(0);
+        for (const fieldType of fieldTypes) {
+          expect(fieldType).to.be('string');
+        }
+      });
+
+      await PageObjects.settings.setFieldTypeFilter('number');
+
+      await retry.try(async function() {
+        const fieldTypes = await PageObjects.settings.getFieldTypes();
+        expect(fieldTypes.length).to.be.above(0);
+        for (const fieldType of fieldTypes) {
+          expect(fieldType).to.be('number');
+        }
+      });
     });
   });
-});
+}

test/functional/apps/management/_index_pattern_popularity.js

@@ -1,109 +1,80 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esClient
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const log = getService('log');
+  const PageObjects = getPageObjects(['settings', 'common']);
 
-import PageObjects from '../../../support/page_objects';
-
-bdd.describe('index result popularity', function describeIndexTests() {
-  bdd.before(function () {
-    // delete .kibana index and then wait for Kibana to re-create it
-    return esClient.deleteAndUpdateConfigDoc()
-    .then(function () {
-      return PageObjects.settings.navigateTo();
-    });
-  });
-
-  bdd.beforeEach(function be() {
-    return PageObjects.settings.createIndexPattern();
-  });
-
-  bdd.afterEach(function ae() {
-    return PageObjects.settings.removeIndexPattern();
-  });
-
-  bdd.describe('change popularity', function indexPatternCreation() {
-    const fieldName = 'geo.coordinates';
-
-    // set the page size to All again, https://github.com/elastic/kibana/issues/5030
-    // TODO: remove this after issue #5030 is closed
-    function fix5030() {
-      return PageObjects.settings.setPageSize('All')
+  describe('index result popularity', function describeIndexTests() {
+    before(function () {
+      // delete .kibana index and then wait for Kibana to re-create it
+      return kibanaServer.uiSettings.replace({})
       .then(function () {
-        return PageObjects.common.sleep(1000);
-      });
-    }
-
-    bdd.beforeEach(function () {
-      // increase Popularity of geo.coordinates
-      return PageObjects.settings.setPageSize('All')
-      .then(function () {
-        return PageObjects.common.sleep(1000);
-      })
-      .then(function openControlsByName() {
-        PageObjects.common.debug('Starting openControlsByName (' + fieldName + ')');
-        return PageObjects.settings.openControlsByName(fieldName);
-      })
-      .then(function increasePopularity() {
-        PageObjects.common.debug('increasePopularity');
-        return PageObjects.settings.increasePopularity();
+        return PageObjects.settings.navigateTo();
       });
     });
 
-    bdd.afterEach(function () {
-      // Cancel saving the popularity change (we didn't make a change in this case, just checking the value)
-      return PageObjects.settings.controlChangeCancel();
+    beforeEach(function be() {
+      return PageObjects.settings.createIndexPattern();
     });
 
-    bdd.it('should update the popularity input', function () {
-      return PageObjects.settings.getPopularity()
-      .then(function (popularity) {
-        PageObjects.common.debug('popularity = ' + popularity);
+    afterEach(function ae() {
+      return PageObjects.settings.removeIndexPattern();
+    });
+
+    describe('change popularity', function indexPatternCreation() {
+      const fieldName = 'geo.coordinates';
+
+      // set the page size to All again, https://github.com/elastic/kibana/issues/5030
+      // TODO: remove this after issue #5030 is closed
+      async function fix5030() {
+        await PageObjects.settings.setPageSize('All');
+        await PageObjects.common.sleep(1000);
+      }
+
+      beforeEach(async function () {
+        // increase Popularity of geo.coordinates
+        await PageObjects.settings.setPageSize('All');
+        await PageObjects.common.sleep(1000);
+        log.debug('Starting openControlsByName (' + fieldName + ')');
+        await PageObjects.settings.openControlsByName(fieldName);
+        log.debug('increasePopularity');
+        await PageObjects.settings.increasePopularity();
+      });
+
+      afterEach(async function () {
+        // Cancel saving the popularity change (we didn't make a change in this case, just checking the value)
+        await PageObjects.settings.controlChangeCancel();
+      });
+
+      it('should update the popularity input', async function () {
+        const popularity = await PageObjects.settings.getPopularity();
+        log.debug('popularity = ' + popularity);
         expect(popularity).to.be('1');
         PageObjects.common.saveScreenshot('Settings-indices-result-popularity-updated');
       });
-    });
 
-    bdd.it('should be reset on cancel', function () {
-      // Cancel saving the popularity change
-      return PageObjects.settings.controlChangeCancel()
-      .then(function () {
-        return fix5030();
-      })
-      .then(function openControlsByName() {
-        return PageObjects.settings.openControlsByName(fieldName);
-      })
-      // check that its 0 (previous increase was cancelled)
-      .then(function getPopularity() {
-        return PageObjects.settings.getPopularity();
-      })
-      .then(function (popularity) {
-        PageObjects.common.debug('popularity = ' + popularity);
+      it('should be reset on cancel', async function () {
+        // Cancel saving the popularity change
+        await PageObjects.settings.controlChangeCancel();
+        await fix5030();
+        await PageObjects.settings.openControlsByName(fieldName);
+        // check that it is 0 (previous increase was cancelled
+        const popularity = await PageObjects.settings.getPopularity();
+        log.debug('popularity = ' + popularity);
         expect(popularity).to.be('0');
       });
-    });
 
-    bdd.it('can be saved', function () {
-      // Saving the popularity change
-      return PageObjects.settings.controlChangeSave()
-      .then(function () {
-        return fix5030();
-      })
-      .then(function openControlsByName() {
-        return PageObjects.settings.openControlsByName(fieldName);
-      })
-      // check that its 0 (previous increase was cancelled)
-      .then(function getPopularity() {
-        return PageObjects.settings.getPopularity();
-      })
-      .then(function (popularity) {
-        PageObjects.common.debug('popularity = ' + popularity);
+      it('can be saved', async function () {
+        // Saving the popularity change
+        await PageObjects.settings.controlChangeSave();
+        await fix5030();
+        await PageObjects.settings.openControlsByName(fieldName);
+        const popularity = await PageObjects.settings.getPopularity();
+        log.debug('popularity = ' + popularity);
         expect(popularity).to.be('1');
         PageObjects.common.saveScreenshot('Settings-indices-result-popularity-saved');
       });
-    });
-  }); // end 'change popularity'
-}); // end index result popularity
+    }); // end 'change popularity'
+  }); // end index result popularity
+}

test/functional/apps/management/_index_pattern_results_sort.js

@@ -1,124 +1,122 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esClient
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const retry = getService('retry');
+  const PageObjects = getPageObjects(['settings', 'common']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('index result field sort', function describeIndexTests() {
+    before(function () {
+      // delete .kibana index and then wait for Kibana to re-create it
+      return kibanaServer.uiSettings.replace({});
+    });
 
-bdd.describe('index result field sort', function describeIndexTests() {
-  bdd.before(function () {
-    // delete .kibana index and then wait for Kibana to re-create it
-    return esClient.deleteAndUpdateConfigDoc();
-  });
+    const columns = [{
+      heading: 'name',
+      first: '@message',
+      last: 'xss.raw',
+      selector: function () {
+        return PageObjects.settings.getTableRow(0, 0).getVisibleText();
+      }
+    }, {
+      heading: 'type',
+      first: '_source',
+      last: 'string',
+      selector: function () {
+        return PageObjects.settings.getTableRow(0, 1).getVisibleText();
+      }
+    }];
 
-  const columns = [{
-    heading: 'name',
-    first: '@message',
-    last: 'xss.raw',
-    selector: function () {
-      return PageObjects.settings.getTableRow(0, 0).getVisibleText();
-    }
-  }, {
-    heading: 'type',
-    first: '_source',
-    last: 'string',
-    selector: function () {
-      return PageObjects.settings.getTableRow(0, 1).getVisibleText();
-    }
-  }];
+    columns.forEach(function (col) {
+      describe('sort by heading - ' + col.heading, function indexPatternCreation() {
+        before(function () {
+          return PageObjects.settings.navigateTo()
+          .then(function () {
+            return PageObjects.settings.clickKibanaIndicies();
+          });
+        });
 
-  columns.forEach(function (col) {
-    bdd.describe('sort by heading - ' + col.heading, function indexPatternCreation() {
-      bdd.before(function () {
+        beforeEach(function () {
+          return PageObjects.settings.createIndexPattern();
+        });
+
+        afterEach(function () {
+          return PageObjects.settings.removeIndexPattern();
+        });
+
+        it('should sort ascending', function () {
+          return PageObjects.settings.sortBy(col.heading)
+          .then(function getText() {
+            return col.selector();
+          })
+          .then(function (rowText) {
+            PageObjects.common.saveScreenshot(`Settings-indices-column-${col.heading}-sort-ascending`);
+            expect(rowText).to.be(col.first);
+          });
+        });
+
+        it('should sort descending', function () {
+          return PageObjects.settings.sortBy(col.heading)
+          .then(function sortAgain() {
+            return PageObjects.settings.sortBy(col.heading);
+          })
+          .then(function getText() {
+            return col.selector();
+          })
+          .then(function (rowText) {
+            PageObjects.common.saveScreenshot(`Settings-indices-column-${col.heading}-sort-descending`);
+            expect(rowText).to.be(col.last);
+          });
+        });
+      });
+    });
+
+    describe('field list pagination', function () {
+      const EXPECTED_DEFAULT_PAGE_SIZE = 25;
+      const EXPECTED_FIELD_COUNT = 85;
+      const EXPECTED_LAST_PAGE_COUNT = 10;
+      const LAST_PAGE_NUMBER = 4;
+
+      before(function () {
         return PageObjects.settings.navigateTo()
         .then(function () {
-          return PageObjects.settings.clickKibanaIndicies();
+          return PageObjects.settings.createIndexPattern();
         });
       });
 
-      bdd.beforeEach(function () {
-        return PageObjects.settings.createIndexPattern();
-      });
-
-      bdd.afterEach(function () {
+      after(function () {
         return PageObjects.settings.removeIndexPattern();
       });
 
-      bdd.it('should sort ascending', function () {
-        return PageObjects.settings.sortBy(col.heading)
-        .then(function getText() {
-          return col.selector();
-        })
-        .then(function (rowText) {
-          PageObjects.common.saveScreenshot(`Settings-indices-column-${col.heading}-sort-ascending`);
-          expect(rowText).to.be(col.first);
+      it('makelogs data should have expected number of fields', function () {
+        return retry.try(function () {
+          return PageObjects.settings.getFieldsTabCount()
+          .then(function (tabCount) {
+            expect(tabCount).to.be('' + EXPECTED_FIELD_COUNT);
+          });
         });
       });
 
-      bdd.it('should sort descending', function () {
-        return PageObjects.settings.sortBy(col.heading)
-        .then(function sortAgain() {
-          return PageObjects.settings.sortBy(col.heading);
-        })
-        .then(function getText() {
-          return col.selector();
-        })
-        .then(function (rowText) {
-          PageObjects.common.saveScreenshot(`Settings-indices-column-${col.heading}-sort-descending`);
-          expect(rowText).to.be(col.last);
+      it('should have correct default page size selected', function () {
+        return PageObjects.settings.getPageSize()
+        .then(function (pageSize) {
+          expect(pageSize).to.be('' + EXPECTED_DEFAULT_PAGE_SIZE);
         });
       });
-    });
-  });
 
-  bdd.describe('field list pagination', function () {
-    const EXPECTED_DEFAULT_PAGE_SIZE = 25;
-    const EXPECTED_FIELD_COUNT = 85;
-    const EXPECTED_LAST_PAGE_COUNT = 10;
-    const LAST_PAGE_NUMBER = 4;
+      it('should have the correct number of rows per page', async function () {
+        for (let pageNum = 1; pageNum <= LAST_PAGE_NUMBER; pageNum += 1) {
+          await PageObjects.settings.goToPage(pageNum);
+          const pageFieldNames = await retry.tryMethod(PageObjects.settings, 'getFieldNames');
+          await PageObjects.common.saveScreenshot(`Settings-indexed-fields-page-${pageNum}`);
 
-    bdd.before(function () {
-      return PageObjects.settings.navigateTo()
-      .then(function () {
-        return PageObjects.settings.createIndexPattern();
-      });
-    });
-
-    bdd.after(function () {
-      return PageObjects.settings.removeIndexPattern();
-    });
-
-    bdd.it('makelogs data should have expected number of fields', function () {
-      return PageObjects.common.try(function () {
-        return PageObjects.settings.getFieldsTabCount()
-        .then(function (tabCount) {
-          expect(tabCount).to.be('' + EXPECTED_FIELD_COUNT);
-        });
-      });
-    });
-
-    bdd.it('should have correct default page size selected', function () {
-      return PageObjects.settings.getPageSize()
-      .then(function (pageSize) {
-        expect(pageSize).to.be('' + EXPECTED_DEFAULT_PAGE_SIZE);
-      });
-    });
-
-    bdd.it('should have the correct number of rows per page', async function () {
-      for (let pageNum = 1; pageNum <= LAST_PAGE_NUMBER; pageNum += 1) {
-        await PageObjects.settings.goToPage(pageNum);
-        const pageFieldNames = await PageObjects.common.tryMethod(PageObjects.settings, 'getFieldNames');
-        await PageObjects.common.saveScreenshot(`Settings-indexed-fields-page-${pageNum}`);
-
-        if (pageNum === LAST_PAGE_NUMBER) {
-          expect(pageFieldNames).to.have.length(EXPECTED_LAST_PAGE_COUNT);
-        } else {
-          expect(pageFieldNames).to.have.length(EXPECTED_DEFAULT_PAGE_SIZE);
+          if (pageNum === LAST_PAGE_NUMBER) {
+            expect(pageFieldNames).to.have.length(EXPECTED_LAST_PAGE_COUNT);
+          } else {
+            expect(pageFieldNames).to.have.length(EXPECTED_DEFAULT_PAGE_SIZE);
+          }
         }
-      }
-    });
-  }); // end describe pagination
-}); // end index result field sort
+      });
+    }); // end describe pagination
+  }); // end index result field sort
+}

test/functional/apps/management/_initial_state.js

@@ -1,61 +1,59 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esClient
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const log = getService('log');
+  const PageObjects = getPageObjects(['settings', 'common']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('initial state', function () {
+    before(function () {
+      // delete .kibana index and then wait for Kibana to re-create it
+      return kibanaServer.uiSettings.replace({})
+      .then(function () {
+        return PageObjects.settings.navigateTo();
+      })
+      .then(function () {
+        return PageObjects.settings.clickKibanaIndicies();
+      });
+    });
 
-bdd.describe('initial state', function () {
-  bdd.before(function () {
-    // delete .kibana index and then wait for Kibana to re-create it
-    return esClient.deleteAndUpdateConfigDoc()
-    .then(function () {
-      return PageObjects.settings.navigateTo();
-    })
-    .then(function () {
-      return PageObjects.settings.clickKibanaIndicies();
+    it('should load with time pattern checked', function () {
+      return PageObjects.settings.getTimeBasedEventsCheckbox().isSelected()
+      .then(function (selected) {
+        PageObjects.common.saveScreenshot('Settings-initial-state');
+        expect(selected).to.be.ok();
+      });
+    });
+
+    it('should load with name pattern unchecked', function () {
+      return PageObjects.settings.getTimeBasedIndexPatternCheckbox().isSelected()
+      .then(function (selected) {
+        expect(selected).to.not.be.ok();
+      });
+    });
+
+    it('should contain default index pattern', function () {
+      const defaultPattern = 'logstash-*';
+
+      return PageObjects.settings.getIndexPatternField().getProperty('value')
+      .then(function (pattern) {
+        expect(pattern).to.be(defaultPattern);
+      });
+    });
+
+    it('should not select the time field', function () {
+      return PageObjects.settings.getTimeFieldNameField().isSelected()
+      .then(function (timeFieldIsSelected) {
+        log.debug('timeField isSelected = ' + timeFieldIsSelected);
+        expect(timeFieldIsSelected).to.not.be.ok();
+      });
+    });
+
+    it('should not be enable creation', function () {
+      return PageObjects.settings.getCreateButton().isEnabled()
+      .then(function (enabled) {
+        expect(enabled).to.not.be.ok();
+      });
     });
   });
-
-  bdd.it('should load with time pattern checked', function () {
-    return PageObjects.settings.getTimeBasedEventsCheckbox().isSelected()
-    .then(function (selected) {
-      PageObjects.common.saveScreenshot('Settings-initial-state');
-      expect(selected).to.be.ok();
-    });
-  });
-
-  bdd.it('should load with name pattern unchecked', function () {
-    return PageObjects.settings.getTimeBasedIndexPatternCheckbox().isSelected()
-    .then(function (selected) {
-      expect(selected).to.not.be.ok();
-    });
-  });
-
-  bdd.it('should contain default index pattern', function () {
-    const defaultPattern = 'logstash-*';
-
-    return PageObjects.settings.getIndexPatternField().getProperty('value')
-    .then(function (pattern) {
-      expect(pattern).to.be(defaultPattern);
-    });
-  });
-
-  bdd.it('should not select the time field', function () {
-    return PageObjects.settings.getTimeFieldNameField().isSelected()
-    .then(function (timeFieldIsSelected) {
-      PageObjects.common.debug('timeField isSelected = ' + timeFieldIsSelected);
-      expect(timeFieldIsSelected).to.not.be.ok();
-    });
-  });
-
-  bdd.it('should not be enable creation', function () {
-    return PageObjects.settings.getCreateButton().isEnabled()
-    .then(function (enabled) {
-      expect(enabled).to.not.be.ok();
-    });
-  });
-});
+}

test/functional/apps/management/_kibana_settings.js

@@ -1,56 +1,57 @@
-
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esClient
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const log = getService('log');
+  const PageObjects = getPageObjects(['settings', 'common']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('creating and deleting default index', function describeIndexTests() {
+    before(function () {
+      // delete .kibana index and then wait for Kibana to re-create it
+      return kibanaServer.uiSettings.replace({})
+      .then(function () {
+        return PageObjects.settings.navigateTo();
+      })
+      .then(function () {
+        return PageObjects.settings.clickKibanaIndicies();
+      })
+      .then(function () {
+        return PageObjects.settings.createIndexPattern();
+      })
+      .then(function () {
+        return PageObjects.settings.navigateTo();
+      });
+    });
 
-bdd.describe('creating and deleting default index', function describeIndexTests() {
-  bdd.before(function () {
-    // delete .kibana index and then wait for Kibana to re-create it
-    return esClient.deleteAndUpdateConfigDoc()
-    .then(function () {
-      return PageObjects.settings.navigateTo();
-    })
-    .then(function () {
-      return PageObjects.settings.clickKibanaIndicies();
-    })
-    .then(function () {
-      return PageObjects.settings.createIndexPattern();
-    })
-    .then(function () {
-      return PageObjects.settings.navigateTo();
+    after(async function afterAll() {
+      await PageObjects.settings.navigateTo();
+      await PageObjects.settings.clickKibanaIndicies();
+      await PageObjects.settings.removeIndexPattern();
+    });
+
+    it('should allow setting advanced settings', function () {
+      return PageObjects.settings.clickKibanaSettings()
+      .then(function TestCallSetAdvancedSettingsForTimezone() {
+        PageObjects.common.saveScreenshot('Settings-advanced-tab');
+        log.debug('calling setAdvancedSetting');
+        return PageObjects.settings.setAdvancedSettings('dateFormat:tz', 'America/Phoenix');
+      })
+      .then(function GetAdvancedSetting() {
+        PageObjects.common.saveScreenshot('Settings-set-timezone');
+        return PageObjects.settings.getAdvancedSettings('dateFormat:tz');
+      })
+      .then(function (advancedSetting) {
+        expect(advancedSetting).to.be('America/Phoenix');
+      });
+    });
+
+    after(function () {
+      return PageObjects.settings.clickKibanaSettings()
+      .then(function TestCallSetAdvancedSettingsForTimezone() {
+        PageObjects.common.saveScreenshot('Settings-advanced-tab');
+        log.debug('calling setAdvancedSetting');
+        return PageObjects.settings.setAdvancedSettings('dateFormat:tz', 'UTC');
+      });
     });
   });
-
-
-  bdd.it('should allow setting advanced settings', function () {
-    return PageObjects.settings.clickKibanaSettings()
-    .then(function TestCallSetAdvancedSettingsForTimezone() {
-      PageObjects.common.saveScreenshot('Settings-advanced-tab');
-      PageObjects.common.debug('calling setAdvancedSetting');
-      return PageObjects.settings.setAdvancedSettings('dateFormat:tz', 'America/Phoenix');
-    })
-    .then(function GetAdvancedSetting() {
-      PageObjects.common.saveScreenshot('Settings-set-timezone');
-      return PageObjects.settings.getAdvancedSettings('dateFormat:tz');
-    })
-    .then(function (advancedSetting) {
-      expect(advancedSetting).to.be('America/Phoenix');
-    });
-  });
-
-  bdd.after(function () {
-    return PageObjects.settings.clickKibanaSettings()
-    .then(function TestCallSetAdvancedSettingsForTimezone() {
-      PageObjects.common.saveScreenshot('Settings-advanced-tab');
-      PageObjects.common.debug('calling setAdvancedSetting');
-      return PageObjects.settings.setAdvancedSettings('dateFormat:tz', 'UTC');
-    });
-  });
-
-
-});
+}

test/functional/apps/management/_scripted_fields.js

@@ -13,386 +13,396 @@
 
 import expect from 'expect.js';
 
-import {
-  bdd,
-  esClient
-} from '../../../support';
+export default function ({ getService, getPageObjects }) {
+  const kibanaServer = getService('kibanaServer');
+  const log = getService('log');
+  const remote = getService('remote');
+  const retry = getService('retry');
+  const PageObjects = getPageObjects(['common', 'header', 'settings', 'visualize', 'discover']);
 
-import PageObjects from '../../../support/page_objects';
+  describe('scripted fields', () => {
 
-bdd.before(async function () {
-  await PageObjects.remote.setWindowSize(1200,800);
-  // delete .kibana index and then wait for Kibana to re-create it
-  await esClient.deleteAndUpdateConfigDoc({ 'dateFormat:tz':'UTC' });
-  await PageObjects.settings.navigateTo();
-  await PageObjects.settings.clickKibanaIndicies();
-  await PageObjects.settings.createIndexPattern();
-  await esClient.updateConfigDoc({ 'dateFormat:tz':'UTC' });
-});
-
-bdd.describe('creating and using Lucence expression scripted fields', function describeIndexTests() {
-
-  const scriptedExpressionFieldName = 'ram_expr1';
-
-  bdd.it('should create scripted field', async function () {
-    await PageObjects.settings.navigateTo();
-    await PageObjects.settings.clickKibanaIndicies();
-    const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
-    await PageObjects.settings.clickScriptedFieldsTab();
-    await PageObjects.common.debug('add scripted field');
-    await PageObjects.settings
-      .addScriptedField(scriptedExpressionFieldName,
-        'expression', 'number', null, '1', 'doc[\'machine.ram\'].value / (1024 * 1024 * 1024)'
-      );
-    await PageObjects.common.try(async function() {
-      expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
-    });
-  });
-
-  bdd.it('should see scripted field value in Discover', async function () {
-    const fromTime = '2015-09-17 06:31:44.000';
-    const toTime = '2015-09-18 18:31:44.000';
-    await PageObjects.common.navigateToApp('discover');
-    await PageObjects.common.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
-    await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.discover.clickFieldListItem(scriptedExpressionFieldName);
-    await PageObjects.common.try(async function() {
-      await PageObjects.discover.clickFieldListItemAdd(scriptedExpressionFieldName);
-    });
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      const rowData = await PageObjects.discover.getDocTableIndex(1);
-      expect(rowData).to.be('September 18th 2015, 18:20:57.916 18');
+    before(async function () {
+      await remote.setWindowSize(1200,800);
+      // delete .kibana index and then wait for Kibana to re-create it
+      await kibanaServer.uiSettings.replace({ 'dateFormat:tz':'UTC' });
+      await PageObjects.settings.navigateTo();
+      await PageObjects.settings.clickKibanaIndicies();
+      await PageObjects.settings.createIndexPattern();
+      await kibanaServer.uiSettings.update({ 'dateFormat:tz':'UTC' });
     });
 
-  });
-
-  bdd.it('should filter by scripted field value in Discover', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedExpressionFieldName);
-    await PageObjects.common.debug('filter by the first value (14) in the expanded scripted field list');
-    await PageObjects.discover.clickFieldListPlusFilter(scriptedExpressionFieldName, '14');
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      expect(await PageObjects.discover.getHitCount()).to.be('31');
+    after(async function afterAll() {
+      await PageObjects.settings.navigateTo();
+      await PageObjects.settings.clickKibanaIndicies();
+      await PageObjects.settings.removeIndexPattern();
     });
-  });
 
-  bdd.it('should visualize scripted field in vertical bar chart', async function () {
-    const expectedChartValues = [ '14 31', '10 29', '7 24', '11 24', '12 23',
-      '20 23', '19 21', '6 20', '17 20', '30 20', '13 19', '18 18', '16 17', '5 16',
-      '8 16', '15 14', '3 13', '2 12', '9 10', '4 9'
-    ];
-    await PageObjects.discover.removeAllFilters();
-    await PageObjects.discover.clickFieldListItem(scriptedExpressionFieldName);
-    await PageObjects.discover.clickFieldListItemVisualize(scriptedExpressionFieldName);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.visualize.collapseChart();
-    await PageObjects.settings.setPageSize('All');
-    const data = await PageObjects.visualize.getDataTableData();
-    await PageObjects.common.debug('getDataTableData = ' + data.split('\n'));
-    await PageObjects.common.debug('data=' + data);
-    await PageObjects.common.debug('data.length=' + data.length);
-    await PageObjects.common.saveScreenshot('Visualize-vertical-bar-chart');
-    expect(data.trim().split('\n')).to.eql(expectedChartValues);
-  });
+    describe('creating and using Lucence expression scripted fields', function describeIndexTests() {
 
-});
+      const scriptedExpressionFieldName = 'ram_expr1';
 
-bdd.describe('creating and using Painless numeric scripted fields', function describeIndexTests() {
+      it('should create scripted field', async function () {
+        await PageObjects.settings.navigateTo();
+        await PageObjects.settings.clickKibanaIndicies();
+        const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
+        await PageObjects.settings.clickScriptedFieldsTab();
+        await log.debug('add scripted field');
+        await PageObjects.settings
+          .addScriptedField(scriptedExpressionFieldName,
+            'expression', 'number', null, '1', 'doc[\'machine.ram\'].value / (1024 * 1024 * 1024)'
+          );
+        await retry.try(async function() {
+          expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
+        });
+      });
 
-  const scriptedPainlessFieldName = 'ram_Pain1';
+      it('should see scripted field value in Discover', async function () {
+        const fromTime = '2015-09-17 06:31:44.000';
+        const toTime = '2015-09-18 18:31:44.000';
+        await PageObjects.common.navigateToApp('discover');
+        await log.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.discover.clickFieldListItem(scriptedExpressionFieldName);
+        await retry.try(async function() {
+          await PageObjects.discover.clickFieldListItemAdd(scriptedExpressionFieldName);
+        });
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          const rowData = await PageObjects.discover.getDocTableIndex(1);
+          expect(rowData).to.be('September 18th 2015, 18:20:57.916 18');
+        });
+
+      });
+
+      it('should filter by scripted field value in Discover', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedExpressionFieldName);
+        await log.debug('filter by the first value (14) in the expanded scripted field list');
+        await PageObjects.discover.clickFieldListPlusFilter(scriptedExpressionFieldName, '14');
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          expect(await PageObjects.discover.getHitCount()).to.be('31');
+        });
+      });
+
+      it('should visualize scripted field in vertical bar chart', async function () {
+        const expectedChartValues = [ '14 31', '10 29', '7 24', '11 24', '12 23',
+          '20 23', '19 21', '6 20', '17 20', '30 20', '13 19', '18 18', '16 17', '5 16',
+          '8 16', '15 14', '3 13', '2 12', '9 10', '4 9'
+        ];
+        await PageObjects.discover.removeAllFilters();
+        await PageObjects.discover.clickFieldListItem(scriptedExpressionFieldName);
+        await PageObjects.discover.clickFieldListItemVisualize(scriptedExpressionFieldName);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.visualize.collapseChart();
+        await PageObjects.settings.setPageSize('All');
+        const data = await PageObjects.visualize.getDataTableData();
+        await log.debug('getDataTableData = ' + data.split('\n'));
+        await log.debug('data=' + data);
+        await log.debug('data.length=' + data.length);
+        await PageObjects.common.saveScreenshot('Visualize-vertical-bar-chart');
+        expect(data.trim().split('\n')).to.eql(expectedChartValues);
+      });
 
-  bdd.it('should create scripted field', async function () {
-    await PageObjects.settings.navigateTo();
-    await PageObjects.settings.clickKibanaIndicies();
-    const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
-    await PageObjects.settings.clickScriptedFieldsTab();
-    await PageObjects.common.debug('add scripted field');
-    await PageObjects.settings
-      .addScriptedField(scriptedPainlessFieldName, 'painless', 'number', null, '1', 'doc[\'machine.ram\'].value / (1024 * 1024 * 1024)');
-    await PageObjects.common.try(async function() {
-      expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
     });
-  });
 
-  bdd.it('should see scripted field value in Discover', async function () {
-    const fromTime = '2015-09-17 06:31:44.000';
-    const toTime = '2015-09-18 18:31:44.000';
-    await PageObjects.common.navigateToApp('discover');
-    await PageObjects.common.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
-    await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName);
-    await PageObjects.common.try(async function() {
-      await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName);
+    describe('creating and using Painless numeric scripted fields', function describeIndexTests() {
+
+      const scriptedPainlessFieldName = 'ram_Pain1';
+
+      it('should create scripted field', async function () {
+        await PageObjects.settings.navigateTo();
+        await PageObjects.settings.clickKibanaIndicies();
+        const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
+        await PageObjects.settings.clickScriptedFieldsTab();
+        await log.debug('add scripted field');
+        const script = 'doc[\'machine.ram\'].value / (1024 * 1024 * 1024)';
+        await PageObjects.settings.addScriptedField(scriptedPainlessFieldName, 'painless', 'number', null, '1', script);
+        await retry.try(async function() {
+          expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
+        });
+      });
+
+      it('should see scripted field value in Discover', async function () {
+        const fromTime = '2015-09-17 06:31:44.000';
+        const toTime = '2015-09-18 18:31:44.000';
+        await PageObjects.common.navigateToApp('discover');
+        await log.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName);
+        await retry.try(async function() {
+          await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName);
+        });
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          const rowData = await PageObjects.discover.getDocTableIndex(1);
+          expect(rowData).to.be('September 18th 2015, 18:20:57.916 18');
+        });
+      });
+
+      it('should filter by scripted field value in Discover', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName);
+        await log.debug('filter by the first value (14) in the expanded scripted field list');
+        await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName, '14');
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          expect(await PageObjects.discover.getHitCount()).to.be('31');
+        });
+      });
+
+      it('should visualize scripted field in vertical bar chart', async function () {
+        const expectedChartValues = [ '14 31', '10 29', '7 24', '11 24', '12 23',
+          '20 23', '19 21', '6 20', '17 20', '30 20', '13 19', '18 18', '16 17', '5 16',
+          '8 16', '15 14', '3 13', '2 12', '9 10', '4 9'
+        ];
+        await PageObjects.discover.removeAllFilters();
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName);
+        await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.visualize.collapseChart();
+        await PageObjects.settings.setPageSize('All');
+        const data = await PageObjects.visualize.getDataTableData();
+        await log.debug('getDataTableData = ' + data.split('\n'));
+        await log.debug('data=' + data);
+        await log.debug('data.length=' + data.length);
+        await PageObjects.common.saveScreenshot('Visualize-vertical-bar-chart');
+        expect(data.trim().split('\n')).to.eql(expectedChartValues);
+      });
+
     });
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      const rowData = await PageObjects.discover.getDocTableIndex(1);
-      expect(rowData).to.be('September 18th 2015, 18:20:57.916 18');
+
+    describe('creating and using Painless string scripted fields', function describeIndexTests() {
+
+      const scriptedPainlessFieldName2 = 'painString';
+
+      it('should create scripted field', async function () {
+        await PageObjects.settings.navigateTo();
+        await PageObjects.settings.clickKibanaIndicies();
+        const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
+        await PageObjects.settings.clickScriptedFieldsTab();
+        await log.debug('add scripted field');
+        await PageObjects.settings
+          .addScriptedField(scriptedPainlessFieldName2, 'painless', 'string', null, '1',
+          'if (doc[\'response.raw\'].value == \'200\') { return \'good\'} else { return \'bad\'}');
+        await retry.try(async function() {
+          expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
+        });
+      });
+
+      it('should see scripted field value in Discover', async function () {
+        const fromTime = '2015-09-17 06:31:44.000';
+        const toTime = '2015-09-18 18:31:44.000';
+        await PageObjects.common.navigateToApp('discover');
+        await log.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await retry.try(async function() {
+          await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName2);
+        });
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          const rowData = await PageObjects.discover.getDocTableIndex(1);
+          expect(rowData).to.be('September 18th 2015, 18:20:57.916 good');
+
+        });
+      });
+
+      it('should filter by scripted field value in Discover', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await log.debug('filter by "bad" in the expanded scripted field list');
+        await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName2, 'bad');
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          expect(await PageObjects.discover.getHitCount()).to.be('27');
+        });
+        await PageObjects.discover.removeAllFilters();
+      });
+
+      it('should visualize scripted field in vertical bar chart', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName2);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.visualize.collapseChart();
+        await PageObjects.settings.setPageSize('All');
+        const data = await PageObjects.visualize.getDataTableData();
+        await log.debug('getDataTableData = ' + data.split('\n'));
+        await log.debug('data=' + data);
+        await log.debug('data.length=' + data.length);
+        expect(data.trim().split('\n')).to.eql([ 'good 359', 'bad 27' ]);
+      });
+
     });
-  });
 
-  bdd.it('should filter by scripted field value in Discover', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName);
-    await PageObjects.common.debug('filter by the first value (14) in the expanded scripted field list');
-    await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName, '14');
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      expect(await PageObjects.discover.getHitCount()).to.be('31');
+
+    describe('creating and using Painless boolean scripted fields', function describeIndexTests() {
+
+      const scriptedPainlessFieldName2 = 'painBool';
+
+      it('should create scripted field', async function () {
+        await PageObjects.settings.navigateTo();
+        await PageObjects.settings.clickKibanaIndicies();
+        const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
+        await PageObjects.settings.clickScriptedFieldsTab();
+        await log.debug('add scripted field');
+        await PageObjects.settings
+          .addScriptedField(scriptedPainlessFieldName2, 'painless', 'boolean', null, '1',
+          'doc[\'response.raw\'].value == \'200\'');
+        await retry.try(async function() {
+          expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
+        });
+      });
+
+      it('should see scripted field value in Discover', async function () {
+        const fromTime = '2015-09-17 06:31:44.000';
+        const toTime = '2015-09-18 18:31:44.000';
+        await PageObjects.common.navigateToApp('discover');
+        await log.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await retry.try(async function() {
+          await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName2);
+        });
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          const rowData = await PageObjects.discover.getDocTableIndex(1);
+          expect(rowData).to.be('September 18th 2015, 18:20:57.916 true');
+
+        });
+      });
+
+      it('should filter by scripted field value in Discover', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await log.debug('filter by "true" in the expanded scripted field list');
+        await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName2, 'true');
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          expect(await PageObjects.discover.getHitCount()).to.be('359');
+        });
+        await PageObjects.discover.removeAllFilters();
+      });
+
+      it('should visualize scripted field in vertical bar chart', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName2);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.visualize.collapseChart();
+        await PageObjects.settings.setPageSize('All');
+        const data = await PageObjects.visualize.getDataTableData();
+        await log.debug('getDataTableData = ' + data.split('\n'));
+        await log.debug('data=' + data);
+        await log.debug('data.length=' + data.length);
+        expect(data.trim().split('\n')).to.eql([ 'true 359', 'false 27' ]);
+      });
+
     });
-  });
 
-  bdd.it('should visualize scripted field in vertical bar chart', async function () {
-    const expectedChartValues = [ '14 31', '10 29', '7 24', '11 24', '12 23',
-      '20 23', '19 21', '6 20', '17 20', '30 20', '13 19', '18 18', '16 17', '5 16',
-      '8 16', '15 14', '3 13', '2 12', '9 10', '4 9'
-    ];
-    await PageObjects.discover.removeAllFilters();
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName);
-    await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.visualize.collapseChart();
-    await PageObjects.settings.setPageSize('All');
-    const data = await PageObjects.visualize.getDataTableData();
-    await PageObjects.common.debug('getDataTableData = ' + data.split('\n'));
-    await PageObjects.common.debug('data=' + data);
-    await PageObjects.common.debug('data.length=' + data.length);
-    await PageObjects.common.saveScreenshot('Visualize-vertical-bar-chart');
-    expect(data.trim().split('\n')).to.eql(expectedChartValues);
-  });
 
-});
+    describe('creating and using Painless date scripted fields', function describeIndexTests() {
 
-bdd.describe('creating and using Painless string scripted fields', function describeIndexTests() {
+      const scriptedPainlessFieldName2 = 'painDate';
 
-  const scriptedPainlessFieldName2 = 'painString';
+      it('should create scripted field', async function () {
+        await PageObjects.settings.navigateTo();
+        await PageObjects.settings.clickKibanaIndicies();
+        const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
+        await PageObjects.settings.clickScriptedFieldsTab();
+        await log.debug('add scripted field');
+        await PageObjects.settings
+          .addScriptedField(scriptedPainlessFieldName2, 'painless', 'date',
+          { format: 'Date', datePattern: 'YYYY-MM-DD HH:00' }, '1',
+          'doc[\'utc_time\'].value.getMillis() + (1000) * 60 * 60');
+        await retry.try(async function() {
+          expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
+        });
+      });
 
-  bdd.it('should create scripted field', async function () {
-    await PageObjects.settings.navigateTo();
-    await PageObjects.settings.clickKibanaIndicies();
-    const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
-    await PageObjects.settings.clickScriptedFieldsTab();
-    await PageObjects.common.debug('add scripted field');
-    await PageObjects.settings
-      .addScriptedField(scriptedPainlessFieldName2, 'painless', 'string', null, '1',
-      'if (doc[\'response.raw\'].value == \'200\') { return \'good\'} else { return \'bad\'}');
-    await PageObjects.common.try(async function() {
-      expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
-    });
-  });
+      it('should see scripted field value in Discover', async function () {
+        const fromTime = '2015-09-17 19:22:00.000';
+        const toTime = '2015-09-18 07:00:00.000';
+        await PageObjects.common.navigateToApp('discover');
+        await log.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
+        await PageObjects.header.setAbsoluteRange(fromTime, toTime);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await retry.try(async function() {
+          await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName2);
+        });
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          const rowData = await PageObjects.discover.getDocTableIndex(1);
+          expect(rowData).to.be('September 18th 2015, 06:52:55.953 2015-09-18 07:00');
+        });
+      });
 
-  bdd.it('should see scripted field value in Discover', async function () {
-    const fromTime = '2015-09-17 06:31:44.000';
-    const toTime = '2015-09-18 18:31:44.000';
-    await PageObjects.common.navigateToApp('discover');
-    await PageObjects.common.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
-    await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.common.try(async function() {
-      await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName2);
-    });
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      const rowData = await PageObjects.discover.getDocTableIndex(1);
-      expect(rowData).to.be('September 18th 2015, 18:20:57.916 good');
+      it('should filter by scripted field value in Discover', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await log.debug('filter by "2015-09-17 23:00" in the expanded scripted field list');
+        await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName2, '2015-09-17 23:00');
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await retry.try(async function() {
+          expect(await PageObjects.discover.getHitCount()).to.be('1');
+        });
+        await PageObjects.discover.removeAllFilters();
+      });
+
+      it('should visualize scripted field in vertical bar chart', async function () {
+        await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
+        await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName2);
+        await PageObjects.header.waitUntilLoadingHasFinished();
+        await PageObjects.visualize.waitForVisualization();
+        await PageObjects.visualize.collapseChart();
+        await PageObjects.settings.setPageSize('All');
+        const data = await PageObjects.visualize.getDataTableData();
+        await log.debug('getDataTableData = ' + data.split('\n'));
+        await log.debug('data=' + data);
+        await log.debug('data.length=' + data.length);
+        expect(data.trim().split('\n')).to.eql([
+          '2015-09-17 20:00 1',
+          '2015-09-17 21:00 1',
+          '2015-09-17 23:00 1',
+          '2015-09-18 00:00 1',
+          '2015-09-18 03:00 1',
+          '2015-09-18 04:00 1',
+          '2015-09-18 04:00 1',
+          '2015-09-18 04:00 1',
+          '2015-09-18 04:00 1',
+          '2015-09-18 05:00 1',
+          '2015-09-18 05:00 1',
+          '2015-09-18 05:00 1',
+          '2015-09-18 05:00 1',
+          '2015-09-18 06:00 1',
+          '2015-09-18 06:00 1',
+          '2015-09-18 06:00 1',
+          '2015-09-18 06:00 1',
+          '2015-09-18 07:00 1',
+          '2015-09-18 07:00 1',
+          '2015-09-18 07:00 1',
+        ]);
+      });
 
     });
   });
-
-  bdd.it('should filter by scripted field value in Discover', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.common.debug('filter by "bad" in the expanded scripted field list');
-    await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName2, 'bad');
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      expect(await PageObjects.discover.getHitCount()).to.be('27');
-    });
-    await PageObjects.discover.removeAllFilters();
-  });
-
-  bdd.it('should visualize scripted field in vertical bar chart', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName2);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.visualize.collapseChart();
-    await PageObjects.settings.setPageSize('All');
-    const data = await PageObjects.visualize.getDataTableData();
-    await PageObjects.common.debug('getDataTableData = ' + data.split('\n'));
-    await PageObjects.common.debug('data=' + data);
-    await PageObjects.common.debug('data.length=' + data.length);
-    expect(data.trim().split('\n')).to.eql([ 'good 359', 'bad 27' ]);
-  });
-
-});
-
-
-bdd.describe('creating and using Painless boolean scripted fields', function describeIndexTests() {
-
-  const scriptedPainlessFieldName2 = 'painBool';
-
-  bdd.it('should create scripted field', async function () {
-    await PageObjects.settings.navigateTo();
-    await PageObjects.settings.clickKibanaIndicies();
-    const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
-    await PageObjects.settings.clickScriptedFieldsTab();
-    await PageObjects.common.debug('add scripted field');
-    await PageObjects.settings
-      .addScriptedField(scriptedPainlessFieldName2, 'painless', 'boolean', null, '1',
-      'doc[\'response.raw\'].value == \'200\'');
-    await PageObjects.common.try(async function() {
-      expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
-    });
-  });
-
-  bdd.it('should see scripted field value in Discover', async function () {
-    const fromTime = '2015-09-17 06:31:44.000';
-    const toTime = '2015-09-18 18:31:44.000';
-    await PageObjects.common.navigateToApp('discover');
-    await PageObjects.common.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
-    await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.common.try(async function() {
-      await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName2);
-    });
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      const rowData = await PageObjects.discover.getDocTableIndex(1);
-      expect(rowData).to.be('September 18th 2015, 18:20:57.916 true');
-
-    });
-  });
-
-  bdd.it('should filter by scripted field value in Discover', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.common.debug('filter by "true" in the expanded scripted field list');
-    await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName2, 'true');
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      expect(await PageObjects.discover.getHitCount()).to.be('359');
-    });
-    await PageObjects.discover.removeAllFilters();
-  });
-
-  bdd.it('should visualize scripted field in vertical bar chart', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName2);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.visualize.collapseChart();
-    await PageObjects.settings.setPageSize('All');
-    const data = await PageObjects.visualize.getDataTableData();
-    await PageObjects.common.debug('getDataTableData = ' + data.split('\n'));
-    await PageObjects.common.debug('data=' + data);
-    await PageObjects.common.debug('data.length=' + data.length);
-    expect(data.trim().split('\n')).to.eql([ 'true 359', 'false 27' ]);
-  });
-
-});
-
-
-bdd.describe('creating and using Painless date scripted fields', function describeIndexTests() {
-
-  const scriptedPainlessFieldName2 = 'painDate';
-
-  bdd.it('should create scripted field', async function () {
-    await PageObjects.settings.navigateTo();
-    await PageObjects.settings.clickKibanaIndicies();
-    const startingCount = parseInt(await PageObjects.settings.getScriptedFieldsTabCount());
-    await PageObjects.settings.clickScriptedFieldsTab();
-    await PageObjects.common.debug('add scripted field');
-    await PageObjects.settings
-      .addScriptedField(scriptedPainlessFieldName2, 'painless', 'date',
-      { format: 'Date', datePattern: 'YYYY-MM-DD HH:00' }, '1',
-      'doc[\'utc_time\'].value.getMillis() + (1000) * 60 * 60');
-    await PageObjects.common.try(async function() {
-      expect(parseInt(await PageObjects.settings.getScriptedFieldsTabCount())).to.be(startingCount + 1);
-    });
-  });
-
-  bdd.it('should see scripted field value in Discover', async function () {
-    const fromTime = '2015-09-17 19:22:00.000';
-    const toTime = '2015-09-18 07:00:00.000';
-    await PageObjects.common.navigateToApp('discover');
-    await PageObjects.common.debug('setAbsoluteRange (' + fromTime + ') to (' + toTime + ')');
-    await PageObjects.header.setAbsoluteRange(fromTime, toTime);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.common.try(async function() {
-      await PageObjects.discover.clickFieldListItemAdd(scriptedPainlessFieldName2);
-    });
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      const rowData = await PageObjects.discover.getDocTableIndex(1);
-      expect(rowData).to.be('September 18th 2015, 06:52:55.953 2015-09-18 07:00');
-    });
-  });
-
-  bdd.it('should filter by scripted field value in Discover', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.common.debug('filter by "2015-09-17 23:00" in the expanded scripted field list');
-    await PageObjects.discover.clickFieldListPlusFilter(scriptedPainlessFieldName2, '2015-09-17 23:00');
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.common.try(async function() {
-      expect(await PageObjects.discover.getHitCount()).to.be('1');
-    });
-    await PageObjects.discover.removeAllFilters();
-  });
-
-  bdd.it('should visualize scripted field in vertical bar chart', async function () {
-    await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
-    await PageObjects.discover.clickFieldListItemVisualize(scriptedPainlessFieldName2);
-    await PageObjects.header.waitUntilLoadingHasFinished();
-    await PageObjects.visualize.waitForVisualization();
-    await PageObjects.visualize.collapseChart();
-    await PageObjects.settings.setPageSize('All');
-    const data = await PageObjects.visualize.getDataTableData();
-    await PageObjects.common.debug('getDataTableData = ' + data.split('\n'));
-    await PageObjects.common.debug('data=' + data);
-    await PageObjects.common.debug('data.length=' + data.length);
-    expect(data.trim().split('\n')).to.eql([
-      '2015-09-17 20:00 1',
-      '2015-09-17 21:00 1',
-      '2015-09-17 23:00 1',
-      '2015-09-18 00:00 1',
-      '2015-09-18 03:00 1',
-      '2015-09-18 04:00 1',
-      '2015-09-18 04:00 1',
-      '2015-09-18 04:00 1',
-      '2015-09-18 04:00 1',
-      '2015-09-18 05:00 1',
-      '2015-09-18 05:00 1',
-      '2015-09-18 05:00 1',
-      '2015-09-18 05:00 1',
-      '2015-09-18 06:00 1',
-      '2015-09-18 06:00 1',
-      '2015-09-18 06:00 1',
-      '2015-09-18 06:00 1',
-      '2015-09-18 07:00 1',
-      '2015-09-18 07:00 1',
-      '2015-09-18 07:00 1'
-    ]);
-  });
-
-});
+}