Commit graph

18 commits

Author SHA1 Message Date
Spencer
c93160484e
[5.6] [dev/mocha/junit] report real skipped test count and errors from hooks (#15465) (#15478)
* [dev/mocha/junit] report real skipped test count and errors from hooks (#15465)

* [dev/mocha/junit] report real skipped test count and errors from hooks

* typo

* [grunt/simplemocha] update simplemocha patterns to exclude fixtures
2017-12-27 12:48:08 -07:00
Spencer
5c5e03a968
[5.6] [CI] Produce junit test reports (#15281) (#15442)
* [CI] Produce junit test reports (#15281)

* [mocha] use custom reporter for legible results in jenkins

* [jest] use custom result processor for legible results in jenkins

* [karma] enable junit output on CI

* [mocha/junitReporter] accept rootDirectory as configuration

* [jest/reporter] use reporters option added in jest 20

* [toolingLog] remove black/white specific colors

* [dev/mocha/junit] no reason for junit to be a "reporter"

* typos

* [dev/mocha/junit] use else if

* [karma/junit] use string#replace for explicitness

* [junit] use test file path as "classname"

* [ftr/mocha] no longer a "console" specific reporter

* [dev/jest] fix relative path to babel options
2017-12-05 19:41:07 -07:00
Spencer
098242d687 Remove use of npm ls in grunt tasks (#11965)
* [grunt/build] refactor _build:notice task to not depend on npm

The _build:notice task used to rely on the output of `npm ls` to determine where modules were defined, but the task now just asks `license-checker` to include the `realPath` of the modules it describes in it's output, which is ultimately the same thing but works with `yarn` too.

* [grunt/licenses] convert to use lib/packages/getInstalledPackages()

* [grunt/notice/generate] test generateNoticeText()

* [grunt/licenses] tested assertLicensesValid()

* [npm] remove npm dev dep

* [tasks/lib/packages] do not include kibana in "installed packages"

* [tasks/lib/notice] join all notices with the same separator

(cherry picked from commit 5c04ff65fb)
2017-05-24 08:41:38 -07:00
Spencer
8e7b7ebce7 [ftr] make room for more projects (#11848)
Squashed commit of the following:

commit 659ea986fdeb9a5ff2ca1fa5360cccb01c671ede
Author: spalger <spalger@users.noreply.github.com>
Date:   Wed May 17 09:19:22 2017 -0700

    [test/*/fixtures] rename es_archives to es_archiver

commit d3667457c78e88e2d6974f3c38dd0fe61b846b91
Author: spalger <spalger@users.noreply.github.com>
Date:   Wed May 17 08:22:03 2017 -0700

    [ftr/config] fix default directory value functions

commit 9a6a2cc0b295e2281e83da04fcea40e9d6f00781
Author: spalger <spalger@users.noreply.github.com>
Date:   Wed May 17 07:30:52 2017 -0700

    fix import paths

commit fcb65a877d54b5b1d36b8c81f1264b36845e826e
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 21:39:57 2017 -0700

    [grunt/ftr] use named exports for configs

commit 7d7f38c7615cdbf8eb0119efc0f2a5188bca8792
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 18:50:04 2017 -0700

    [test] remove unnecessary directory definitions

commit 0c28984669768482f0a2ee7fc2800d5bcaf49025
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 18:46:29 2017 -0700

    [ftr/config] make default directories relative to config path

commit cd2f33612624cacffec138797f3fc0f4ecb46cca
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 18:32:12 2017 -0700

    [test/common] put server config into common

commit 7851ed811a236576c63bd20850b3ef2099be2a4e
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 18:18:20 2017 -0700

    [grunt] "deprecate" test:api:runner task

commit b2ac4c26593a1947c94f0168191fe8123ff74122
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 18:15:41 2017 -0700

    [ftr] accept the project name as an unnamed arg

commit 47e292894fc70c0a04883403c50c5d2ae0738d76
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 17:56:34 2017 -0700

    [ftr/grunt] convert ftr task to multi-task with config

commit 83375855f88e5e7b3fa8b6a1c5d24a9f54766ce5
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 17:54:54 2017 -0700

    [test/functional] move fixtures into test/functional project

commit 05994e9c92cf134c58f831c285b3b522a801acbc
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 17:02:51 2017 -0700

    [src/test_utils] merge with test/utils directory

commit c77ee5ed36b8b7eadf876cb6d9482a49dfc92b66
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 16:53:00 2017 -0700

    [test/api_integration] migrate api tests to functional test runner

commit ca328c34648dd7e07f70e1844e07cfc392e41103
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 16:50:12 2017 -0700

    [esArchiver] refresh modified indices after load

commit cde74a540850fd97578f441d6dccaefd1444e656
Author: spalger <spalger@users.noreply.github.com>
Date:   Tue May 16 16:46:48 2017 -0700

    [test/functional] move shared services into test/common

commit 0ea2646aea5817f6d1595e6ae0d356c426f138f0
Author: spalger <spalger@users.noreply.github.com>
Date:   Mon May 15 22:51:23 2017 -0700

    [scripts/mocha] run _mocha script when debugging

commit 1cc80600d90e318d4738920aa557d124075a4570
Author: spalger <spalger@users.noreply.github.com>
Date:   Mon May 15 22:48:12 2017 -0700

    [ftr/config] allow child config files to have no testFiles config

commit 2bb6c957443b18cebc419baa6f9db301c8f4dc4f
Author: spalger <spalger@users.noreply.github.com>
Date:   Mon May 15 21:52:26 2017 -0700

    [ftr] move screenshots into test/functional
(cherry picked from commit d3ba428796)
2017-05-17 20:30:28 -07:00
Spencer
72f716b5e6 [5.x] [functionalTestRunner] replace intern (#10910) (#11171)
* [esArchiver] combine elasticDump and ScenarioManager (#10359)

* As a part of bringing functional testing to plugins, esArchiver gives these plugins a way to capture and reload es indexes without needing to write a bunch of custom code. It works similarly to the elasticDump and ScenarioManager tools that it replaces.

Differences:
  - Streaming implementation allows for much larger archives
  - CLI for creating and using archives
  - Configurable archive location
  - Stores the data in gzipped files (better for source control, searching, large archives)
  - Automatically identifies and upgrades Kibana config documents

Methods:
  - `#load(name)`: import an archive
  - `#loadIfNeeded(name)`: import an archive, but skip the documents what belong to any existing index
  - `#unload(name)`: delete the indexes stored in an archive

CLI operations:
  - `./bin/es_archiver save <name> [index patterns...]`: save the mapping and documents in one or more indexes that match the wild-card patterns into an the `<name>` archive
  - `./bin/es_archiver load <name>`: load the mapping and documents from the `<name>` archive

* [functional_tests/common/nagivate] check for statusPage

* [es_archiver] move bins into new scripts dir

* [functional_tests/apps/context] use esArchiver

* [esArchiver] general improvements after showing to a few folks

 - remove auto-upgrading config doc logic (until we have better access to kibana version info)
 - export unload command
 - remove preemptive checks in favor of reacting to errors
 - use type "doc" vs "hit" for doc records (consistency)
 - wrote a bunch of pending tests to think though and plan

* [esArchiver] make log a stream that writes to itself

* [esArchiver] fill in stats and archive format tests

* [esArchiver] splitup action logic

* [esArchiver/cli] fix cli --help output and comment

* [esArchiver] remove type-based param coercion

* [esArchiver/log] use strings for log levels

* [esArchvier] remove unused var

* [esArchiver/indexDocRecordsStream] add tests

* [esArchive] fill in remaining tests

* [esArchiver] fix dem tests

* [eslint] remove unused vars

* [esArchiver/loadIfNeeded] fix call to load()

* [esArchiver] remove loadDumpData helpers

* [esArchiver] update archives for 5.x

* [functionalTestRunner] replace intern (#10910)

* [functional_test_runner] replace functional testing tools with custom/pluggable solution

* [functional_test_runner] Convert unit tests to commonjs format

* [functional_test_runner] Fix dashboard test in wrong mode

* [functional_test_runner] Add dashboardLandingPage test subject

* [functional_test_runner] Get Visualize page object

* [functional_test_runner] Fix outdated references

* [functional_test_runner] Fix more outdated refs

* [functional_test_runner] Remove duplicate tests

* [functional_test_runner] Improve test readability

* [functional_test_runner] 😞 So many duplicate methods

* [functional_test_runner] Move mgmt `before` outside toplevel describe

* [functional_test_runner] Settings page obj missing methods

* [functional_test_runner] Add improvements from @gammon

* [functional_test_runner] Fix return statements in async funcs

* [functional_test_runner] Move before() to correct scope

* [functional_test_runner] Add after() hooks to remove index patterns

* [functional_test_runner] Attempt to fix vertical bar chart tests

* [functional_test_runner] Clean up

* [functional_test_runner] Reinstate unit tests

* [functional_test_runner] Set default loglevel back to info

* [functional_test_runner] Replace `context`s with `describe`s

* [functional_test_runner] Better error handling

* [functional_test_runner] Add in new Tile Map tests

* Incorporate changes from master

* [functional_test_runner] validate that every test file has a single top-level suite

* Update contributing doc with link to full doc

* [docs] Spelling and grammar fixes

* docs: writing and running functional tests

* [docs] Move plugin doc to plugin area

* [docs] Housekeeping. Doc in wrong place

* [docs] Remove dup doc file

* [grunt] Only run mocha_setup when running tests, not every grunt task

* [eslint] remove use of context()
2017-04-12 08:44:53 -04:00
jasper
8a748e75c7 Use spec reporter for Mocha tests (#9476)
Backports PR #9470

**Commit 1:**
Use spec reporter for Mocha tests

Provides context for exceptions and node warnings (ex: UnhandledPromiseRejectionWarning)

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* Original sha: 786b54d049
* Authored by Tyler Smalley <tyler.smalley@elastic.co> on 2016-12-13T22:38:57Z
2016-12-13 20:28:09 -08:00
jasper
1cce23cbd9 [server/logger] downgrade EPIPE errors to debug level (#9041)
Backports PR #9023

**Commit 1:**
[server/logger] downgrade EPIPE errors to debug level

* Original sha: 4703b367aa
* Authored by spalger <email@spalger.com> on 2016-11-09T23:48:48Z

**Commit 2:**
fix variable naming

* Original sha: fd272c5b3b
* Authored by spalger <email@spalger.com> on 2016-11-11T01:06:55Z
2016-11-15 01:39:04 -07:00
Nicolás Bevacqua
7e191b82dd [fix] @spalger identified as NSA mole. Run server-side tests in src/ui directory. 2016-07-06 15:14:25 -03:00
Matthew Bargar
81d48659cc [API] Add Highland's nil to mocha's list of globals 2016-04-29 16:47:29 -04:00
spalger
7773625690 [simplemocha] always perform mocha setup if grunt task is loaded 2016-01-14 10:32:28 -07:00
spalger
7b15ee05d1 [mochaSetup] added notes to explain the purpose for the workarounds in place 2016-01-13 16:50:55 -07:00
spalger
c87aec3dae [mocha] move setup work into module
with https://github.com/elastic/kibana/pull/5553 we added command line flags the told mocha it was supposed to use babel. This changes the strategy here and instead uses mocha's -r option (and it's mocha.opts file to specify it). This means that using mocha directly from the command line still works, and that we have a place where we can do other setup work.
2016-01-12 16:26:42 -07:00
Jonathan Budzenski
90969d9308 [functional testing] Add url configs and util to get pages 2015-10-21 14:26:42 -05:00
Jonathan Budzenski
db8f7cdf22 [functional testing] Load scenarios 2015-10-14 15:33:25 -05:00
spalger
1c0720a9e5 goodbye mocha timeouts 2015-08-09 23:19:11 -07:00
spalger
2bcf846993 get the server tests passing again 2015-07-26 10:05:57 -07:00
Spencer Alger
d45c1c7900 [grunt:version] added tests 2015-02-19 20:25:34 -07:00
Chris Cowan
2b9802030c Closes #2377 - Migrate config from > GA
- Closes #2377
- Added testing framework for server side tests
- Fixed jshint
- Added config migration
- Added framework for server initialization
2015-01-23 11:34:29 -07:00