Commit graph

3163 commits

Author SHA1 Message Date
Brian Seeders
99ff78c3f8
Re-add pipeline for flaky test runner job (#48781) (#49907)
* Revert "Revert "Add pipeline for flaky test runner job (#46740)""

This reverts commit 7d96a13fad.

Also reconcile changes to Jenkinsfile since original revert happened

* Fix param parsing and add missed change

* Add missing variable
2019-11-15 11:56:12 -05:00
Dmitry Lemeshko
331be3c384
[7.4] FTR: clear browser storage between test suites (#48683) (#49477)
* FTR: clear browser storage between test suites (#48683)

* clear browser storage on test suite completion

* [logstash/pipeline_list] fix tests dependency

* always load new index data

* remove files
2019-11-07 09:57:53 -05:00
Spencer
38560d24bc
[7.4] use a custom wrapper around chance with longer defaults… (#49869)
* use a custom wrapper around chance with longer defaults to avoid conflicts (#49653)

* use a custom wrapper around chance with longer defaults to avoid conflicts

* fix a reference to chance

* fix another reference to chance service

# Conflicts:
#	x-pack/test/api_integration/services/index.ts
#	x-pack/test/saml_api_integration/apis/security/saml_login.ts
#	x-pack/test/saml_api_integration/config.ts
#	x-pack/test/saml_api_integration/services.ts

* remove 7.4 specific chance reference
2019-10-31 10:28:02 -07:00
Spencer
9f21652487
[7.4] FTR: fix "DevToolsActivePort file doesn't exist" error o… (#49835) 2019-10-31 08:52:00 -07:00
spalger
0b70c19a1a disable flaky test #49088 2019-10-23 21:14:40 -07:00
Spencer
a96d582aea
[7.4] standardize notice-comment usage (#48677) (#48816)
* standardize notice-comment usage

* regenerate notice.txt
2019-10-21 15:55:38 -07:00
Rashmi Kulkarni
610cd91cf7
Management Saved object test (#48691) (#48707) 2019-10-18 18:03:31 -07:00
Dmitry Lemeshko
6672cf3054
Improve recently failed tests (#48389) (#48574)
* [apps/management] improve import objects tests

* [page_objects/dashboard_page] replace link navigation with direct url

* [dashboard/time_zones] reset date format via API

* update jest snapshot

* [management/_kibana_settings] reset date format via API

* revert changees to dashboard tests

* run ciGroup 7 & 8 15x times

* Revert "run ciGroup 7 & 8 15x times"

This reverts commit 8be64d52dc.
2019-10-17 22:12:50 +02:00
Dmitry Lemeshko
03fcbe1ab8
Fix several recently failed functional tests (#47976) (#48105)
* small tweaks to make tests more stable

* fix empty list check, update more tests

* fix
2019-10-14 17:48:33 +02:00
Matthias Wilhelm
f28b1db903
Fix visualize app chart type test (#48071) (#47807) 2019-10-14 11:11:18 +02:00
Bhavya RM
8dbdd01ecb
Fixing gauge test in firefox (#47082) (#47896)
Unskip gauge test in firefox
2019-10-10 17:28:16 -04:00
Matthias Wilhelm
eb13fa3691
Fix homepage app sample data dashboard test failures in cloud (#47737) (#47835)
* Improve launchSampleData of HomePageProvider for cloud testing

* Migrate to typescript
2019-10-10 18:16:39 +02:00
Matthias Wilhelm
4f3945447a
Fix failing cloud test that encodes portions of the URL (#47600) (#47790)
* Improve generation of Kibana base path

* Review adaptions + convert to Typescript
2019-10-10 09:49:18 +02:00
Spencer
9293ea9ff5
[7.4] refactor failed_tests_reporter to use TS, no octokit (#4… (#47632)
* refactor failed_tests_reporter to use TS, no octokit (#46993)

* refactor failed_tests_reporter to use TS, no octokit

* update renovate config

* ensure that all kbn-test files are in ts project

* fix some type errors

* add some more tests

* [kbn-test/githubapi] cleanup and document

* collect log messages as strings instead of message objects

* ensure issue is open when updating body

* improve readability of getKibanaIssues

* expose axios helpers from dev-utils

* fix request params for fetching github issues and validate locally

* include a README for failed_tests_reporter

* improve axios error helpers

# Conflicts:
#	package.json
#	packages/kbn-test/src/index.ts
#	packages/kbn-test/src/mocha/junit_report_generation.js
#	renovate.json5
#	src/dev/jest/junit_reporter.js
#	yarn.lock

* ensure all of kbn-test is in ts project
2019-10-09 09:47:19 -07:00
Matthias Wilhelm
de271780fc
[discover] fix shared links tests for cloud testing (#47572) (#47655)
* Add method to open permalinks menu entry if available

* Migrate to TypeScript
2019-10-09 10:02:41 +02:00
Court Ewing
dc105a41b5
[core] Do not overwrite saved object references if not specified (#47248) (#47520)
* saved objects: allow partial update without references

For normal attributes, the update API for saved objects supports partial
updates, where it will only attempt to change those attributes you
specify. References should behave the same way otherwise they will be
replaced entirely if you call update without specifying the original
references.
2019-10-08 10:37:24 -04:00
Dmitry Lemeshko
b5654b4660
[services/remote] print Firefox console logs in stdout (#46700) (#47532)
* [services/remote] print Firefox console logs in stdout

* [ftr/firefox] setup a socket for firefox to write stdout to

* remove unused imports

* unsubscribe from polling logs on cleanup

* tear down more completely on cleanup
2019-10-08 09:00:39 +02:00
Dmitry Lemeshko
7e253b58a2
[functional/services] update webdriver lib and types (#47381) (#47446)
* [functional/services] refactor using new types

* [code/history] wrap forward navigation with browser service
2019-10-07 13:28:24 +02:00
Robert Oskamp
8019844732
[ML] Fix flaky setValue for some elements (#46693) (#47096)
Use setValue with clearing via keyboard
2019-10-02 15:33:15 +02:00
Spencer
f2ecf10c32
[7.4] [dev-utils] implement basic KbnClient util for talking t… (#47004)
* [dev-utils] implement basic KbnClient util for talking to Kibana server

* update KbnClient to expose full KibanaServerService API

* expose request() function and uriencode helper

* [uiSettings] retry read on conflicts auto upgrading

* expose function for resolving a Kibana server url

* only use apis in test hooks

* run x-pack-ciGroup2 60 times

* log retries as errors so they are included in console output for job

* bump

* Revert "run x-pack-ciGroup2 60 times"

This reverts commit 6b6f392edf.

* refactor urlencode tag to be a little clearer

* support customizing maxAttempts in request method
2019-10-01 00:12:37 -07:00
Dmitry Lemeshko
c4bace0f33
Fix tsvb chart test (#44657) (#46860)
* enable switch index patterns test

* run ciGroup6 20x times

* [page_objects/visual_builder_page] retry selecting index pattern

* Revert "run ciGroup6 20x times"

This reverts commit 2151ea9905.

* [services/combo_box] clear input in seperate function

* run ciGroup12 30x times

* update logging message

* Revert "run ciGroup12 30x times"

This reverts commit 08fac11139.
2019-09-28 09:06:01 +03:00
Spencer
a0fe3e1efc
[7.4] Fix disappearing screenshots in CI (#46382) (#46663) 2019-09-26 07:31:32 -07:00
Spencer
c572c01ebc
[7.4] Re-split ciGroups after pipeline rollout (#46375) (#46644)
* Re-split ciGroups after pipeline rollout

Revert "Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)"

This reverts commit 9a109f2170.

Revert "set IS_PIPELINE_JOB in intake jobs (#45850)"

This reverts commit b1a01effa8.

* Split one of the slow test suites up to try to make overall CI faster

* Disable visualRegression groups, they are being handled in other work

* Revert "Split one of the slow test suites up to try to make overall CI faster"

This reverts commit 1213239545.

* Move some different xpack ciGroup8 suites around
2019-09-25 17:22:06 -07:00
Spencer
5cc48622af
[7.4] Update angular+jquery related packages (#46393) (#46633)
* Update angular related packages

* upgrade jQuery too

* don't delete screenshots in pipeline jobs

* use ng-change so that listener is triggered after ng-model is updated

* match @types/angular versions

* update yarn.lock

* fix test

$routeChangeSuccess is no longer broadcast when resolving a redirect route, so this test has been adjusted to validate that

# Conflicts:
#	package.json
#	x-pack/package.json
#	yarn.lock
2019-09-25 15:11:58 -07:00
Dmitry Lemeshko
ad4d2cc336
Re-enable tsvb math aggs test (#46188) (#46256)
* re-enable tsvb  Math Aggregation test

* run ciGroup6 15x times

* Revert "run ciGroup6 15x times"

This reverts commit 19d94befcb.
2019-09-22 12:18:48 +03:00
Dmitry Lemeshko
8c3988d47c
Fix/markdown functional tests (#46216) (#46263)
* [page_objects/visual_builder_page] improve text area input clearing and typing value

* run ciGroup6 15x times

* Revert "run ciGroup6 15x times"

This reverts commit d94491769b.
2019-09-22 12:13:10 +03:00
spalger
cebb9f4fca skip flaky suite (#46305)
(cherry picked from commit 5b67cd5e4f)
2019-09-21 11:35:57 -07:00
liza-mae
d2687ea30b
Revert hiding Kibana icon, test Percy rendering (#43990) (#46169)
* Revert hiding Kibana icon, test Percy rendering

* Update jobs.yml

* Use percy token for kibana dev

* Update Percy token to read from kibana-issues

* Read correct field

* increase timeout

* Run each visual job 10 times

* Revert last commit

* Increase timeout

* Disable visual tests and revert percy token

* Fix jest tests

* Another jest test fix
2019-09-19 15:35:58 -06:00
Lee Drengenberg
1136d5eac2
use correct id in setting default index pattern in dashboard tests (#45601) (#46012)
* replace guid with logstash-* in data for default index pattern

* revert change to data archive, instead fix index pattern id
2019-09-18 18:30:11 -05:00
spalger
58fc22e699 disable flaky suite (#46085)
(cherry picked from commit 22e15b8bcc)
2019-09-18 14:26:49 -07:00
Dmitry Lemeshko
42357cf5b6
Fix/tsvb functional tests (#45868) (#46071)
* [page_objects/visual_builder_page] increase timeout & remove FF sleep

* unskip flaky test suites

* run ciGroup6 12x times

* fix tsvb_markdown tests

* skip aggs test

* fix waiting

* run FF tests 8x times

* Revert "run FF tests 8x times"

This reverts commit 75ef2ced46.

* Revert "run ciGroup6 12x times"

This reverts commit 217dbb818c.

* Update test/functional/page_objects/visual_builder_page.ts

review fix

Co-Authored-By: Spencer <email@spalger.com>
2019-09-18 22:49:47 +02:00
Dmitry Lemeshko
a4ab656449
Update chromedriver to 77 (#45614) (#46026)
* bump chromedriver to v77

* fix file import
2019-09-18 22:39:41 +02:00
Brian Seeders
7677f02390
[7.4] Run pipeline scripts with runbld (#45844) (#46021)
Change how runbld is invoked, and skip junit processing by default
2019-09-18 13:17:03 -04:00
Spencer
d5af3d5ea1
[7.4] [ftr/savedObjects] add simple saved object api client to… (#45941)
* [ftr/savedObjects] add simple saved object api client to ftr services

* fix typo

* use consistent spacing

* fix types and TS-ify the rest of kibanaServer service

* expose server urls with better API

* tweak status api response types

* fix http body param name

* second arg to axios post is the post data

* use standardized error handling behavior

* Revert "use standardized error handling behavior"

This reverts commit 7e9a7f8dc5.

* revert unnecessary changes
2019-09-17 18:36:19 -07:00
Spencer
327a75502c
[7.4] set IS_PIPELINE_JOB in intake jobs (#45850) (#45921)
* set IS_PIPELINE_JOB in intake jobs

* firefox smoke doesn't get an initial build

* firefox smoke doesn't get an initial build (oss edition)

* re-enable all jobs
2019-09-17 14:42:39 -07:00
Dmitry Lemeshko
5e1072e452
[test/functional/services] tsfy FailureDebugging, SavedQueryManagementComponent & Snapshots (#45682) (#45866)
* [test/functional/services] tsfy failure_debugging, snapshots, saved_query_management_component

* update renovate config

* fix path in snapshot service
2019-09-17 18:06:15 +02:00
spalger
14c9c72702 [kbn-es] make custom snapshot url a feature (#45750)
* [kbn-es] make custom snapshot url a feature

* download es snapshot first

(cherry picked from commit 510566d66b)
(cherry picked from commit f85c7402f6)
2019-09-15 08:14:09 -07:00
Lee Drengenberg
5e38b2dfb6
correct path with this missing slash (#45666) (#45709) 2019-09-13 18:55:37 -05:00
Spencer
ead2468742
[7.4] Call failure reporter directly from Jenkinsfile (#45551) (#45669)
* Run failed_tests/cli once per node

* force failures for testing

* Revert "force failures for testing"

This reverts commit 3ddd9240fe.
2019-09-13 16:03:06 -07:00
Dmitry Lemeshko
583b568a2e
Change data-test-subj selector logic to support value with spaces (#43682) (#45506)
* update test-subj-selector

* update locators

* adjust whitespaces removal

* fix another locator

* fix ML test locators
2019-09-12 20:25:42 +02:00
Spencer
77eb3d3c53 Revert "Revert "Revert "Revert "[ci] compress jobs for CI stab… (#45454)
* Revert "Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)"""

This reverts commit 148b8c0f90.

* sync changes with Jenkinsfile

(cherry picked from commit 9a109f2170)
2019-09-11 15:28:34 -07:00
Brian Seeders
14b69dc6ea Jenkins pipeline with parallel cigroups (#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""

This reverts commit 078ac2897f.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c465a5483b1a994cb510a182fef766e9cc.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203e26922a93483feb0ebb8bb3fdcc3daf8c.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4184)

# Conflicts:
#	src/dev/ci_setup/setup.sh
#	src/dev/failed_tests/cli.js
#	test/plugin_functional/test_suites/core_plugins/applications.js
#	test/scripts/jenkins_xpack_ci_group.sh
2019-09-11 12:23:57 -07:00
Dmitry Lemeshko
cb681edb4c
Functional tests: convert more test/services to TS (#45176) (#45386)
* convert more test/services to TS

* Update test/functional/services/combo_box.ts

Co-Authored-By: Tre' <wayne.seymour@elastic.co>

* Update test/functional/services/combo_box.ts

Co-Authored-By: Tre' <wayne.seymour@elastic.co>

* Update test/functional/services/combo_box.ts

Co-Authored-By: Tre' <wayne.seymour@elastic.co>

* fix lint error
2019-09-11 17:47:26 +02:00
spalger
ddf4f15537 disable another flaky suite (#45323) (#45330)
(cherry picked from commit 95bf9c832a)
2019-09-10 20:23:25 -07:00
spalger
d7dc73e434 disable jest suite that has no enabled tests (#44250)
(cherry picked from commit 61e1b83722)
2019-09-10 18:56:40 -07:00
spalger
76b3c38f32 disable flaky suite (#45105)
(cherry picked from commit a7f5062741)
2019-09-10 18:19:35 -07:00
spalger
b0702e13b4 skip flaky suite (#45089)
(cherry picked from commit 3a8d19d3a0)
2019-09-10 18:00:19 -07:00
spalger
023cc9c0d4 disable flaky test (#45315)
(cherry picked from commit d50892100b)
2019-09-10 17:31:28 -07:00
Dmitry Lemeshko
c2aea564b7
FTR: fix WebDriver Actions calls (#44605) (#45000)
* [services/web_element_wrapper] add actions API methods

* fixes

* [services/browser] fix dragAnDrop to work in both w3c/non-w3c modes

* small review changes

* dragAndDrop: add default values for offset
2019-09-06 17:39:49 +02:00
Dmitry Lemeshko
a5a834a3d9
[visualize/_tsvb_time_series] fix flaky test (#44505) (#44971) 2019-09-06 12:56:02 +02:00