Commit graph

238 commits

Author SHA1 Message Date
spalger
a81d8b55ab rename visual_baseline -> baseline_capture 2020-07-14 15:01:39 -07:00
Spencer
fa93a81ba6
[kbn/optimizer] implement "requiredBundles" property of KP plugins (#70911)
Co-authored-by: Josh Dover <me@joshdover.com>
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-09 18:43:17 -07:00
spalger
589a891bb3 jenkins_xpack_saved_objects_field_metrics.sh expects to be run from the KIBANA_DIR in CI 2020-07-09 16:00:09 -07:00
spalger
3153dff96b fix visual regression job (#70999) 2020-07-09 14:58:11 -07:00
Spencer
b6a06dfb61
Revert #64011 and subsequent fixes (#71137)
* Revert "temporarily disable firefox functional tests in PRs (#71116)"

This reverts commit 54bd07f81b.

* Revert "[savedObjects field count] run in baseline job (#70999)"

This reverts commit 53ee7a762d.

* Revert "[CI] Add pipeline task queue framework and merge workers into one (#64011)"

This reverts commit 465ed21194.

* Revert revert of change to jenkins_xpack_visual_regression.sh

Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-07-08 13:05:12 -07:00
Spencer
53ee7a762d
[savedObjects field count] run in baseline job (#70999)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-07-07 14:32:04 -07:00
Brian Seeders
465ed21194
[CI] Add pipeline task queue framework and merge workers into one (#64011) 2020-07-07 14:14:30 -04:00
Rudolf Meijering
2f905e7d86
CI Reporter for saved objects field count (#70580)
* CI Reporter for saved objects field count

* Metrics needs to be an array

* Fix type failures

* Link to field count issue

* Revert "Link to field count issue"

This reverts commit 8c0126b838.

* Break down field count per type

* Don't log total metric as metrics report already calculates this

* Add saved objects field count ci metrics test to codeowners

* Address review comments

* Add field count CI metrics for disabled plugins

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-07 16:53:47 +02:00
liza-mae
fd307a0635
Update network idle timeout (#70629)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-02 14:37:02 -06:00
MadameSheema
257c115f66
[SIEM] Reenabling Cypress tests (#70397)
* reenabling cypress

* skips Overview tests

* skips search bar test

* skips URL test
2020-07-02 07:49:01 +02:00
spalger
848c239bff disable page load metrics task in baseline job too 2020-06-29 08:34:44 -07:00
MadameSheema
1c9c0fc339
renames SIEM to Security Solution (#70070) 2020-06-26 19:58:51 +02:00
Tyler Smalley
bcc62095f0 [SECURITY] Disables Cypress tests
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2020-06-24 16:21:46 -07:00
Mikhail Shustov
c8c20e4ca8
Add functional test for Kibana embedded in iframe (#68544)
* convert kbn test config into TS

* add test  for Kibana embedded in iframe

* run embedded tests in functional suite

* ignore tls errors in functional tests by default

* switch test to https

* remove env vars mutation

* allow to pass ssl config to Kibana

* pass ssl config to axios

* adopt KbnClient interfaces

* adopt KibanaServer

* use KbnRequester in security service

* set sameSiteCookies:None in test

* acceptInsecureCerts in chrome

* remove leftovers

* fix type error

* remove unnecessary field

* address comments

* refactor plugin

* refactor test

* make acceptInsecureCerts configurable

* run firefox tests on ci

* up TS version

* fix firefox.sh script

* fix path

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-06-18 12:02:56 +02:00
Spencer
7e328417ab
[kbn/optimizer] share all plugin bundles (#68986)
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-06-15 22:15:31 -07:00
Spencer
45c81ce1ab
run page_load_metrics tests in visual regresssion jobs (#68570)
Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-06-09 10:41:41 -07:00
Dmitry Lemeshko
5064ae4a4c
[QA] Code coverage: unskip tests, collect tests results, exclude bundles from report (#64477)
* custom nyc configs, run all tests

* copy tests results back to original folder
2020-06-05 09:34:25 -06:00
Dmitry Lemeshko
6f57fa0b2d
Report page load asset size (#66224)
* performance_tests: draft version

* update cli runner and script

* ingest metrics

* save asset type for plugins

* Update src/dev/performance/ingest_metrics.ts

Co-authored-by: Spencer <email@spalger.com>

* follow review comments

* fix size calc, add FTR config, move src to kbn/test

* fix import, remove unused interface

* Update packages/kbn-test/src/page_load_metrics/capture_page_load_metrics.ts

Co-authored-by: Spencer <email@spalger.com>

* start chromium with no-sandbox

* add logging

* check page contains expected element, cut apps to 5

* fix locator & typo

* Update packages/kbn-test/src/page_load_metrics/navigation.ts

Co-authored-by: Spencer <email@spalger.com>

* Update navigation.ts

* Update navigation.ts

* bump puppeteer version

* fix typo

* update navigation script

* update config file

* update yarn.lock

* fix

* take screenshot on failure

* update screenshot title

* fix screenshot saving and error

* invalid locator

* Revert "invalid locator"

This reverts commit 3007539a69.

* run script in a loop 10 times

* Revert "run script in a loop 10 times"

This reverts commit 6cfa219140.

* path config value directly

* fix screenshots directory setup

* update maps locator, common for landing and new map

Co-authored-by: Spencer <email@spalger.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Mikhail Shustov <restrry@gmail.com>
2020-06-05 16:42:55 +02:00
Xavier Mouligneau
1216b0f7cd
[SECURITY] Rename siem plugin to security_solution (#67902)
* rename siem to security_solution

* rename siem to security solution inside of code

* rename translation keys

* fix snapshot

* replace siem for security solution in tutorial

* missing translation to be renamed

* fix types for api test integration

* updates runner file to match the new path

* change category for kibana settings

* miss renaming in advance settings

* fixes cypress tests

* fix api integration test

* fix new translation

* fix unit test

* update translation i18n

* update translation i18n II

Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com>
2020-06-04 05:35:13 -04:00
Frank Hassanabad
957915b7e5
[SIEM][Lists] Adds circular dependency checker for lists plugin
## Summary

* Added dependency checker for the public and common folders for lists
2020-05-28 16:45:29 -06:00
Brian Seeders
2c86a22582
[CI] Remove --detectOpenHandles to parallelize Jest tests again (#67413) 2020-05-27 12:44:41 -04:00
Tre
4853f24705
Ingest code coverage (#62336) 2020-05-13 15:09:42 -06:00
Josh Dover
fd4074f2cd
Remove slapshot contract tests (#66277) 2020-05-12 14:30:47 -06:00
Uladzislau Lasitsa
c5d4d9e68a
Migrate test plugins ⇒ NP (kbn_tp_run_pipeline) (#64780)
* Migrated kbn_tp_run_pipeline to the new Platform

* Updated config.

* Added tsconfig.

* Updated index.ts for tests

* updated tsconfig

* Updated typescript task

* fixed tests

* Fixed tests

* updated comment

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-05-07 10:12:46 +03:00
Mikhail Shustov
4a6ecc84a6
move core provier to NP. allows to run tests on every page (#64929)
* move core provier to NP. allows to run tests on every page

* remove no-base-path

* pass whole KbnTestServer config

* fix build path

* another id

* np

* build kibana tests plugins. it is used in xpack tests

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-05-07 08:26:11 +02:00
Gidi Meir Morris
94127d803a
[Alerting] migrates acceptance and functional test fixtures to KP (#64888)
This PR migrates the vast majority of Alerting legacy code to the Kibana Platform.

This includes:

1. Removed legacy Task Manager
2. Migrates Fixture plugins in Alerting, Triggers UI and Task Manager Perf

This does not includes:

1. The PagerDuty simulator due to a lack of support for custom responses in the platform. issue opened. https://github.com/elastic/kibana/issues/65045
2. The Webhooks simulator due to a lack of support for custom authorisation. Requires investigation.
2020-05-06 09:57:26 +01:00
Corey Robertson
b8bea5b8b2
[Canvas] move files from legacy/plugins to plugins (#65283)
* Moves the files

* Rename legacy/plugins/canvas paths

* Correct paths

* Fix shareable_runtime and storybook

* ESLint Fixes

* Fix jest test and paths
2020-05-05 20:52:56 -04:00
Dmitry Lemeshko
bac638a37e
Update jest config for coverage (#64648)
* set files to track for coverage collection

* increase timeout to 4h

* trying to add detectOpenHandles to avoid worker stuck

* update config

* make config paths more common

* update configs

* update jest oss config

* exclude 'tests' folder for coverage
2020-04-29 20:00:14 +03:00
Dmitry Lemeshko
c0bf910a70
Re-enable Firefox functional testing on CI (#64159)
* bring testing on Firefox back

* disable some tests

* skip more tests

* cut more suites for Firefox

* skip more tests for Firefox

* replace smoke tag with includeFirefox

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-04-24 23:59:14 +03:00
Ryland Herrick
40f82229f0
[SIEM] Server cutover to New Platform (#63430)
* Move server code into NP folder

* NP config is not yet used
* Relative imports are somewhat broken

* Move common folder into NP

* Move cypress folder into NP

* Move scripts folder into NP

* Move misc. config into NP folder

A few of these were moved into the cypress folder as they're
cypress-specific.

I tried to update all the relative paths but some are likely broken. I'm
not going to know until other stuff is fixed, though.

* Move value for siem index pattern into common/constants

The other default values live in there, this is no different.

* Update paths following file move

If this was referencing the full project, it now references both paths
(legacy for UI, and NP for server).

* Fix typescript errors related to module resolution

These are mostly updating imports to the common/ folder on the UI side
(since things changed relative to those files).

* Replace Legacy Config with NP Config

* Updates plugin to use NP config
  * defines new config previously coming from savedObjects config
* cleans up legacy types

 Conflicts:
	x-pack/plugins/siem/server/lib/detection_engine/routes/rules/export_rules_route.ts
	x-pack/plugins/siem/server/lib/detection_engine/routes/rules/import_rules_route.ts
	x-pack/plugins/siem/server/lib/detection_engine/rules/types.ts
	x-pack/plugins/siem/server/plugin.ts
	x-pack/plugins/siem/server/routes/index.ts
	x-pack/plugins/siem/server/types.ts

* Remove local SIEM tsconfig

This was originally added to address an issue with tsserver, but that
issue is no longer relevant. The presence of this file confuses
typescript into thinking that siem is a separate TS project.

* Update kibana.json to declare our dependencies

These are not necessarily correct in terms of what's required/optional,
but this is what's declared in our types.

* Remove legacy plugin instantiation

* Removes legacy instantiation of server plugin, which is now handled by
NP
* Loosens legacy config spec so we no longer have to duplicate config
types

* Update tests with NP config

These were written against the old Hapi config function; now, we just
have a POJO.

* Update es_archiver helpers' paths

I'm not quite sure if these are working yet, but they're no longer
throwing errors.

* Ignore restricted path on script

This was cribbed from infra, who has made a similar change.

* Ignore restricted path on temporary savedObject mappings import

This will be changed subsequently when we switch to the NP form of
savedObject type registration.

* Add symlink to lockfile

* Fix paths on circular deps script

* Add separate config for Rule and Timeline saved objects

We had previously used the savedObjects' config, but those are not
currently exposed to us on New Platform. For now, we're going to split
this into two sets of values for the SOs we deal with
importing/exporting within the SIEM app, with the same defaults as
savedObjects.

* Fixing relative paths within cypress

These are strings that wouldn't be caught by typescript.
2020-04-17 12:31:54 -05:00
Dmitry Lemeshko
50228df498
Code coverage: run more configs with functional tests (#63680)
* run endpoint tests with coverage

* add more configs

* add plugin_functional config
2020-04-16 20:40:55 +03:00
liza-mae
29c1aad2bb
Fix visual tests (#62660) 2020-04-06 16:48:37 -06:00
Gidi Meir Morris
e7a4ca261b
[Event Log] adds query support to the Event Log (#62015)
* added Start api on Event Log plugin

* added empty skeleton for Event Log FTs

* added functional test to public find events api

* added test for pagination

* fixed unit tests

* added support for date ranges

* removed unused code

* replaces valdiation typing

* Revert "replaces valdiation typing"

This reverts commit 711c098e9b.

* replaces match with term

* added sorting

* fixed saved objects nested query

* updated plugin FTs path

* Update x-pack/plugins/encrypted_saved_objects/README.md

Co-Authored-By: Aleh Zasypkin <aleh.zasypkin@gmail.com>

* Update x-pack/plugins/encrypted_saved_objects/README.md

Co-Authored-By: Aleh Zasypkin <aleh.zasypkin@gmail.com>

* remofed validation from tests

* fixed typos

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
2020-04-06 18:02:58 +01:00
Uladzislau Lasitsa
5d5c012f3b
Migrate test plugins ⇒ NP (kbn_tp_sample_panel_action) (#60749)
* Migrated sample_action to NP. Panel action tests returned to the test flow.

* fixed names

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-04-01 17:11:33 +03:00
Spencer
84cb048020
[ci/es] always pull snapshots, break early when they're not av… (#62040)
* [ci/es] always pull snapshots, break early when they're not available

* move to setup.sh after deps are installed

Co-authored-by: spalger <spalger@users.noreply.github.com>
2020-03-31 13:25:14 -07:00
Dmitry Lemeshko
3fe48fe8b3
Fix empty coverage folder, collect coverage when some tests fail (#61349)
* create coverage folder only if json file being saved

* skip some tests

* unskip test

* do not fail coverage run if some tests fail

* unskip fixed test
2020-03-26 13:50:58 +03:00
Gidi Meir Morris
8fd317c55a
[Alerting] Adds navigation by consumer and alert type to alerting (#58997)
Adds Navigation APIs to Alerting.

Parts to this PR:

Adds a client side (Public) plugin to Alerting, including two APIs: registerNavigation & registerDefaultNavigation. These allow a plugin to register navigation handlers for any alerts which it is the consumer of- one for specific AlertTypes and one for a default handler for all AlertTypes created by the plugin.
The Alert Details page now uses these navigation handlers for the View In App button. If there's an AlertType specific handler it uses that, otherwise it uses a default one and if the consumer has not registered a handler - it remains disabled.
A generic Alerting Example plugin that demonstrates usage of these APIs including two AlertTypes - one that always fires, and another that checks how many people are in Outer Space and allows you to trigger based on that. 😉 To enable the plugin run yarn start --ssl --run-examples
2020-03-19 09:49:05 +00:00
Spencer
5408f45b52
expand max-old-space-size for xpack jest tests (#59455)
* expand max-old-space-size for xpack jest tests

* turns out we are already at 4GB

* limit to 6GB for now
2020-03-05 15:59:33 -07:00
Spencer
4bc9e8b4a8
Fix visual baseline job (#59348)
* Establish Percy baselines

* move Jenkinsfile changed back to `.ci` directory

* rename xpack workers

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-03-05 13:01:42 -07:00
Dmitry Lemeshko
464f9071e9
Code coverage: skip failing tests for coverage run (#59121)
* filter out failing tests

* more skipped tests

* fix tests

* move json files to save space
2020-03-03 20:05:13 +03:00
MadameSheema
02f89f04cc
[SIEM] Cypress preparation for Jenkins (#59013)
* updates events viewer test

* updates login tasks

* updates ml conditional links

* updates url state

* updates timeline screen

* updates timeline tasks

* updates test files

* adds jenkins needed files

* ignoring isAttached lines due to a known error in Cypress (https://github.com/cypress-io/cypress/issues/4408)

* updates loop script

* updates readme with new cypress command explanation

* removes skip

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-03-03 17:51:35 +01:00
Brian Seeders
c90cfe208d
[CI] Pipeline refactoring (#56447) 2020-03-02 15:04:27 -05:00
Spencer
841e64e0c1
run jest with --detectOpenHandles on CI to figure out what i… (#58543)
* run jest with `--detectOpenHandles` on CI to figure out what is happening with pauses

* focus tests on jest integration

* force kill child processes in config reload test

* skip flaky suite

* increase timeout for looking for installed packages

* run all tests again
2020-02-27 14:19:24 -07:00
Dmitry Lemeshko
d6aff77485
remove NODE_ENV=test (#58037)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-20 13:12:04 +01:00
Jonathan Budzenski
76d475a64c
Refactor test entry by runner (#44679)
Over the last few years we've increased the number of test runners.
Entry points by test type have become too lage of a category for unique
names, so this moves top level test scripts under yarn and replaces test
types with the runner name.

e.g. `yarn test:browser` -> `yarn test:karma`

Closes #41133

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-18 16:21:36 -06:00
Spencer
4e8ab56497
build immutable bundles for new platform plugins (#53976)
* build immutable bundles for new platform plugins

* only inspect workers if configured to do so

* [navigation] use an index.scss file

* add yarn.lock symlink

* set pluginScanDirs in test so fixtures stay consistent

* cleanup helpers a little

* fix type error

* support KBN_OPTIMIZER_MAX_WORKERS for limiting workers via env

* test support for KBN_OPTIMIZER_MAX_WORKERS

* expand the available memory for workers when only running one or two

* add docs about KBN_OPTIMIZER_MAX_WORKERS environment variable

* fix README link

* update kbn/pm dist

* implement bundle caching/reuse

* update kbn/pm dist

* don't check for cache if --no-cache is passed

* update renovate config

* standardize on index.scss, move console styles over

* add support for --no-cache to cli

* include worker config vars in optimizer version

* ignore concatenated modules

* update integration test

* add safari to browserslist to avoid user-agent warnings in dev

* update docs, clean up optimizer message/misc naming

* always handle initialized messages, don't ignore states that are attached to specific events

* reword caching docs, add environment var to disable caching

* tweak logging and don't use optimizer.useBundleCache as that's disabled in dev

* handle change notifications

* batch changes for 1 second

* rename CompilerState type to CompilerMsg

* getChanges() no longer needs to assign changes to dirs

* remove unused deps

* split up run_worker.ts and share cacheKey generation logic

* add a couple docs

* update tests and remove unused imports

* specify files when creating bundle cache key

* remove one more unused import

* match existing dev cli output more closely

* update kbn/pm dist

* set KBN_NP_PLUGINS_BUILT to avoid warning in CI

* avoid extending global window type

* add note to keep pluginScanDirs in sync

* pass browserslistEnv in workerConfig so it is used for cache key

* load commons.bundle.js in parallel too

* emit initialized+success states if all bundles are cached

* load bootstraps as quickly as possible

* skip flaky suite

* bump

* update jest snapshots

* remove hashing from cache key generation

* remove unnecessary non-null assertion

* improve docs and break up Optimizer#run()

* remove unused import

* refactor kbn/optimizer to break up observable logic, implement more helpful cache invalidation logic with logging

* fix tests

* add initializing phase

* avoid rxjs observable constructor

* remove unnecessary rxjs helper, add tests for bundle cache

* update consumers of optimizer

* update readme with new call style

* replace "new platform" with "kibana platform"

* fix a couple more renames

* add support for several plain-text file formats

* fix naming of OptimizerMsg => OptimizerUpdate, use "store" naming too

* one more OptimizerMsg update

* ensure bundles are not cached when cache config is false

* test for initializing states and bundle cache events

* remove unnecessary timeout change

* Remove unnecessary helpers

* Add tests for BundleCache class

* Add tests for Bundle class

* test summarizeEvent$

* missing paths are no longer listed in mtimes map

* add tests for optimizer/cache_keys

* Add some extra docs

* Remove labeled loop

* add integration test for kbn-optimizer watcher components

* querystring-browser removed

* tweak logging a smidge, improve info and final message

* remove unused imports

* remove duplication of getModuleCount() method

* move type annotation that validates things

* clear up the build completion message

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-12 19:42:42 -07:00
Dmitry Lemeshko
25fb555a53
Code coverage: run functional tests from repo copy (#55195)
* run functional tests from cloned folders

* do not copy kibana.yml

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-01-29 14:36:47 +01:00
Brian Seeders
99f224097c
Remove matrix build support (#54202) 2020-01-27 17:10:34 -05:00
Dmitry Lemeshko
26ce6104a9
Code coverage setup on CI (#49003)
* running code coverage in CI

* apply review feedback

* add custom function to upload merged coverage reports

* fix artifacts upload without coverage

* add file extension to fix validation

* Check code_coverage is set

* run oss tests via grunt task

* review fixes

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-01-08 21:08:48 +01:00
Stacey Gammon
7e67d1f86c
Embeddable examples on the platform and included with --run-examples flag (#52111)
* Add a new platform embeddable example plugin

* Remove extra hello world test impl.

* cleanup

* code review updates

* Change example to highlight and have parent filter out children

* Fix deep comparison of embeddable prop

* adjust help text
2019-12-16 15:03:46 -05:00