## Summary
Kibana requires security to be enabled and a platinum or better license
to run in FIPS mode.
Since not all FTR configs assume these conditions will be enabled, we
cant run every test. So these failing tests will be skipped when these
overrides are enforced.
This does not mean that the functionality is not supported in FIPS mode.
## What is the point?
Running these tests in FIPS mode is not necessarily to check that the
functionality works as expected, it is to make sure Kibana does not
crash due to unsupported algorithm usage (`md4`, `md5`, etc).
When running in FIPS mode, Node will throw an `unsupported envelope
function` error (with FIPS enabled) if it encounters an unsupported
algorithm, so the more lines of code covered, the more assurance we can
have that features will work in FIPS mode.
## Nature of the changes
To skip a test, a `tag` is added: `this.tags('skipFIPS')`
`this.tags` is only available for `describe('description', function()
{...});`
There should not be any logical changes, just tests wrapped in an extra
block.
I tried to make the wording in the new `describe` block "flow" 😅 if you
prefer different wording in the new `describe` block - please add a
change!
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Nikita Indik <nikita.indik@elastic.co>
* [ftr] automatically determine config run order
* split lens config into two groups
* support ftr configs always running against CI
* Split detection_engine_api_integration rule exception list tests
* Add configs from previous commit
* [ftr] remove testMetadata and maintain a unique lifecycle instance per run
* Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run"
This reverts commit d2b4fdb824.
* Split alerting_api_integration/security_and_spaces tests
* Add groups to yaml
* Revert "Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run""
This reverts commit 56232eea68.
* stop ES more forcefully and fix timeout
* only cleanup lifecycle phases when the cleanup is totally complete
* only use kill when cleaning up an esTestInstance
* fix broken import
* fix runOptions.alwaysUseSource implementation
* fix config access
* fix x-pack/ccs config
* fix ml import file paths
* update kibana build id
* revert array.concat() change
* fix baseConfig usage
* fix pie chart data
* split up maps tests
* pull in all of group5 so that es archives are loaded correctly
* add to ftr configs.yml
* fix pie chart data without breaking legacy version
* fix more pie_chart stuff in new vis lib
* restore normal PR tasks
* bump kibana-buildkite-library
* remove ciGroup validation
* remove the script which is no longer called from checks.sh
* [CI] Auto-commit changed files from 'yarn kbn run build -i @kbn/pm'
* adapt flaky test runner scripts to handle ftrConfig paths
* fix types in alerting_api_integration
* improve flaky config parsing and use non-local var name for passing explicit configs to ftr_configs.sh
* Split xpack dashboard tests
* Add configs
* [flaky] remove key from ftr-config steps
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
* restore cypress builds
* remove ciGroups from FTR config files
* fixup some docs
* add temporary script to hunt for FTR config files
* use config.base.js naming for clarity
* use script to power ftr_configs.yml
* remove usage of removed x-pack/scripts/functional_tests
* fix test names in dashboard snapshots
* bump kibana-buildkite-library
* Try retrying only failed configs
* be a little quieter about trying to get testStats from configs with testRunners defined
* Remove test code
* bump kibana-buildkite-library
* update es_snapshot and on_merge jobs too
* track duration and exit code for each config and print it at the end of the script
* store results in order, rather than by key, in case there are duplicates in $config
* bash is hard
* fix env source and use +e rather than disabling e for whole file
* bash sucks
* print config summary in jest jobs too
* define results in jest_parallel.sh
* simplify config summary print, format times a little better
* fix reference to unbound time variable, use better variable name
* skip the newline between each result
* finish with the nitpicking
* sync changes with ftr_configs.sh
* refuse to execute config files which aren't listed in the .buildkite/ftr_configs.yml
* fix config.edge.js base config import paths
* fix some readmes
* resolve paths from ftr_configs manifest
* fix readConfigFile tests
* just allow __fixtures__ configs
* list a few more cypress config files
* install the main branch of kibana-buildkite-library
* split up lens group1
* move ml data_visualizer tests to their own config
* fix import paths
* fix more imports
* install specific commit of buildkite-pipeline-library
* sort configs in ftr_configs.yml
* bump kibana-buildkite-library
* remove temporary script
* fix env var for limiting config types
* Update docs/developer/contributing/development-functional-tests.asciidoc
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* produce a JUnit report for saved objects field count
* apply standard concurrency limits from flaky test runner
* support customizing FTR concurrency via the env
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* chore(NA): create new x-pack cigroups and rebalancing them all
* chore(NA): better cigroups balancing
* chore(NA): push rollup tests back into ciGroup1
* chore(NA): move some functional ml tests from cigroup3 into cigroup13
* chore(NA): move some more tests into ciGroup13
* chore(NA): use a single top level describe at x-pack/test/functional/apps/ml
* chore(NA): move settings into ciGroup13
* temporary test for es snapshots env
* Revert "temporary test for es snapshots env"
This reverts commit 789ebe7b9c.
* docs(NA): add missing documentation on the function tests describe split
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Alerts][License] Define minimum license required for each alert type (#84997)
* Define minimum license required for each alert type
* fixed typechecks
* fixed tests
* fixed tests
* fixed due to comments
* fixed due to comments
* removed file
* removed casting to LicenseType
* [Alerts][License] Add license checks to alerts HTTP APIs and execution (#85223)
* [Alerts][License] Add license checks to alerts HTTP APIs and execution
* fixed typechecks
* resolved conflicts
* resolved conflicts
* added router tests
* fixed typechecks
* added license check support for alert task running
* fixed typechecks
* added integration tests
* fixed due to comments
* fixed due to comments
* fixed tests
* fixed typechecks
* [Alerting UI][License] Disable alert types in UI when the license doesn't support it. (#85496)
* [Alerting UI][License] Disable alert types in UI when the license doesn't support it.
* fixed typechecks
* added licensing for alert list and details page
* fixed multy select menu
* fixed due to comments
* fixed due to comments
* fixed due to comments
* fixed typechecks
* fixed license error message
* fixed license error message
* fixed typechecks
* fixed license error message
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Define minimum license required for each action type (#58668)
* Add minimum required license
* Require at least gold license as a minimum license required on third party action types
* Use strings for license references
* Ensure license type is valid
* Fix some tests
* Add servicenow to gold
* Add tests
* Set license requirements on other built in action types
* Use jest.Mocked<ActionType> instead
* Change servicenow to platinum
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Make actions config mock and license state mock use factory pattern and jest mocks (#59370)
* Add license checks to action HTTP APIs (#59153)
* Initial work
* Handle errors in update action API
* Add unit tests for APIs
* Make action executor throw when action type isn't enabled
* Add test suite for basic license
* Fix ESLint errors
* Fix failing tests
* Attempt 1 to fix CI
* ESLint fixes
* Create sendResponse function on ActionTypeDisabledError
* Make disabled action types by config return 403
* Remove switch case
* Fix ESLint
* Add license checks within alerting / actions framework (#59699)
* Initial work
* Handle errors in update action API
* Add unit tests for APIs
* Verify action type before scheduling action task
* Make actions plugin.execute throw error if action type is disabled
* Bug fixes
* Make action executor throw when action type isn't enabled
* Add test suite for basic license
* Fix ESLint errors
* Stop action task from re-running when license check fails
* Fix failing tests
* Attempt 1 to fix CI
* ESLint fixes
* Create sendResponse function on ActionTypeDisabledError
* Make disabled action types by config return 403
* Remove switch case
* Fix ESLint
* Fix confusing assertion
* Add comment explaining double mock
* Log warning when alert action isn't scheduled
* Disable action types in UI when license doesn't support it (#59819)
* Initial work
* Handle errors in update action API
* Add unit tests for APIs
* Verify action type before scheduling action task
* Make actions plugin.execute throw error if action type is disabled
* Bug fixes
* Make action executor throw when action type isn't enabled
* Add test suite for basic license
* Fix ESLint errors
* Stop action task from re-running when license check fails
* Fix failing tests
* Attempt 1 to fix CI
* ESLint fixes
* Return enabledInConfig and enabledInLicense from actions get types API
* Disable cards that have invalid license in create connector flyout
* Create sendResponse function on ActionTypeDisabledError
* Make disabled action types by config return 403
* Remove switch case
* Fix ESLint
* Disable when creating alert action
* Return minimumLicenseRequired in /types API
* Disable row in connectors when action type is disabled
* Fix failing jest test
* Some refactoring
* Card in edit alert flyout
* Sort action types by name
* Add tooltips to create connector action type selector
* Add tooltips to alert flyout action type selector
* Add get more actions link in alert flyout
* Add callout when creating a connector
* Typos
* remove float right and use flexgroup
* replace pixels with eui variables
* turn on sass lint for triggers_actions_ui dir
* trying to add padding around cards
* Add callout in edit alert screen when some actions are disabled
* improve card selection for Add Connector flyout
* Fix cards for create connector
* Add tests
* ESLint issue
* Cleanup
* Cleanup pt2
* Fix type check errors
* moving to 3-columns cards for connector selection
* Change re-enable to enable terminology
* Revert "Change re-enable to enable terminology"
This reverts commit b497dfd6b6.
* Add re-enable comment
* Remove unecessary fragment
* Add type to actionTypeNodes
* Fix EuiLink to not have opacity of 0.7 when not hovered
* design cleanup in progress
* updating classNames
* using EuiIconTip
* Remove label on icon tip
* Fix failing jest test
Co-authored-by: Andrea Del Rio <delrio.andre@gmail.com>
* Add index to .index action type test
* PR feedback
* Add isErrorThatHandlesItsOwnResponse
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Andrea Del Rio <delrio.andre@gmail.com>
