Changes search service/search session infrastructure to improve performance, stability, and resiliency by ensuring that search sessions don’t add additional load on a cluster when the feature is not used
* [ftr] automatically determine config run order
* split lens config into two groups
* support ftr configs always running against CI
* Split detection_engine_api_integration rule exception list tests
* Add configs from previous commit
* [ftr] remove testMetadata and maintain a unique lifecycle instance per run
* Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run"
This reverts commit d2b4fdb824.
* Split alerting_api_integration/security_and_spaces tests
* Add groups to yaml
* Revert "Revert "[ftr] remove testMetadata and maintain a unique lifecycle instance per run""
This reverts commit 56232eea68.
* stop ES more forcefully and fix timeout
* only cleanup lifecycle phases when the cleanup is totally complete
* only use kill when cleaning up an esTestInstance
* fix broken import
* fix runOptions.alwaysUseSource implementation
* fix config access
* fix x-pack/ccs config
* fix ml import file paths
* update kibana build id
* revert array.concat() change
* fix baseConfig usage
* fix pie chart data
* split up maps tests
* pull in all of group5 so that es archives are loaded correctly
* add to ftr configs.yml
* fix pie chart data without breaking legacy version
* fix more pie_chart stuff in new vis lib
* restore normal PR tasks
* bump kibana-buildkite-library
* remove ciGroup validation
* remove the script which is no longer called from checks.sh
* [CI] Auto-commit changed files from 'yarn kbn run build -i @kbn/pm'
* adapt flaky test runner scripts to handle ftrConfig paths
* fix types in alerting_api_integration
* improve flaky config parsing and use non-local var name for passing explicit configs to ftr_configs.sh
* Split xpack dashboard tests
* Add configs
* [flaky] remove key from ftr-config steps
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
* restore cypress builds
* remove ciGroups from FTR config files
* fixup some docs
* add temporary script to hunt for FTR config files
* use config.base.js naming for clarity
* use script to power ftr_configs.yml
* remove usage of removed x-pack/scripts/functional_tests
* fix test names in dashboard snapshots
* bump kibana-buildkite-library
* Try retrying only failed configs
* be a little quieter about trying to get testStats from configs with testRunners defined
* Remove test code
* bump kibana-buildkite-library
* update es_snapshot and on_merge jobs too
* track duration and exit code for each config and print it at the end of the script
* store results in order, rather than by key, in case there are duplicates in $config
* bash is hard
* fix env source and use +e rather than disabling e for whole file
* bash sucks
* print config summary in jest jobs too
* define results in jest_parallel.sh
* simplify config summary print, format times a little better
* fix reference to unbound time variable, use better variable name
* skip the newline between each result
* finish with the nitpicking
* sync changes with ftr_configs.sh
* refuse to execute config files which aren't listed in the .buildkite/ftr_configs.yml
* fix config.edge.js base config import paths
* fix some readmes
* resolve paths from ftr_configs manifest
* fix readConfigFile tests
* just allow __fixtures__ configs
* list a few more cypress config files
* install the main branch of kibana-buildkite-library
* split up lens group1
* move ml data_visualizer tests to their own config
* fix import paths
* fix more imports
* install specific commit of buildkite-pipeline-library
* sort configs in ftr_configs.yml
* bump kibana-buildkite-library
* remove temporary script
* fix env var for limiting config types
* Update docs/developer/contributing/development-functional-tests.asciidoc
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* produce a JUnit report for saved objects field count
* apply standard concurrency limits from flaky test runner
* support customizing FTR concurrency via the env
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Christiane (Tina) Heiligers <christiane.heiligers@elastic.co>
* Fix link to Cloud deployment URL in upgrade step. (#109528)
* [Upgrade Assistant] Refactor CITs
* Rename UA steps to fix_issues_step and fix_logs_step. (#109526)
* Rename tests accordingly.
* [Upgrade Assistant] Cleanup scss (#109524)
* [Upgrade Assistant] Update readme (#109502)
* Add "Back up data" step to UA (#109543)
* Add backup step with static content and link to Snapshot and Restore.
* Add snapshot_restore locator.
* Remove unnecessary describe block from Upgrade Step tests.
* Remove unused render_app.tsx.
* Change copy references of 'deprecation issues' to 'deprecation warnings'. (#109963)
* [Upgrade Assistant] Address design feedback for ES deprecations page (#109726)
* [Upgrade Assistant] Add checkpoint feature to Overview page (#109449)
* Add on-Cloud state to Upgrade Assistant 'Back up data' step (#109956)
* [Upgrade Assistant] Refactor external links to use locators (#110435)
* [Upgrade Assistant] Use AppContext for services instead of KibanaContext (#109801)
* Remove kibana context dependency in favour of app context
* Add missing type to ContextValue
* Fix mock type
* Refactor app mount flow and types
* Refactor to use useServices hook
* Fix linter issues
* Keep mount_management_section and initialize breadcrumbs and api there
* Remove useServices and usePlugins in favour of just useAppContext
* Remove unnecessary mocks
* [Upgrade Assistant] Enable functional and a11y tests (#109909)
* [Upgrade Assistant] Remove version from UA nav title (#110739)
* [Upgrade Assistant] New Kibana deprecations page (#110101)
* Use injected lib.handleEsError instead of importing it in Upgrade Assistant API route handlers. (#111067)
* Add tests for UA back up data step on Cloud (#111066)
* Update UA to consume snapshotsUrl as provided by the Cloud plugin. (#111239)
* Skip flaky UA Backup step polling test.
* [Upgrade Assistant] Refactor kibana deprecation service mocks (#111168)
* [Upgrade Assistant] Remove unnecessary EuiScreenReaderOnly from stat panels (#111518)
* Remove EuiScreenReaderOnly implementations
* Remove unused translations
* Remove extra string after merge conflict
* Use consistent 'issues' and 'critical' vs. 'warning' terminology in UA. (#111221)
* Refactor UA Overview to support step-completion (#111243)
* Refactor UA Overview to store step-completion state at the root and delegate step-completion logic to each step component.
* Add completion status to logs and issues steps
* [Upgrade Assistant] External links with checkpoint time-range applied (#111252)
* Bound query around last checkpoint date
* Fix tests
* Also test discover url contains search params
* Small refactor
* Keep state about lastCheckpoint in parent component
* Remove space
* Address CR changes
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Upgrade Assistant] Tests for updating step state accordingly if API poll receives count followed by error (#111701)
* Add test for logs count polling
* Test when count api fails
* [Upgrade Assistant] Add a11y tests for es deprecation flyouts (#110843)
* [Upgrade Assistant] Set fix_logs step as incomplete if log collection is not enabled (#111827)
* set step as incomplete if toggle is disabled
* Fix test names
* Remove unnecessary mocks
* [Upgrade Assistant] Update copy to use "issues" instead of "warnings" (#111817)
* Create common deprecation issues panel component in UA (#111231)
* Refine success state behavior and add tests.
* Refactor components into a components directory.
* Refactor SCSS to colocate styles with their components.
* Refactor tests to reduce boilerplate and clarify conditions under test.
* [Upgrade Assistant] Fix Kibana deprecations warning message
* [Upgrade Assistant] Add support for API keys when reindexing (#111451)
* [Upgrade Assistant] Update readme (#112154)
* [Upgrade Assistant] Make infra plugin optional (#111960)
* Make infra plugin optional
* Fix CR requests
* [Upgrade Assistant] Improve flyout information architecture (#111713)
* Make sure longstrings inside flyout body are text-wrap
* Show resolved badge for reindex flyout and row
* Finish off rest of ES deprecation flyouts
* Refactor deprecation badge into its own component
* Add tests for kibana deprecations
* Add tests for es deprecations
* Also check that we have status=error before rendering error callout
* Check for non-complete states instead of just error
* Small refactor
* Default deprecation is not resolvable
* Add a bit more spacing between title and badge
* Address CR changes
* Use EuiSpacer instead of flexitems
* [Upgrade Assistant] Update readme (#112195)
* [Upgrade Assistant] Add integration tests for Overview page (#111370)
* Add a11y tests for when overview page has toggle enabled
* Add functional and accessibility tests for overview page
* Load test files
* Fix linter error
* Navigate before asserting
* Steps have now completion state
* Remove duped word
* Run setup only once, not per test
* Address CR changes
* No need to renavigate to the page
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Upgrade Assistant] Add note about compatibility headers (#110469)
* Improve error states for Upgrade Assistant deprecation issues (#112457)
* Simplify error state for deprecation issues panels. Remove <EsStatsError />.
* Rename components from stats -> panel.
* Create common error-reporting component for use in both Kibana and ES deprecations pages.
* Align order of loading, error, and success states between these pages.
* Change references to 'deprecations' -> 'deprecation issues'.
* Fix tests for panels.
* Add API integration test for handling auth error.
* Fix TS errors.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Change count poll time to 15s (#112669)
* [Upgrade Assistant] Add permissions check to logs step (#112420)
* [Upgrade Assistant] Refactor telemetry (#112177)
* [Upgrade Assistant] Check for ML upgrade mode before enabling flyout actions (#112555)
* Add missing error handlers for deprecation logging route (#113109)
* [Upgrade Assistant] Batch reindex docs (#112960)
* [UA] Added batch reindexing docs link to the ES deprecations page. Added a link from "batch reindexing" docs page to "start or resume reindex" docs page and from there to ES reindexing docs page. Also renamed "reindexing operation" to "reindexing task" for consistency.
* [Upgrade Assistant] Added docs build files
* Update x-pack/plugins/upgrade_assistant/public/application/components/es_deprecations/es_deprecations.tsx
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
* Update x-pack/plugins/upgrade_assistant/public/application/components/es_deprecations/es_deprecations.tsx
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
* [Upgrade Assistant] Added review suggestions and fixed eslint issues
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Upgrade Assistant] Improve error messages for GET /api/upgrade_assistant/reindex/<index> (#112961)
* Add support for single manual steps to Upgrade Assistant. (#113344)
* Revert "[Upgrade Assistant] Refactor telemetry (#112177)" (#113665)
This reverts commit 991d24bad2.
* [Upgrade Assistant] Use skipFetchFields when creating the indexPattern in order to avoid errors if index doesn't exist (#113821)
* Use skipFetchFields when creating the indexPatter in order to avoid errors when index doesnt exist
* Address CR feedback
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Upgrade Assistant] Hide system indices from es deprecations list (#113627)
* Refactor reindex routes into separate single and batch reindex files. Apply version precheck to batch routes. (#113822)
* [Upgrade Assistant] Remove ML/Watcher logic (#113224)
* Add show upgrade flag to url (#114243)
* [Upgrade Assistant] Delete deprecation log cache (#114113)
* [Upgrade Assistant] Add upgrade system indices section (#110593)
* [Upgrade Assistant] Reindexing progress (#114275)
* [Upgrade Assistant] Added reindexing progress in % to the reindex flyout and es deprecations table
* [Upgrade Assistant] Renamed first argument in `getReindexProgressLabel` to `reindexTaskPercComplete` for consistency
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Upgrade Assistant] Remove Fix manually heading when there are no manual steps
* Add rolling upgrade interstitials to UA (#112907)
* Refactor FixLogsStep to be explicit in which props are passed to DeprecationLoggingToggle.
* Centralize error-handling logic in the api service, instead of handling it within each individual API request. Covers:
- Cloud backup status
- ES deprecations
- Deprecation logging
- Remove index settings
- ML
- Reindexing
Also:
- Handle 426 error state and surface in UI.
- Move ResponseError type into common/types.
* Add note about intended use case of status API route.
* Add endpoint dedicated to surfacing the cluster upgrade state, and a client-side poll.
* Merge App and AppWithRouter components.
* [Upgrade Assistant] Added "accept changes" header to the warnings list in the reindex flyout (#114798)
* Refactor kibana deprecation tests (#114763)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fix linter issues
* Remove unused translation
* Prefer master changes over 7.x for ml docs
* Prefer master changes over 7.x
* Skip tests
* Move everything to a single describe
* Fix types
* Add missing prop to mock
* [Upgrade Assistant] Removed "closed index" warning from reindex flyout (#114861)
* [Upgrade Assistant] Removed "closed index" warning that reindexing might take longer than usual, which is not the case
* [Upgrade Assistant] Also deleted i18n strings that are not needed anymore
* Add LevelIconTips to be more explicit about the difference between critical and warning issues. (#115121)
* Extract common DeprecationFlyoutLearnMoreLink component and change wording to 'Learn more'. (#115117)
* [Upgrade Assistant] Reindexing cancellation (#114636)
* [Upgrade Assistant] Updated the reindexing cancellation to look less like an error
* [Upgrade Assistant] Fixed an i18n issue and updated a jest snapshot
* [Upgrade Assistant] Updated cancelled reindexing state with a unified label and cross icon
* [Upgrade Assistant] Fixed snapshot test
* [Upgrade Assistant] Updated spacing to the reindex cancel button
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fix test errors (#115183)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Upgrade Assistant] Overview page UI clean up (#115258)
- Scaling down deprecation issue panel title size to flow with typographic hierarchy.
- Removing panel around deprecation logging switch to reduce visual elements.
- Using success instead of green color for migration complete message.
* Revert "Revert "[Upgrade Assistant] Refactor telemetry (#112177)" (#113665)" (#114804)
This reverts commit c385d49887.
* Add migration to remove obsolete attributes from telemetry saved object.
* Refactor UA telemetry constants by extracting it from common/types.
* [Upgrade Assistant] Rename upgrade_status to migration_status (#114755)
* [Upgrade Assistant] Swapped reindexing flyouts order (#115046)
* [Upgrade Assistant] Changed reindexing steps order, replaced a warning callout with a text element
* [Upgrade Assistant] Fixed reindex flyout test and changed warning callout from danger color to warning color
* [Upgrade Assistant] Fixed the correct status to show warnings
* [Upgrade Assistant] Fixed i18n strings
* [Upgrade Assistant] Moved reindex with warnings logic into a function
* [Upgrade Assistant] Updated reindex flyout copy
* [Upgrade Assistant] Also added a trailing period to the reindex step 3
* [Upgrade Assistant] Fixed i18n strings and step 3 wording
* [Upgrade Assistant] Added docs changes
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [Upgrade Assistant] Hide features that don't need to be migrated from flyout (#115535)
* Filter out system indices that dont require migration on server side
* Rename to attrs to migration
* Update flyout snapshot.
* Refine Upgrade Assistant copy. (#115472)
* Remove unused file
* Fix kibanaVersion dep
* Updated config.ts to fix UA test
UA functional API integration test to check cloud backup status creates a snapshot repo, which fails to be created with my changes to config.ts `'path.repo=/tmp/repo,/tmp/repo_1,/tmp/repo_2,'`. Adding `/tmp/cloud-snapshots/'` to the config fixes the test.
* Address CR changes
* Add missing error handler for system indices migration (#116088)
* Fix broken tests
* Fix test
* Skip tests
* Fix linter errors and import
* [Upgrade Assistant] Fix typo in retrieval of cluster settings (#116335)
* Fix typos
* Fix typo also in server tests
* Make sure log collection remains enabled throughout the test
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Fix type errors
* Fix integration test types
* Fix accessibility test type errors
* Fix linter errors in shared_imports
* Fix functional test types
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: Alison Goryachev <alison.goryachev@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Yulia Čech <6585477+yuliacech@users.noreply.github.com>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Dmitry Borodyansky <dborodyansky@gmail.com>
* Initial commit
* Properly handle signal history
* Fix#95258 - cardinality sort bug
* Init threshold rule
* Create working threshold rule
* Fix threshold signal generation
* Fix tests
* Update mappings
* ALERT_TYPE_ID => RULE_TYPE_ID
* Add tests
* Fix types
* Adds RAC rule type migration
* Fix threshold tests (remove outputIndex)
* Add threshold rule type to ruleTypeMappings
* Add kbn-securitysolution-rules package for sharing with alerting framework
* Fix type errors
* Fix find_rules tests
* First round of test fixes
* Fix issues from merge conflicts
* Use ruleDataClient getReader() for reading
* Fixes to 'generating_signals' tests
* Remove more refs to legacy schema
* Linting
* Quick type fix
* Bug fixes
* Add saved query rule type
* Linting
* Fix types
* Signal generation tests
* Test updates
* Update some more refs
* build_alert tests
* Cleanup
* Ref updates
* Revert "Ref updates"
This reverts commit 4d1473d6b0.
* Update status field
* Test fixes
* Another test
* Got a little too aggressive with search/replace
* let's see where we're at
* Fix
* Test fixes
* cleanup
* Fix cases API integration test config, flaky DE tests
* Move flattenWithPrefix to package / skip signal migration tests
* Fix unit tests
* Use new schema for bulk rule creation
* event: { kind } => event.kind
* Fix signal migration API tests
* Fix ml integration test
* Fix threat match integration tests
* Fix ML rule type tests and add correct producer to all rule types
* Update threat match API integration test
* Remove dupe properties
* Type fix
* Fix ML producer in functional test
* Fix generating_signals tests
* Remove usage of RuleDataClient-based execution log client
* Don't check output index version if rule registry enabled
* Fix bulk duplicate rule
* Fix duplicate rule test
* Fix readPrivileges and timestamp check logic
* Fixes for eql and exceptions tests... disable open_close_signals
* Type fixes / keyword test fixes
* Additional test fixes
* Unit test fixes + signal -> kibana.alert
* Test fixes for exceptions
* Fix read_resolve_rules test
* Various test fixes with marshallmain
* Sort search results
* Fix create_rules tests
* Disable writer cache for integration tests
* Disable writer cache for cases integration tests
* Fix types in rule_data_plugin_service
* Fix ordering in exceptions tests
* Remove rule_registry.enabled flag
* Fix signals migration tests
* Don't check signals index before creation
* Fix cypress config
* Fix type error
* create_migrations tests
* Skip flaky test
* Helpful comment
* Fixes from merge conflicts
* Pretend that signals index exists
* Fix type errors
* Skip flaky tests
* Fix threat matching test
* Clean up
* Reverting default ruleRegistry experimental flag (breaks unit tests)
* Reenable rule registry experimental feature by default
* Execute DE rule migration in 8.0
Co-authored-by: Marshall Main <marshall.main@elastic.co>
* [Snapshot % Restore] Added server side pagination and sorting to get snapshots route, refactored snapshots table to use EuiBasicTable with controlled pagination instead of EuiInMemoryTable
* [Snapshot & Restore] Added server side sorting by shards and failed shards counts
* [Snapshot & Restore] Fixed i18n errors
* [Snapshot & Restore] Added server side sorting by repository
* [Snapshot & Restore] Implemented server side search request for snapshot, repository and policy name
* [Snapshot & Restore] Fixed eslint errors
* [Snapshot & Restore] Removed uncommented code
* [Snapshot & Restore] Fixed pagination/search bug
* [Snapshot & Restore] Fixed pagination/search bug
* [Snapshot & Restore] Fixed text truncate bug
* [Snapshot & Restore] Fixed non existent repository search error
* Update x-pack/plugins/snapshot_restore/public/application/sections/home/snapshot_list/components/snapshot_search_bar.tsx
Co-authored-by: CJ Cenizal <cj@cenizal.com>
* Update x-pack/plugins/snapshot_restore/public/application/sections/home/snapshot_list/components/snapshot_empty_prompt.tsx
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
* [Snapshot & Restore] Fixed missing i18n and no snapshots callout
* [Snapshot & Restore] Moved "getSnapshotSearchWildcard" to a separate file and added unit tests
* [Snapshot & Restore] Added api integration tests for "get snapshots" endpoint (pagination, sorting, search)
* [Snapshot & Restore] Renamed SnapshotSearchOptions/SnapshotTableOptions to -Params and added the link to the specs issue
* [Snapshot & Restore] Fixed search wildcard to also match string in the middle of the value, not only starting with the string. Also updated the tests following the code review suggestions to make them more readable.
* [Snapshot & Restore] Added incremental search back to snapshots list and a debounce of 500ms
* [Snapshot & Restore] Updated snapshot search debounce value and extracted it into a constant
* [Snapshot & Restore] Renamed debounceValue to cachedListParams and added a comment why debounce is used
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: CJ Cenizal <cj@cenizal.com>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
* cancel the previous session
* split to 3 tasks
* fixes
* cancellation
* updated tests
* split out and improve jest tests
* cleanup previous session properly
* don't fail delete and cancel if item was already cleaned up
* test
* test
* ignore resource_not_found_exception when deleting an already cleared \ expired async search
* jest
* update jest
* api int
* fix jest
* testssss
* Code review @dosant
* types
* remove any
* Fix merge
* type
* test
* jest
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [data.search] Add search session methods to search service contract
* Fix types
* Fix tests and switch to cancel
* Update docs
* Fix types/tests
* Fix tests
* Update status of SO before cancelling search requests
* Add API integration test
* Fix types
* Update expiration route to use config defaultExpiration
* Fix test
* Update docs
* New logic for extend
* Remove declare module
* Search Sessions: Unskip Flaky Functional Test
* Review feedback
* fix ts
* Save all search sessions and then manage them based on their persisted state
* Get default search session expiration from config
* randomize sleep time
* fix test
* Remove test that is no longer valid
* fix test
* Make sure we poll, and dont persist, searches not in the context of a session
* Added keepalive unit tests
* fix ts
* code review @lukasolson
* ts
* More tests, rename onScreenTimeout to completedTimeout
* lint
* lint
* Delete async seaches
* Support saved object pagination
Fix get search status tests
* better PersistedSearchSessionSavedObjectAttributes ts
* test titles
* Fix undefined bug
* Remove runAt from monitoring task
Increase testing trackingInterval (caused bug)
* support workload histograms that take into account overdue tasks
* Update touched when changing session status to complete \ error
* removed test
* Updated management test data
* Rename configs
* delete tap first
add comments
* Use DataRequestHandlerContext in maps
* ts
* Fixed ts
Co-authored-by: Lukas Olson <olson.lukas@gmail.com>
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Anton Dosov <anton.dosov@elastic.co>
Co-authored-by: Gidi Meir Morris <github@gidi.io>
