**Epic:** https://github.com/elastic/kibana/issues/118324
**Tickets:** https://github.com/elastic/kibana/issues/119603, https://github.com/elastic/kibana/issues/119597, https://github.com/elastic/kibana/issues/91265, https://github.com/elastic/kibana/issues/118511
## Summary
The legacy rule execution logging implementation is replaced by a new one that introduces a new model for execution-related data, a new saved object and a new, cleaner interface and implementation.
- [x] The legacy data model is deleted (`IRuleStatusResponseAttributes`, `IRuleStatusSOAttributes`)
- [x] The legacy `siem-detection-engine-rule-status` saved object type is deleted and marked as deleted in `src/core`
- [x] A new data model is introduced (`x-pack/plugins/security_solution/common/detection_engine/schemas/common/rule_monitoring.ts`). This data model doesn't contain a mixture of successful and failed statuses, which should simplify client-side code (e.g. the code of Rule Management and Monitoring tables, as well as Rule Details page).
- [x] A new `siem-detection-engine-rule-execution-info` saved object is introduced (`x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log/rule_execution_info/saved_object.ts`).
- [x] This SO has 1:1 association with the rule SO, so every rule can have 0 or 1 execution info associated with it. This SO is used in order to 1) update the last execution status and metrics and 2) fetch execution data for N rules more efficiently comparing to the legacy SO.
- [x] The logic of creating or updating this SOs is based on the "upsert" approach (planned in https://github.com/elastic/kibana/issues/118511). It does not fetch the SO by rule id before updating it anymore.
- [x] Rule execution logging logic is rewritten (see `x-pack/plugins/security_solution/server/lib/detection_engine/rule_execution_log`). The previous rule execution log client is split into two objects: `IRuleExecutionLogClient` for using it from route handlers, and `IRuleExecutionLogger` for writing logs from rule executors.
- [x] `IRuleExecutionLogger` instance is scoped to the currently executing rule and space id. There's no need to pass rule id, name, type etc to `.logStatusChange()` every time.
- [x] Rule executors and related functions are updated.
- [x] API routes are updated, including the rule preview route which uses a special "spy" implementation of `IRuleExecutionLogger`. A rule returned from an API endpoint now has optional `execution_summary` field of type `RuleExecutionSummary`.
- [x] UI is updated to use the new data model of `RuleExecutionSummary`:
- [x] Rule Management and Monitoring tables
- [x] Rule Details page
- [x] A new API route is introduced for fetching rule execution events: `/internal/detection_engine/rules/{ruleId}/execution/events`. It is used for rendering the Failure History tab (last 5 failures) and is intended to be used in the coming UI of Rule Execution Log on the Details page.
- [x] Rule Details page and Failure History tab are updated to use the new data models and API routes.
- [x] I used `react-query` for fetching execution events
- [x] See `x-pack/plugins/security_solution/public/detections/containers/detection_engine/rules/use_rule_execution_events.tsx`
- [x] The lib is updated to the latest version
- [x] Tests and fixed and updated according to all the changes
- [x] Components related to rule execution statuses are all moved to `x-pack/plugins/security_solution/public/detections/components/rules/rule_execution_status`.
- [x] I left a lot of `// TODO: https://github.com/elastic/kibana/pull/121644` comments in the code which I'm planning to address and remove in a follow-up PR. Lots of clean up work is needed, but I'd like to unblock the work on Rule Execution Log UI.
## In the next episodes
- Address and remove `// TODO: https://github.com/elastic/kibana/pull/121644` comments in the code
- Make sure that SO id generation for `siem-detection-engine-rule-execution-info` is safe and future-proof. Sync with the Core team. If there are risks, we will need to choose between risks and performance (reading the SO before updating it). It would be easy to submit a fix if needed.
- Add APM integration. Use `withSecuritySpan` in methods of `rule_execution_log` citizens.
- Add comments to the code and README.
- Add test coverage.
- Etc...
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### For maintainers
- [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
* [maps] fetch geometry from fields API
* tslint, eslint
* fix elasticsearch_geo_utils unit test
* more clean up of unit test
* i18n
* clean up
* eslint
* update functional test expects
* eslint
* remove unused turfCircle import
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
add performance tests for Login and Home pages
- [x] Rename and reduce the number of environment variables for performance test CI job
- [x] ITERATION_COUNT_ENV => PERF_TEST_COUNT
- [x] DISABLE_CACHE, TEST_RUNNER, TEST_THROTTLE_NETWORK removed
- [x] Use c2-16 queue for performance tests
- [x] Run performance tests by running ES and Kibana separately
- [x] Add warmup phase to performance tests (Not reported to APM). APM Label: PERF_TEST_PHASE
- [x] Add playwright dependency
- [x] Throttle network speed ( downloadThroughput: 750.000 B/s, uploadThroughput: 750.000 B/s, latency: 100 ms )
- [x] Disable cache by using - page.route (‘**’, ….)
- [x] Login and Home pages performance tests.
* chore(NA): splits types from code on @kbn/securitysolution-io-ts-utils
* chore(NA): fix package.json
* chore(NA): merge and solve conflicts with main
* chore(NA): remove any from types
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* chore(NA): splits types from code on @kbn/rule-data-utils
* chore(NA): remove old style imports for this pkg
* chore(NA): eslint fix
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add Log [2]
* Add Log [1]
* Add Log [1]
* Finish update log
* Use moment for time unit
* Fix moment and add types for moment-duration-format
* Remove unsed value ungroupedRatioAlertReasonDescription
* Script trusted to delete the update keys in JP and CZ
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
