## Summary
This PR adds data drift detection workflow from Trained models to Data
comparison view. It also renames Data comparison to Data Drift.
**From the new map view in Trained model list:**
- Clicking on the index icon in the map view will give an option/action
to Analyze data drift
a68163ab-8a83-4378-8cf3-ea49f4480a06
- If model has detected related indices, it will also give an option to
Analyze data drift in the Transform actions
**From the data comparison/drift page:**
- Default screen with list of available data views and saved search will
be shown
<img width="1470" alt="Screen Shot 2023-09-07 at 00 22 01"
src="db13b8b7-9d90-4220-b03e-9f9d12ab53e9">
- But can also customize index patterns for the data sets to analyze.
Upon 'analyzing', a new data view will be created if needed (either
permanently or temporarily).
<img width="1271" alt="Screen Shot 2023-08-29 at 16 56 57"
src="e000e920-162b-4369-8762-70b6244e50e7">
<img width="1470" alt="Screen Shot 2023-09-07 at 00 22 49"
src="6577a530-c3b0-4ab9-95e4-d1d8fd1c9f0a">
- If there exists a data view with exact combination of index patterns
and time field, it will use that data view
- If there exists a data view with the same index patterns but different
time field, it will create a new data view with name
`{referencePattern},{comparisonPattern}-{timeField}`
- If no data view exists that matches, it will create a new data view
with name `{referencePattern},{comparisonPattern}`
## For reviewers:
- **appex-sharedux**: [Small change in the exported type interface for
BaseSavedObjectFinder](https://github.com/elastic/kibana/pull/162853/files#diff-5e2e62df8aba5ac9445962bfa00eee933a386110d0a24dfe6ac0f300a796ccc3)
to correctly list `children` as an accepted prop. This prop which is
used for the `toolsRight`.
- **security-solution**: Renaming of `Data comparison` to `Data Drift`
## Tests:
[Flaky test suite runner with Data Drift
test](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3216#018accc2-d33b-4cd6-a178-589e6698b675)
... successful after 50 runs✅
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**ML Management page**
- Fixes general page loading issues.
- Ensures only enabled features are shown as tabs
- Ensures only jobs for enabled features can be exported and imported.
- Ensures only enabled features are listed in the saved object sync
output.
- On trained models tab:
- Only lists DFA models if NLP is disabled.
- Only lists non-DFA models if DFA is disabled.
**Anomaly Detection**
- Hides node information in anomaly detection jobs list.
- Hides the Exclude frozen data option in the Use full time range
selector in job wizards.
**Data frame analytics**
- Hides all node and license level information.
**Trained models**
- Only lists DFA models if NLP is disabled.
- Only lists non-DFA models if DFA is disabled.
- Hides all node and license level information.
- Hides DFA nodes
**Notifications and memory usage**
- Ensures only enabled features are mentioned. Including selectable
types in the search bar filters.
**Integrations with other plugins**
- Changes registration for integrations into other plugins so they only
happen if the relevant feature is enabled.
- Client side: UI actions, cases, embeddables, alerts, maps.
- Server side: Sample data sets, cases
**AIOPS**
- Hides the Exclude frozen data option in the Use full time range
selector on all pages
**Notes for non ML team reviewers**
**response-ops**
I've divided the
[persistable_state.ts](https://github.com/elastic/kibana/pull/163724#diff-e02dc0b6cb5b63965372b1f4a84d2287cba31a15ab525ab7983f02d09f23879f)
test into basic and trial version.
The ML cases attachments should only be registered if anomaly detection
is available in a trial or platinum license. This was a bug which I
noticed when making serverless changes.
**Observability**
I've made a few minor changes to the nav menu, fixing names of ML
features and adding the missing Change point detection AIOPs page.
**Security solution**
I've made a few minor changes to the nav menu, fixing names of ML
features and adding some missing ML features.
I think the icons being used will need to be revisited before release as
we have [official ML
icons](https://elastic.github.io/eui/#/display/icons#apps) but not for
every page. So we should probably either have new icons created or all
agree on which standard non-ML icons should be used for the ones which
are missing.
**Search**
The NLP feature is currently disabled in main, I believe this was an
attempt to stop ML anomaly detection alert rules from being registered.
I've reenabled NLP and changed the way we're registering the alerts.
They will now only be registered if the anomaly detection feature is
enabled.
Fixes https://github.com/elastic/kibana/issues/163372
- Adds function to ML plugin server side setup contract
(`setFeaturesEnabled`) to enable/disable features. Features being,
anomaly detection, data frame analytics or natural language processing.
- Each serverless plugin (search, observability and security) now calls
`setFeaturesEnabled` to enable different features on setup.
- Adds three new feature capabilities, `isADEnabled`, `isDFAEnabled`,
`isNLPEnabled`
- Updates the capabilities switcher to toggle these capabilities based
on the shared feature API. This currently has a bug where the switcher
isn't triggered for API tag checking.
- deeplinks are now only registered based on the feature capabilities,
these control what is shown in the nav menu for search and observability
projects.
- Adds the Machine Learning nav menu section to the `serverless_search`
plugin, to match the `serverless_observability` plugin.
- Does not updated the `serverless_security` plugin's nav menu. (update,
serverless_observability nav menu also now does not contain ML)
Relates to https://github.com/elastic/kibana/issues/160891
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
