# Backport
This will backport the following commits from `main` to `8.x`:
- [[ML] Move FieldStatsFlyout, parseInterval, validators to packages.
(#193657)](https://github.com/elastic/kibana/pull/193657)
<!--- Backport version: 8.9.8 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Walter
Rafelsberger","email":"walter.rafelsberger@elastic.co"},"sourceCommit":{"committedDate":"2024-09-24T12:00:55Z","message":"[ML]
Move FieldStatsFlyout, parseInterval, validators to packages.
(#193657)\n\n## Summary\r\n\r\nPart of #159917.\r\n\r\nMoves code from
`plugins/ml/common|public` to packages that is used by\r\ntransforms
too.\r\n\r\nWhile the transforms plugin is maintained by the ML team
too, the\r\ntransform plugin itself is independently available from the
ML UI in the\r\nKibana management section. We should try to avoid that
the transform\r\nplugin is directly depending on the `ml` plugin. This
PR moves some code\r\nfrom `plugins/ml/common|public` to packages so
that we can remove `ml`\r\nfrom the list of `requiredBundles` of the
`transform` plugin.\r\n\r\nThe packages were created with these
commands:\r\n\r\n```\r\nnode scripts/generate package
@kbn/ml-field-stats-flyout --dir
./x-pack/packages/ml/field_stats_flyout\r\nnode scripts/generate package
@kbn/ml-parse-interval --dir ./x-pack/packages/ml/parse_interval\r\nnode
scripts/generate package @kbn/ml-validators --dir
./x-pack/packages/ml/validators\r\n```\r\n\r\nThe following commands
were used to check missing jsdoc comments
and\r\nexports:\r\n\r\n```\r\nnode scripts/build_api_docs --plugin
@kbn/ml-field-stats-flyout --stats comments\r\nnode
scripts/build_api_docs --plugin @kbn/ml-field-stats-flyout --stats
exports\r\nnode scripts/build_api_docs --plugin @kbn/ml-parse-interval
--stats comments\r\nnode scripts/build_api_docs --plugin
@kbn/ml-parse-interval --stats exports\r\nnode scripts/build_api_docs
--plugin @kbn/ml-validators --stats comments\r\nnode
scripts/build_api_docs --plugin @kbn/ml-validators --stats
exports\r\n```\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [x] This was checked for breaking
API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b2d85d5f8f10f7c460c3a3b732844a19c8b20d7b","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":[":ml","release_note:skip","Feature:Transforms","v9.0.0","backport:prev-minor","v8.16.0"],"number":193657,"url":"https://github.com/elastic/kibana/pull/193657","mergeCommit":{"message":"[ML]
Move FieldStatsFlyout, parseInterval, validators to packages.
(#193657)\n\n## Summary\r\n\r\nPart of #159917.\r\n\r\nMoves code from
`plugins/ml/common|public` to packages that is used by\r\ntransforms
too.\r\n\r\nWhile the transforms plugin is maintained by the ML team
too, the\r\ntransform plugin itself is independently available from the
ML UI in the\r\nKibana management section. We should try to avoid that
the transform\r\nplugin is directly depending on the `ml` plugin. This
PR moves some code\r\nfrom `plugins/ml/common|public` to packages so
that we can remove `ml`\r\nfrom the list of `requiredBundles` of the
`transform` plugin.\r\n\r\nThe packages were created with these
commands:\r\n\r\n```\r\nnode scripts/generate package
@kbn/ml-field-stats-flyout --dir
./x-pack/packages/ml/field_stats_flyout\r\nnode scripts/generate package
@kbn/ml-parse-interval --dir ./x-pack/packages/ml/parse_interval\r\nnode
scripts/generate package @kbn/ml-validators --dir
./x-pack/packages/ml/validators\r\n```\r\n\r\nThe following commands
were used to check missing jsdoc comments
and\r\nexports:\r\n\r\n```\r\nnode scripts/build_api_docs --plugin
@kbn/ml-field-stats-flyout --stats comments\r\nnode
scripts/build_api_docs --plugin @kbn/ml-field-stats-flyout --stats
exports\r\nnode scripts/build_api_docs --plugin @kbn/ml-parse-interval
--stats comments\r\nnode scripts/build_api_docs --plugin
@kbn/ml-parse-interval --stats exports\r\nnode scripts/build_api_docs
--plugin @kbn/ml-validators --stats comments\r\nnode
scripts/build_api_docs --plugin @kbn/ml-validators --stats
exports\r\n```\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [x] This was checked for breaking
API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b2d85d5f8f10f7c460c3a3b732844a19c8b20d7b"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193657","number":193657,"mergeCommit":{"message":"[ML]
Move FieldStatsFlyout, parseInterval, validators to packages.
(#193657)\n\n## Summary\r\n\r\nPart of #159917.\r\n\r\nMoves code from
`plugins/ml/common|public` to packages that is used by\r\ntransforms
too.\r\n\r\nWhile the transforms plugin is maintained by the ML team
too, the\r\ntransform plugin itself is independently available from the
ML UI in the\r\nKibana management section. We should try to avoid that
the transform\r\nplugin is directly depending on the `ml` plugin. This
PR moves some code\r\nfrom `plugins/ml/common|public` to packages so
that we can remove `ml`\r\nfrom the list of `requiredBundles` of the
`transform` plugin.\r\n\r\nThe packages were created with these
commands:\r\n\r\n```\r\nnode scripts/generate package
@kbn/ml-field-stats-flyout --dir
./x-pack/packages/ml/field_stats_flyout\r\nnode scripts/generate package
@kbn/ml-parse-interval --dir ./x-pack/packages/ml/parse_interval\r\nnode
scripts/generate package @kbn/ml-validators --dir
./x-pack/packages/ml/validators\r\n```\r\n\r\nThe following commands
were used to check missing jsdoc comments
and\r\nexports:\r\n\r\n```\r\nnode scripts/build_api_docs --plugin
@kbn/ml-field-stats-flyout --stats comments\r\nnode
scripts/build_api_docs --plugin @kbn/ml-field-stats-flyout --stats
exports\r\nnode scripts/build_api_docs --plugin @kbn/ml-parse-interval
--stats comments\r\nnode scripts/build_api_docs --plugin
@kbn/ml-parse-interval --stats exports\r\nnode scripts/build_api_docs
--plugin @kbn/ml-validators --stats comments\r\nnode
scripts/build_api_docs --plugin @kbn/ml-validators --stats
exports\r\n```\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or
functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere
updated or added to match the most common scenarios\r\n- [ ] [Flaky
Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1)
was\r\nused on any tests changed\r\n- [x] This was checked for breaking
API changes and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"b2d85d5f8f10f7c460c3a3b732844a19c8b20d7b"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
## Summary
The `AnomalyExplorerChartsService` was importing the
`SWIM_LANE_LABEL_WIDTH` numerical constant from
`swimlane_container.tsx`. As a result, the _entirety_ of that file was
being included in the async bundle.
`AnomalyExplorerChartsService` is loaded asynchronously on page load by
the embeddable. By relocating the constant to its own file-- as well as
other optimizations (see below)-- we reduce the async page load by 74%,
(in dev mode).
### Before - `351k`
<img width="1220" alt="Screenshot 2024-03-24 at 10 09 31 AM"
src="6cea41f5-fb10-41c3-8951-a4be897174f9">
### After - `93.4k`
<img width="1471" alt="Screenshot 2024-04-05 at 11 45 45 AM"
src="a07e7a19-c1af-4b45-a195-69730fc61b0c">
Unfortunately, this change led to a bloating of async modules, the cause
of which is still unclear. The application async chunk weighed in at 2.2
MB compressed! To get this PR to a shippable state, I refactored
additional code to bring down duplication and bloat.
The result is an `ml` experience that fetches small bundles on demand as
someone interacts with it:
More work can be done to continue to optimize the plugin, of course, but
this set of changes is an excellent start, (and case study on bundle
load).
### Other optimizations in this PR
- Registration of some `start` services are conditional, and contain
their own async calls. I've removed these from the register helper,
(which itself is a brute-force offload of code from the plugin, but is
still loaded every time), and loaded them async if the conditions apply.
- Routing in `ml` use factories to create individual routes. In a lot of
cases, the pages these routes displayed were not asynchronously loaded,
adding tremendous amounts of React code to the root application.
- I made all pages loaded by routes async.
- In some cases, the components themselves were colocated with the route
factory. I moved those to their own files for async import.
- Similarly, some state managers were colocated. Those were moved to
their own files, as well.
- Moved flyouts, modals, expanding rows to `React.lazy` async modules,
(using `dynamic` from `@kbn/shared-ux-utility`.
- Refactored `export * from` directives from `public/shared.ts` to
accurately reflect what is being consumed outside the `ml` plugin, (and
also reduce the size of _that_ bundle.
- Refactored `lodash` imports to submodule imports to enable
tree-shaking, (if ever enabled in webpack).
- Moved `getMlGlobalServices` off of the `app.tsx` file for import by
others, (to avoid depending on the rest of the code there).
- Moved some types to `/common` to avoid importing code, (though,
admittedly, types are compiled away). But it felt cleaner to move them
out.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
From the rule management page in observability, user will be able to
create an anomaly detection rule.
<img width="1789" alt="image"
src="9ce84628-bf1b-44f6-a51b-5bd2ad94d9bd">
<img width="1792" alt="image"
src="0add536c-2e62-4de5-9e06-2beedd0c0fe2">
Refactors outdated approach of sharing the custom data grid component
from the `ml` plugin to the `transform` plugin to use packages instead.
Creates the following packages:
- `@kbn/ml-data-grid`
- `@kbn/ml-date-utils`
- `@kbn/ml-runtime-field-utils`
----
- Imports have been refactored so the `transform` plugin no longer
imports code from packages `@kbn/ml-anomaly-utils` and
`@kbn/ml-data-frame-analytics-utils`.
- Removed again auto-generated `@type` related JSDoc comments. Within
the code those annotations are quite redundant, would be cumbersome to
maintain and they are not necessary to satisfy the "missing comments"
check.
- The `renderCellPopover` callback has been refactored out of the
`DataGrid` component and can now be passed in as an optional prop. It is
only used for Data Frame Analytics and this way we can avoid some
dependency of the transform plugin on DFA related code.
- Some more code has been moved to `@kbn/ml-anomaly-utils` to make
available via packages what's needed for the data grid.
Creates Package `@kbn/ml-anomaly-utils`.
- This moves some of the utility functions, constants and types related
to Anomaly Detection we previously exposed from the `ml` plugin itself
to a package `@kbn/ml-anomaly-utils`, resulting in ~5.9KB reduction of
the plugin's page load bundle size.
- To reduce increases in async bundle size for consuming plugins the
utils have been refactored into individual files to allow deep imports
and get some optimization through that. Some previously duplicated code
in consuming plugins has been deleted and replaced with deep imports of
the corresponding original code in the package.
- Types have been prefixed with `Ml`.
- Constants have been prefixed with `ML_`.
Created package via `node scripts/generate package @kbn/ml-anomaly-utils
--web --dir ./x-pack/packages/ml/anomaly_utils`.
- Moves code from `x-pack/plugins/ml/common/util/errors` that was shared
via `x-pack/plugins/ml/public/shared.ts` to `@kbn/ml-error-utils`.
- `data_visualizer` and `aiops` plugins now use that package instead of
code duplication.
Moves some ML utility code to packages.
- @kbn/ml-agg-utils contains multiple utilities used in combination related to building aggregations.
- @kbn/ml-is-populated-object contains the isPopulatedObject() utility function used across several plugins.
- @kbn/ml-string-hash contains the stringHash() utility function used across several plugins.
* wip: create anomalies layer in maps
* ensure anomaly search respects time filter
* add custom color ramp and connected layer
* update ml api service types
* add tooltip
* update naming and types
* update warning messages. make actual layer default
* remove any
- Gets rid of imports from 'elasticsearch' and replaces them with '@elastic/elasticsearch'.
- Extends isPopulatedObject() to allow an optional array of attributes to check if they exist. Allows us to get rid of the manual and inconsistent usages of hasOwnProperty().
Fixes chart histograms for runtime fields. The runtime field configurations were not passed on to the endpoint to fetch the charts data, so charts ended up being empty with a 0 documents legend.
- Adds schema definitions to transform API endpoints and adds API integration tests.
- The type definitions based on the schema definitions can be used on the client side too.
- Adds apidoc documentation.
