[role="xpack"] [[xpack-ml]] = {ml-cap} [partintro] -- As data sets increase in size and complexity, the human effort required to inspect dashboards or maintain rules for spotting infrastructure problems, cyber attacks, or business issues becomes impractical. Elastic {ml-features} such as {anomaly-detect} and {oldetection} make it easier to notice suspicious activities with minimal human interference. {kib} includes a free *{data-viz}* to learn more about your data. In particular, if your data is stored in {es} and contains a time field, you can use the *{data-viz}* to identify possible fields for {anomaly-detect}: [role="screenshot"] image::user/ml/images/ml-data-visualizer-sample.png[{data-viz} for sample flight data] You can also upload a CSV, NDJSON, or log file. The *{data-viz}* identifies the file format and field mappings. You can then optionally import that data into an {es} index. To change the default file size limit, see <>. If {stack-security-features} are enabled, users must have the necessary privileges to use {ml-features}. Refer to {ml-docs}/setup.html#setup-privileges[Set up {ml-features}]. NOTE: There are limitations in {ml-features} that affect {kib}. For more information, refer to {ml-docs}/ml-limitations.html[{ml-cap}]. -- [[xpack-ml-anomalies]] == {anomaly-detect-cap} The Elastic {ml} {anomaly-detect} feature automatically models the normal behavior of your time series data — learning trends, periodicity, and more — in real time to identify anomalies, streamline root cause analysis, and reduce false positives. {anomaly-detect-cap} runs in and scales with {es}, and includes an intuitive UI on the {kib} *Machine Learning* page for creating {anomaly-jobs} and understanding results. If you have a license that includes the {ml-features}, you can create {anomaly-jobs} and manage jobs and {dfeeds} from the *Job Management* pane: [role="screenshot"] image::user/ml/images/ml-job-management.png[Job Management] You can use the *Settings* pane to create and edit calendars and the filters that are used in custom rules: [role="screenshot"] image::user/ml/images/ml-settings.png[Calendar Management] The *Anomaly Explorer* and *Single Metric Viewer* display the results of your {anomaly-jobs}. For example: [role="screenshot"] image::user/ml/images/ml-single-metric-viewer.png[Single Metric Viewer] You can optionally add annotations by drag-selecting a period of time in the *Single Metric Viewer* and adding a description. For example, you can add an explanation for anomalies in that time period or provide notes about what is occurring in your operational environment at that time: [role="screenshot"] image::user/ml/images/ml-annotations-list.png[Single Metric Viewer with annotations] In some circumstances, annotations are also added automatically. For example, if the {anomaly-job} detects that there is missing data, it annotates the affected time period. For more information, see {ml-docs}/ml-delayed-data-detection.html[Handling delayed data]. The *Job Management* pane shows the full list of annotations for each job. NOTE: The {kib} {ml-features} use pop-ups. You must configure your web browser so that it does not block pop-up windows or create an exception for your {kib} URL. For more information about the {anomaly-detect} feature, see https://www.elastic.co/what-is/elastic-stack-machine-learning[{ml-cap} in the {stack}] and {ml-docs}/ml-ad-overview.html[{ml-cap} {anomaly-detect}]. [[xpack-ml-dfanalytics]] == {dfanalytics-cap} The Elastic {ml} {dfanalytics} feature enables you to analyze your data using {classification}, {oldetection}, and {regression} algorithms and generate new indices that contain the results alongside your source data. If you have a license that includes the {ml-features}, you can create {dfanalytics-jobs} and view their results on the *Data Frame Analytics* page in {kib}. For example: [role="screenshot"] image::user/ml/images/classification.png[{classification-cap} results in {kib}] For more information about the {dfanalytics} feature, see {ml-docs}/ml-dfanalytics.html[{ml-cap} {dfanalytics}]. [[xpack-ml-aiops]] == AIOps Labs AIOps Labs is a part of {ml-app} in {kib} which provides features that use advanced statistical methods to help you interpret your data and its behavior. [discrete] [[explain-log-rate-spikes]] === Explain log rate spikes preview::[] Explain log rate spikes is a feature that uses advanced statistical methods to identify reasons for increases in log rates. It makes it easy to find and investigate causes of unusual spikes by using the analysis workflow view. Examine the histogram chart of the log rates for a given {data-source}, and find the reason behind a particular change possibly in millions of log events across multiple fields and values. You can find explain log rate spikes under **{ml-app}** > **AIOps Labs** where you can select the {data-source} or saved search that you want to analyze. [role="screenshot"] image::user/ml/images/ml-explain-log-rate-before.png[Log event histogram chart] Select a spike in the log event histogram chart to start the analysis. It identifies statistically significant field-value combinations that contribute to the spike and displays them in a table. You can optionally choose to summarize the results into groups. The table also shows an indicator of the level of impact and a sparkline showing the shape of the impact in the chart. Hovering over a row displays the impact on the histogram chart in more detail. You can inspect a field in **Discover** by selecting this option under the **Actions** column. You can also pin a table row by clicking on it then move the cursor to the histogram chart. It displays a tooltip with exact count values for the pinned field which enables closer investigation. Brushes in the chart show the baseline time range and the deviation in the analyzed data. You can move the brushes to redefine both the baseline and the deviation and rerun the analysis with the modified values. [role="screenshot"] image::user/ml/images/ml-explain-log-rate.png[Log rate spike explained] [discrete] [[log-pattern-analysis]] === Log pattern analysis preview::[] Log pattern analysis helps you to find patterns in unstructured log messages and makes it easier to examine your data. It performs categorization analysis on a selected field of a {data-source}, creates categories based on the data and displays them together with a chart that shows the distribution of each category and an example document that matches the category. You can find log pattern analysis under **{ml-app}** > **AIOps Labs** where you can select the {data-source} or saved search that you want to analyze. [role="screenshot"] image::user/ml/images/ml-log-pattern-analysis.png[Log pattern analysis UI] Select a field for categorization and optionally apply any filters that you want, then start the analysis. The analysis uses the same algorithms as a {ml} categorization job. The results of the analysis are shown in a table that makes it possible to open **Discover** and show or filter out the given category there, which helps you to further examine your log messages.