[8.12][DOCS] Fixing Whats New 8.12 (#175102)

Trying to fix the Whats New page for 8.12. It exists in main but not in
8.12 so hopefully this PR fixes that.

docs/user/whats-new.asciidoc

@@ -1,246 +1,151 @@
 [[whats-new]]
 == What's new in {minor-version}
 
+
 Here are the highlights of what's new and improved in {minor-version}.
 For detailed information about this release,
 check the <<release-notes, release notes>>.
 
-Previous versions: {kibana-ref-all}/8.10/whats-new.html[8.10] | {kibana-ref-all}/8.9/whats-new.html[8.9] | {kibana-ref-all}/8.8/whats-new.html[8.8] | {kibana-ref-all}/8.7/whats-new.html[8.7] | {kibana-ref-all}/8.6/whats-new.html[8.6] | {kibana-ref-all}/8.5/whats-new.html[8.5] | {kibana-ref-all}/8.4/whats-new.html[8.4] | {kibana-ref-all}/8.3/whats-new.html[8.3] | {kibana-ref-all}/8.2/whats-new.html[8.2] 
+
+Previous versions: {kibana-ref-all}/8.11/whats-new.html[8.11] | {kibana-ref-all}/8.10/whats-new.html[8.10] | {kibana-ref-all}/8.9/whats-new.html[8.9] | {kibana-ref-all}/8.8/whats-new.html[8.8] | {kibana-ref-all}/8.7/whats-new.html[8.7] | {kibana-ref-all}/8.6/whats-new.html[8.6] | {kibana-ref-all}/8.5/whats-new.html[8.5] | {kibana-ref-all}/8.4/whats-new.html[8.4] | {kibana-ref-all}/8.3/whats-new.html[8.3] | {kibana-ref-all}/8.2/whats-new.html[8.2]
 | {kibana-ref-all}/8.1/whats-new.html[8.1] | {kibana-ref-all}/8.0/whats-new.html[8.0]
 
+[discrete]
+=== Dashboard
+
+[discrete]
+==== Edit {esql} in a dashboard
+We are introducing editing of an {esql} query on a dashboard and allows the users to select among different chart suggestions. This is quite powerful since users don't need to go back to Discover to edit the query and recreate the chart, they can simply adjust the query right there on a dashboard.
+
+[discrete]
+==== Improved ES|QL in-app documentation search
+Many users open the {esql} editor’s documentation popover to familiarize themselves with the commands and find examples. Our search input was searching only on the titles of the commands/functions and not the description of each. As a result users were failing to find what they wanted. For example, if they searched for IP then CIDR_MATCH would not appear, but only TO_IP. This change helps users learn {esql} faster by improving the search.
+
+[role="screenshot"]
+image::images/esql-in-app.png[A screenshot of the {esql} in app documentation]
+
+[discrete]
+==== Improved error messages for {ccs}
+Customers querying data from multiple clusters link:{ref}/modules-cross-cluster-search.html[({ccs-init} queries)] will get more information on why their search failed for each of the visualizations in a dashboard as well as in the Discover application.
+
+[role="screenshot"]
+image::images/improved-errors.png[A screenshot of an improved error message, width=50%]
+
 [discrete]
 === Discover
 
 [discrete]
-==== {esql} in Discover
-
-preview:[] In 8.11 we are introducing link:https://www.elastic.co/guide/en/elasticsearch/reference/master/esql.html[{es} Query Language ({esql})], Elastic’s new piped language for data exploration and investigation. {esql} transforms, enriches, and simplifies your data exploration process. 
+==== Improved long field names handling in {kib}
+Long field names are very normal in Observability and Security datasets. That’s why we adapted multiple elements in Discover, Dashboards, Maps, and Lens such as field selectors, table headers, filter pills, and chart tooltips amongst others to handle long field names. For example, you will notice it when you select a field to set some filters or when you mouse over a chart.
 
 [role="screenshot"]
-image::images/esql-demo.gif[A short video demo of {esql}]
-
-Here is what you can expect: 
-
-* *Easy start:* To begin using {esql} in **Discover**, select **Try {esql}** from the data view menu.
-* *Efficient and easy query building:* {esql} in Discover offers auto-complete and in-app documentation, making it easy to craft powerful queries right from the query bar. 
-* *Comprehensive and powerful data exploration:* Conduct ad-hoc data exploration within **Discover**. Create aggregations, transform data, enrich datasets, and more directly from the query builder. Results are presented in a tabular format or as visualizations. It depends on the query you are executing. 
-* *Contextual visualizations:* When writing {esql} queries in **Discover**, you’ll receive visual representations powered by the **Lens** suggestion engine. Your query’s nature determines the type of visualization you get, such as a metric, histogram, heatmap, and so on. 
-* *Enrichment:* Use the enrich command to enhance your query dataset with fields from another dataset, complete with in-context suggestions for the selected policy, such as hinting the matching field and enriched columns.
-* *In-line visualization editing:* Edit {esql} visualizations directly within **Discover** and dashboards. No need to navigate to **Lens** for quick edits, so you can make changes seamlessly. 
-* *Dashboard integration:* Save your {esql} visualizations to a dashboard directly from **Discover** once you’re satisfied with the results. 
-* *Alerting:* Use {esql} for Observability and Security alerts, setting aggregated values as thresholds. Enhance detection accuracy and receive actionable notifications by emphasizing meaningful trends over isolated incidents, reducing false positives. 
-
-{esql} in **Discover** brings efficiency and power to your data investigations, streamlining your path to insights. 
-
-Learn more about {esql}’s capabilities in the link:{ref-bare}/master/esql.html[{es} {esql} documentation]. 
-
-
-[role="screenshot"]
-image::images/obvs.png[An example of {esql}]
-
-[role="screenshot"]
-image::images/obv-use-case.png[An example using {esql} in an {observability} use case]
-
+image::images/long-field-names.png[A screenshot of the improved long field name handling in {kib}]
 
 [discrete]
-=== Dashboard 
-
-[discrete]
-==== Lens inline editing in dashboards
-
-You can now edit a **Lens** visualization without leaving the dashboard instead of navigating back and forth to the **Lens** editor. Open the panel menu and select **Edit visualization**. A flyout will be open in the dashboard where you can perform any edits to your **Lens** panels. Once happy with your edits, click **Apply and close**. This new editing experience is more convenient and will save you time since the dashboard will not need to reload when saving your changes. 
+==== Improved search for field names by handling spaces like wildcards
+To improve data exploration, we improved the search within the field list by allowing users to do a more flexible search in the fields sidebar with terms containing spaces.
 
 [role="screenshot"]
-image::images/lens-inline-editing.gif[An example of inline editing in Lens]
-
-[discrete]
-==== Links panel
-
-You can now easily navigate from one dashboard to another using the links panel. Better organize your dashboards and make them more performant by chunking them in multiple dashboards with fewer visualizations and linking them together. You can carry over your filters, query, and time range when navigating to other related dashboards. Display your links horizontally or vertically as it better suits your dashboard layout. You can also use the links panel to include external links in your dashboards, such as to your wiki page or other applications. Decide whether you want to open the links in the same browser tab or in a new one.
-
-[role="screenshot"]
-image::images/links-panel.gif[An example of the new links panel]
-
-[role="screenshot"]
-image::images/edit-links-panel.png[An example of the edit links panel flyout]
-
-[role="screenshot"]
-image::images/edit-links.png[An example of how to edit links using the panel flyout]
-
-[discrete]
-==== Color mapping for enhanced data visualization
-
-Color is a fundamental visual element, alongside position and shape, that plays a crucial role in conveying information effectively. 
-
-[role="screenshot"]
-image::images/color-mapping.png[An example of the new color mapping feature]
-
-Here’s what our new color mapping feature offers:
-
-* *Effortless categorization:* Easily assign one or more field categories to specific colors. This makes it simpler than ever to organize and understand your data. 
-* *Guided color selection:* Our intuitive color chooser provides you with predefined palettes that ensure your charts not only look great, but also align seamlessly with different {kib} themes. 
-* *Enhanced color palettes:* We’ve reintroduced a clear concept of color palettes, making it easier for you to select and apply gradients, improving the overall aesthetics of your visualizations. 
-
-Whether you’re working with cartesian, partition, or tag clouds charts, these enhancements are designed to help you make the most of your data. With this feature, you can expect an improved ability to categorize, differentiate, and emphasize data points on your charts, ultimately leading to better insights and more visually appealing dashboards.
-
-[discrete]
-==== {ccs-cap} ({ccs-init}) query inspector
-
-Customers querying data from multiple clusters link:{ref}/modules-cross-cluster-search.html[({ccs-init} queries)] will get more information about each of the cluster's responses. For each of the visualizations in a dashboard as well as in **Discover**, you can look at response times per cluster and shard failures. This is especially important when one or more clusters are not able to provide all the data, so you know that you are looking at partial results and why they fail.
-
-[role="screenshot"]
-image::images/ccs-query-inspector.png[An example of the new {ccs} query inspector, width=50%]
-
-[discrete]
-==== Individual annotation editing from the library
-
-We've introduced full annotation group editing in the **Visualize Library**. Now, you can easily edit shared annotation groups without leaving Lens. No more searching for consuming visualizations.
-
-[role="screenshot"]
-image::images/annotation-editing.png[An example of annotation editing]
+image::images/allow-spaces.png[A screenshot of the search within the field list allowing spaces, width=70%]
 
 [discrete]
 === Machine Learning
- 
-[discrete]
-==== ELSER is improved and is now generally available
-
-In 8.8, we introduced Elastic Learned Sparse Encoder in technical preview. ELSER is Elastic’s text expansion language model for AI search. It offers superior relevance out of the box, without the need for retraining on in-domain data or any other ML or MLOps effort. Deploy it with a couple of clicks from Elastic’s UI and start leveraging the power of AI with your search. 
-
-In 8.11, we're releasing a generally available second version. ELSER model-2 comes in two versions:
-
-* The optimized model, which runs on the linux-x86_64 platform
-* The cross-platform model 
-
-Both ELSER model-2 versions, platform-optimized and cross-platform, show improved relevance compared to the original ELSER release, as measured against the BEIR benchmarks. Importantly, the optimized version also shows significantly improved performance (reduced inference latency). The {ecloud} supports the optimized version and so {ecloud} users will benefit from the materially improved performance of the optimized ELSER model-2.
-
-[role="screenshot"]
-image::images/elser-model-2.png[An example of ELSER model 2, width=60%]
-
-NOTE: The original version of ELSER (the model available prior to 8.11) will remain in technical preview. 
 
 [discrete]
-==== Inference APIs
+==== Unified inference API now integrates OpenAI and HuggingFace
+In 8.11 we introduced a unified inference API that abstracts away the complexity of performing inference on different models for different tasks.
 
-preview:[] We are working to introduce a unified inference API that abstracts away the complexity of performing inference on different models that are trained for different tasks. The API introduces a simple, intuitive syntax of the form:
+We released an MVP iteration of this framework in technical preview which initially supported ELSER in an Elastic deployment and we hinted that in future releases, the inference API will support both internal and external models and will integrate with the LLM ecosystem.
 
-[source,bash]
+And so in 8.12 Elastic’s Inference API is extended to integrate with external models to perform AI search inference using:
+
+* OpenAI embeddings
+* HuggingFace embeddings and
+* ELSER on HuggingFace
+
+AI search with embeddings achieves superior contextual relevance and captures user intent. Inference using these new capabilities involves external calls to the corresponding endpoints on OpenAI and HuggingFace. The power of the inference API lies in its simple, unified syntax that abstracts away the underlying complexity of using different internal and external models for different tasks.
+
+Performing inference on the newly supported models and services is as simple as a call with the simple syntax introduced in 8.11:
+
+[source, bash]
 ----
-POST /_inference/<task_type>/<model_id>
+PUT /_inference/<task_type>/<model_id>
 ----
 
-In 8.11, we're'releasing a contained first link:{ref}/inference-apis.html[MVP iteration of this framework], which initially only supports ELSER. This link:{ref}/semantic-search-elser.html#inference-ingest-pipeline[greatly simplifies the syntax] for creating an inference pipeline.
+Concretely, this is how this syntax shapes up for inference with OpenAI embeddings, showcasing the power of Elastic’s unified inference API:
 
-More importantly, in the future the new inference API will support both internal and external models and will integrate with the LLM ecosystem for our users to have the most powerful AI effortlessly and seamlessly at their fingertips, through a unified, self-explanatory API.
+[source, bash]
+----
+PUT _inference/text_embedding/openai_embeddings
+----
+For a detailed example, see link:{ref}/semantic-search-inference.html[this tutorial]. Bear in mind that you will need an OpenAI account and the corresponding API key, as well as to choose the specific OpenAI embeddings that you want to use.
+
+HuggingFace enables access to many open source models while also providing granular control over how the models are deployed. Tailor the deployment environment to your needs by configuring the number of replicas and whether to run the model on a CPU or GPU.
+
+We will continue enhancing Elastic’s inference API with more capabilities and support for more models and tasks for our users to have the most powerful AI effortlessly and seamlessly.
 
 [discrete]
-==== AIOps: Log rate analysis supports text fields 
+==== First-class support for E5 multilingual embeddings
+ELSER is Elastic’s text expansion language model for AI search in English. It offers superior relevance out of the box, without the need for retraining on in-domain data. ELSER is the AI search model of choice for the English language. ELSER v2 is Generally Available as of 8.11.
 
-preview:[] Continuing enhancing log analysis capabilities with smart AIOps tools for drastically shorter mean time to repair. We now support detection of log rate changes that are due to text fields, for example the common **message** log field. Previously log rate change was limited to detecting spikes and dips caused by keyword fields. By adding text fields, we've incorporated pattern analysis into log rate analysis. That has significantly reduced the time it takes to detect and diagnose events that used to go unnoticed for periods of time. 
+For AI search in languages other than English, you can now use E5 multilingual embeddings straight from the Trained Models UI. Like ELSER, E5 has two versions: an Intel-optimized one and a cross-platform one (which runs on any hardware). The Model Management > Trained Models UI shows you which version of E5 is recommended to deploy based on your cluster’s hardware (also see the next section for the redesigned Trained Models UI). The supported model version of E5 is `multilingual-e5-small`. For more details, see our link:{ml-docs}/ml-nlp-e5.html[documentation]. Note that E5 is used under the MIT license.
+
+[discrete]
+==== A redesigned trained models UI that brings together our AI search capabilities
+In 8.12, we have redesigned the way you can add trained models to your deployment through the Trained Models UI for better guidance and usability.
+
+The flyout to add a trained model includes a tab for ELSER and E5 which can be deployed with one click. The UI also guides you as to the recommended version of each model (Intel-optimized or cross-platform), depending on your underlying hardware. A second tab guides you through deploying any other model on Elastic using the Eland Python client.
 
 [role="screenshot"]
-image::images/text-field-support.png[An example of log rate analysis supporting text fields]
+image::images/trained-models-ui.png[A screenshot of the redesigned trained models UI]
 
 [discrete]
-==== Data drift workflows
- 
-preview:[] In 8.10, we introduced the **Data comparison** view to help you detect data drift. In 8.11, we have renamed it to **Data drift** and we have enhanced it to include workflows that help you visualize changes in the model input data and detect potential model performance degradation over time.
+==== AIOps: Log Rate Analysis is GA
+Log Rate Analysis helps you investigate significant increases or decreases of your log rates fast and easy. It helps you identify the reasons behind these changes. Just click on a spike or dip and it will show you the fields (or combinations of fields) that contribute to these changes and, if it helps, continue your investigation by inspecting your selected field in Discover. We consistently enhanced Log Rate Analysis during the past few releases to support both spikes and dips analysis, support for text fields by leveraging Log Pattern Analysis, integration with Discover and more. In 8.12 we added the ability to easily create a categorization anomaly detection job from the pattern analysis flyout in Discover and importantly Log Rate Analysis becomes GA.
+
+[discrete]
+==== Alerts in Anomaly Explorer
+In 8.12 we have enhanced the Anomaly Explorer UI to include insights about alerts generated by rules that use your anomaly detection jobs.
 
 [role="screenshot"]
-image::images/data-drift.png[An example of data drift workflows]
+image::images/alerts-anomaly.png[A screenshot of the anomaly explorer UI]
 
-[discrete]
-==== Improvements in anomaly detection embeddability and data frame analytics pipelines
+These insights include:
 
-We have improved the UX for attaching anomaly swim lanes and anomaly charts to dashboards. It is now more friendly and consistent with the same functionality from Lens. In addition, you can now attach these ML charts to new dashboards (previously this was only able to be done for existing ones). 
+* a line chart of the alerts count and their correlation with the anomalies detected,
+* an alert context menu when an anomaly swimlane cell is selected,
+* a summary section including the alert duration, start and recovery time and more information and a
+* Details tab from which the user can select to open an alert’s detail page and attach an alert to a new or existing case.
 
-In **Data Frame Analytics**, we previously added the ability to link directly to **Discover** and **Dashboards** from the results data grid filtering for the row’s field/values for all visible columns. For improved usability, you can now do this during the job creation as well. We have also made UX improvements for the deployment of trained models from Data Frame Analytics jobs, including an option to reindex your data at the end of the ingest pipeline creation.
+[role="screenshot"]
+image::images/alerts.png[A screenshot of details of the alerts]
 
 [discrete]
 === ResponseOps
 
 [discrete]
-==== New {esql} rule type
-
-A new {esql} alerting rule type is now available under the existing {es} rule type. This rule type brings all the new functionalities that are available within the new and powerful language, {esql}, to {kib} Alerting to allow and unlock new alerting use cases.
-
-With the new type, users will be able to generate a single alert based on defined {esql} query and preview the query result before saving the rule. When the query returns an empty result no alerts will be generated.
+==== Maintenance window filters
+In 8.12 you can add KQL filters to your <<maintenance-windows,maintenance windows>> to further refine their scope:
 
 [role="screenshot"]
-image::images/esql-rule-type.gif[An example of creating a rule type using {esql}]
+image::images/maintenance-window-filter.png[A screenshot of the create maintenance window UI]
 
 [discrete]
-==== {kib} cases custom field
-
-A new functionality is now available in {kib} cases. Users will be able to add custom fields to the case structure, so they can use it for better classification and case enrichment. As a first step, those fields will be available in the case view only. In the next step, we’re planning to have more field types, dedicated privileges, support filters, and search capabilities in the case table.
+==== Case improvements
+The enhanced case view is now supported by any field filter and any change to the view is saved to local cache to ensure your data won't be lost.
 
 [role="screenshot"]
-image::images/custom-field.gif[An example of creating a custom case field in {kib}]
+image::images/cases.png[A screenshot of the enhanced case view]
+
+There is also a new {kib} <<setup-cases,sub-feature privilege>> that enables you to customize access to case settings.
+
+If you <<add-case-files,add files>> to cases, there is a new option to copy the file hash to your clipboard.
+File hashes are crucial for incident investigation and for verification of file integrity.
+The supported hash functions for case files are MD5, SHA-1, and SHA-256.
 
 [discrete]
-==== Supporting multi levels of term aggregations in {es} rule type 
-
-The existing {es} alerting rule is now supported by multiple selection when grouping by alert fields, which allows you to define multiple layers of term aggregations.
-
-[role="screenshot"]
-image::images/term-aggs.png[An example of creating multiple layers of term aggregations]
-
-[discrete]
-==== Slack connector - allow List
-
-The Slack connector supports a new allow list, so customers will be able to manage the available Slack channels within the alert actions. 
-
-[discrete]
-=== {observability}
-
-[discrete]
-==== {es} alerting rule now available in {observability}
-
-The {es} alerting rule is now available in {observability}. Before this update, {observability} customers who leverage the {es} rule were required to move between **{stack-manage-app}** and **{observability}** to manage their alerts.
-To manage the {observability} roles properly, we added a new field to determine which role can maintain the created rule and its generated alerts. 
-
-[role="screenshot"]
-image::images/alerting-rule.png[An example of creating an altering rule in {observability}]
-
-[discrete]
-=== Global Experience 
-
-[discrete]
-==== Create and manage {esql} enrichment index policies
-
-In support of our new {esql} capabilities, we have added link:{ref}/ingest-enriching-data.html[enrich policies] to our **Index Management** experience. Users can now create their enrichment policies right from here and get started using it right away. Once configured, all enrich policies are available in the **Enrich Policies** tab.
-
-Example of an ES|QL Query using an enrich policy with enrich:
-
-[role="screenshot"]
-image::images/esql-enrich.png[An example of an {esql} query using an enrich policy]
-
-Example of the enrich policies shown in the UI:
-
-[role="screenshot"]
-image::images/enrich-ui.png[An example of where to add an enrich policy in the UI]
-
-[role="screenshot"]
-image::images/enrich-index-management.png[An example of where an enrich policy lives in the UI]
+==== Connector improvements
+PagerDuty alert action is now supported by 2 new fields `links` and `custom_details`.
+ServiceNow ITSM alert action allows users to define incident resolution when alert is recovered to ensure bi-directional sync between the Elastic Alerts and ServiceNow Incidents.
 
 
-[discrete]
-==== Start Discover {esql} from global search
-
-**Discover** searches are powered by KQL by default. You can easily switch to {esql} mode within **Discover** and search your data with this new query language. You can also access {esql} in **Discover** from the global search bar within Elastic. With just a few keystrokes, simply type in “ESQL” and you can access **Discover** with {esql} enabled for you.
-
-[role="screenshot"]
-image::images/start-esql-in-discover.gif[An example of using ES|QL in Discover]
-
-[discrete]
-==== Access {es} connection details
-
-You can now view your {es} endpoint, link:https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html[Cloud ID], and even manage your API keys from many areas in Elastic such as integrations. You can also access the connection details on any page in Elastic using the help menu from the header bar.
-
-[role="screenshot"]
-image::images/endpoints.png[An example of where to find the endpoints in the UI, width=40%]
-
-[discrete]
-==== AWS CloudFormation template updates
-
-When subscribing to Elastic using the link:https://aws.amazon.com/marketplace/pp/prodview-voru33wi6xs7k[AWS Marketplace], users have the option to quickly get set up with an AWS CloudFormation Template. This step allows you to create an Elastic deployment in the AWS region of your choice. We’ve updated the AWS CloudFormation Template with bug fixes and stability improvements to better help you get started.