[DOCS] Adds the whats new page for 8.14 (#183556)

docs/user/whats-new.asciidoc

@@ -1,253 +1,156 @@
 [[whats-new]]
-== What's new in 8.13
+== What's new in 8.14
 
-Here are the highlights of what's new and improved in 8.13.
+Here are the highlights of what's new and improved in 8.14.
 For detailed information about this release,
 check the <<release-notes, release notes>>.
 
-Previous versions: {kibana-ref-all}/8.12/whats-new.html[8.12] | {kibana-ref-all}/8.11/whats-new.html[8.11] | {kibana-ref-all}/8.10/whats-new.html[8.10] | {kibana-ref-all}/8.9/whats-new.html[8.9] | {kibana-ref-all}/8.8/whats-new.html[8.8] | {kibana-ref-all}/8.7/whats-new.html[8.7] | {kibana-ref-all}/8.6/whats-new.html[8.6] | {kibana-ref-all}/8.5/whats-new.html[8.5] | {kibana-ref-all}/8.4/whats-new.html[8.4] | {kibana-ref-all}/8.3/whats-new.html[8.3] | {kibana-ref-all}/8.2/whats-new.html[8.2] 
-| {kibana-ref-all}/8.1/whats-new.html[8.1] | {kibana-ref-all}/8.0/whats-new.html[8.0]
+Previous versions: {kibana-ref-all}/8.13/whats-new.html[8.13] | {kibana-ref-all}/8.12/whats-new.html[8.12] | {kibana-ref-all}/8.11/whats-new.html[8.11] | {kibana-ref-all}/8.10/whats-new.html[8.10] | {kibana-ref-all}/8.9/whats-new.html[8.9] | {kibana-ref-all}/8.8/whats-new.html[8.8] | {kibana-ref-all}/8.7/whats-new.html[8.7] | {kibana-ref-all}/8.6/whats-new.html[8.6] | {kibana-ref-all}/8.5/whats-new.html[8.5] | {kibana-ref-all}/8.4/whats-new.html[8.4] | {kibana-ref-all}/8.3/whats-new.html[8.3] | {kibana-ref-all}/8.2/whats-new.html[8.2] | {kibana-ref-all}/8.1/whats-new.html[8.1] | {kibana-ref-all}/8.0/whats-new.html[8.0]
 
 [discrete]
 === Discover
 
 [discrete]
-==== Empty fields improvements in Discover, Lens & ES|QL 
+==== Transitioning {esql} from Tech Preview to General Availability!
 
-Navigating through extensive data can often lead users to encounter numerous *Empty fields* within their field lists, making it challenging to identify and access valuable data efficiently. To address this issue, we’ve focused our efforts on enhancing the *Empty fields* category. Our solution simplifies the process of finding and accessing fields that contain meaningful data, ensuring a more streamlined and productive data exploration experience for our users. 
-
-In some cases this can have a huge impact on field lists that target indices with many fields, such as in the below example where the field list was reduced from 5,492 fields down to only 238.
-
-[role="screenshot"]
-image::images/empty-fields.png[An image of the available fields before the improvements.]
-
-[role="screenshot"]
-image::images/empty-fields-results.png[An image of the improvements to the available fields.]
+{esql} offers a streamlined way to filter, transform, and analyze data in {es}. Its intuitive design, utilizing "pipes" (|) for step-by-step data exploration, enables you to easily compose powerful queries for detailed analysis. Whether you're a developer, SRE, or Security Analyst, {esql} empowers you to uncover specific events, perform robust statistical analyses, and create compelling visualizations. As we move from tech preview to general availability, discover the enhanced capabilities of {esql} and elevate your data operations.
 
 [discrete]
-==== Discover enhancements
+==== {esql} Query History
 
-Improving the experience in Discover when users are exploring their data helps them find insights quickly. In 8.13 we added the *Auto Interval* to the Histogram, which allows users to quickly select a *Time Interval*. We also enhanced the UI by moving the count of documents to the table and new panel toggle buttons for toggling fields sidebar and histogram.
+We've enhanced the {esql} editor to improve your workflow. You can now view and re-run your last 20 {esql} queries directly within Discover, {esql} charts, alerts, and maps. This feature makes it easier to manage your queries and streamline your tasks. 
 
 [role="screenshot"]
-image::images/auto-interval.png[An image of the new auto interval option, width=60%]
+image::images/query-history-in-discover.png[An image of an ES|QL query history in Discover.]
 
 [role="screenshot"]
-image::images/auto-interval-1.png[An image of the auto option.]
+image::images/query-history-in-dashboard.png[An image of the auto option.]
 
 [discrete]
-==== Cancellation of the {esql} long-running query
+==== Document comparison mode in Discover & {esql}
 
-You can now cancel a long-running {esql} query from the UI.
+You can now select and compare documents or fields. This functionality streamlines analysis and troubleshooting tasks by allowing you to perform detailed comparisons, such as diffing SIP messages of a certain ID across multiple documents in {es}.
+
+[discrete]
+==== Storing {esql} visualizations in saved searches
+
+Changes to {esql} charts in Discover can now be saved along with {esql} query syntax, allowing you to share and manage your Discover views with ease. 
 
 [role="screenshot"]
-image::images/cancel-button.png[]
-
+image::images/esql-viz-saved-search.png[An image of {esql} visualization in saved search.]
 
 [discrete]
-==== Better validation and autocomplete when writing ES|QL queries
+==== {esql} field statistics in Unified Field List
 
-Autocomplete and validation are important tools to ensure the user’s ES|QL is correct and can be executed without errors. With these improvements, we are speeding up their workflow and increasing the quality of the improvements.
+Seeing data statistics while crafting queries can be useful to understand the data batter. To enhance this experience, we've added field stats to the sidebar field popover in {esql} mode, similar to what's available in Discover data view mode.
 
 [role="screenshot"]
-image::images/esql-validation.png[An image of {esql} validation in action.]
+image::images/esql-field-stats.png[An image of the new field stats sidebar popover.]
 
 [discrete]
-==== Color all terms in new color mapping
+==== Custom data view field descriptions
 
-In our latest update, based on early feedback, we've enhanced the user experience with our new color mapping feature introducing a more intuitive default option. Now users can either assign a single color or enable color looping for unassigned terms, providing greater flexibility and a cleaner interface for term assignments. Additionally we've eliminated the maximum limit on assignments, allowing for more comprehensive customization. It's important to note that looping colors or using more than 10 colors can be effective in certain scenarios. However, we recommend limiting the number of colors used to prevent potential misinterpretation of your charts. This update aims to make color mapping more user-friendly and adaptable to your needs.
+We've implemented a much-requested feature that allows you to add custom descriptions to data view fields. This enhancement is going to make a significant difference in how you manage and understand your data in Kibana. You'll see these descriptions in the Unified Field List popover in Discover and Lens, enhancing your user experience. 
 
 [role="screenshot"]
-image::images/color-mapping-enhanced.png[An image of the color mapping feature applied to a vertical bar chart.]
+image::images/custom-descriptions.png[An image of a custom data view field description.]
+
+[discrete]
+=== Dashboard
+
+[discrete]
+==== Links panel Generally Available 
+
+You can now easily navigate from one dashboard to another using the links panel. Better organize your dashboards and make them more performant by chunking them in multiple dashboards with fewer visualizations and linking them together. You can carry over your filters, query and time range when navigating to other related dashboards. Display your links horizontally or vertically as it better suits your dashboard layout. You can also use the links panel to include external links in your dashboards. For example, to your wiki page or other applications. Decide whether you want to open the links in the same browser tab or in a new one.
 
 [role="screenshot"]
-image::images/color-mapping-1.png[An image of the color palette options, width=60%]
+image::images/links-panel.gif[A gif of the links panel in action.]
 
 [discrete]
-==== Visualizing an {esql} query in Observability AI assistant
+==== Controls apply button
 
-The {observability} AI assistant in 8.13 comes with great improvements in query generation and performance. Users can now visualize the generated {esql} queries, edit them using the inline editing flyout, and embed them in a dashboard.
+Controls are a popular way for users to filter their dashboards. Most of the time, users need to filter several of these controls to get the results they are looking for. In order to optimize performance and minimize the number of queries sent to get the data, we added the option to add a button so the controls selection will not be applied until the user clicks on it. That allows users to filter by multiple fields before sending any new requests to fetch the dashboard data.
 
 [role="screenshot"]
-image::https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltd6737b8e5633e948/viz-esql.gif[A gif of an {esql} query in the AI assistant.]
+image::images/controls-apply-button.png[An image of the new controls apply button in the UI.]
 
-[discrete]
-==== Cross-cluster search support in {esql} and in {kib}
-
-Cross-cluster searches are now supported in {esql} and in {kib} the feature has been introduced for both validation and autocomplete. The autocomplete feature will show some documentation about the specific settings when navigating the suggestions.
+This option is off by default and dashboard authors can decide to enable this button by switching off “Apply selections automatically” from the Controls settings.
 
 [role="screenshot"]
-image::https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt4b3db027b6c7951a/ccs-esql-queries.gif[A gif of an {esql} cross cluster query.]
+image::images/control-settings.png[An image of the control settings in the UI.]
 
 [discrete]
-==== Quick fix to help users write {esql}
+==== Gauge chart
 
-Quick fix helps users when they have misspelled a field, index, or policy name (maybe pasting a query from somewhere else) in {esql} mode.
+Users can now easily build beautiful and fully functional gauge charts from Lens. The revamped gauges will adapt to the field selected suggesting automatic minimum, maximum and goal values. The gauge charts in Lens will allow users to migrate from TSVB more easily. Users can select five different types of gauge charts: semicircular, circular, arc, vertical or horizontal. 
 
 [role="screenshot"]
-image::https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt463edc257418dd71/esql-quickfix[A gif showing the quick fix feature in action.]
+image::images/gauge-chart.png[An image of the new gauge chats.]
 
 [discrete]
-==== Create {esql} charts directly from a dashboard
+==== Region map goes GA
 
-Now you can create {esql} charts directly from a dashboard, without the need to go through Discover. Previously, to add a chart to your dashboard, you had to first create it in Discover using {esql} and then save it to the dashboard. This update streamlines the process, allowing you to instantly add {esql} charts right from your dashboard! 
+Users don’t need to navigate the complexity of the Maps app (meant to be used by more advanced geo users) to build a simple map. They can easily do it now from the Lens editor.
 
 [role="screenshot"]
-image::images/esql-charts-1.png[]
+image::images/region-map.png[An image of the region map.]
+
+[discrete]
+=== Machine Learning
+
+[discrete]
+==== Cohere reranking
+
+In 8.14, we have further enhanced Elastic’s inference API by adding support for Cohere’s foundation link:https://cohere.com/blog/rerank-3[Rerank 3 model] rerank-english-v.3.0. This is a natural next step following support for Cohere’s embeddings in 8.13. Elastic is the only vector database that supports Cohere Rerank 3. 
+
+Reranking adds semantic precision on your search results, bumping the most relevant ones to the top of the list. Reranking is two-stage (or multi-stage) by nature and this is very powerful as it gives you flexibility, depending on your use case: You can easily combine it as an additional step without making changes to your current approach. Whether this is vector search, BM25 or hybrid search, reranking will deliver semantic relevance gains at the top of your search results list.
+
+This is particularly important for Retrieval Augmented Generation (RAG), given the LLM’s costs and context windows limits. 
+
+[discrete]
+==== Inference API: RBAC 
+
+Use the new `inference_user` and `inference_admin` built-in roles to easily manage authorization for the inference API and `trained_models` API. The roles include the `manage_inference` and `monitor_inference` privileges which give full access and read access respectively to the inference endpoints.
+
+[discrete]
+==== AIOps: Log Pattern Analysis is Generally Available
+
+In 8.14 Log Pattern Analysis becomes GA. Log Pattern Analysis enables faster and smarter investigation across thousands of log messages in order to analyze, troubleshoot and identify the root cause of an incident. Combine it with Anomaly Detection and our other AIOps features to drastically reduce the MTTR.
+
+[discrete]
+==== Query history in {esql} data visualizer
+
+We have enhanced the {esql} editor to improve usability and support your workflows. You can now view and re-run your last 20 {esql} queries directly within the {esql} Data Visualizer.
 
 [role="screenshot"]
-image::https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt517f402fdeddd49f/ESQLcharts.gif[A gif of editing an {esql} query in the dashboard.]
+image::images/esql-data-viz.png[An image of data visualizer for ES|QL.]
 
 [discrete]
-==== {esql} in Maps
+=== Alerting
 
-You can now create a new documents layer in map using our recently launched https://www.elastic.co/blog/esql-elasticsearch-piped-query-language[ElasticSearch Query Language (ES|QL)]. You can query your data directly from Elasticsearch and leverage the benefits that {esql} brings such as speed and flexible data transformations to your maps. 
+[discrete]
+==== Kibana case actions
+
+Alerting rules now support a new action that enables you to create cases automatically when alerts are detected.
+The case action can aggregate alerts and group them by any alert field and time window.
+For example, you can specify that all alerts that are detected by a certain alerting rule in a given time window (for example 7 days) with the same user will be assigned automatically to the same case. For more information, check out <<cases-action-type>>.
 
 [role="screenshot"]
-image::images/esql-maps.png[An image of the {esql} option for maps.]
+image::images/case-action.gif[A gif showing the new case action.]
+
+[discrete]
+==== {stack-manage-app} Alerts page 
+
+A new alerts page is now available to manage alerts as part of the *{stack-manage-app}* menu. The new page enables you to filter alerts by rule type and solution and get a unified view of the alerts that you have authority to view within the space.
 
 [role="screenshot"]
-image::images/esql-maps-example.png[An image of {esql} applied to a map.]
+image::images/alerts.gif[A looping gif of the new alerts page.]
 
 [discrete]
-==== Controls configuration
+==== Jira additional fields support
 
-We added some improvements to controls for you to easily filter and interact with your dashboards.
-
-* You will be able to decide whether you want the global filters and time range to be applied to your controls narrowing down the available options or whether you prefer to display all possible values without considering them. You will find these options in the Controls settings.
+With this Jira connector enhancement, alerts can create enriched Jira issues with a more comprehensive structure that matches the Jira ticket scheme.
+A new JSON field is now supported as part of the Jira action so you can define any field to be sent from Kibana alerts to Jira tickets. For more information, check out <<jira-action-type>>.
 
 [role="screenshot"]
-image::images/controls-config.png[An image of the filtering control settings.]
-
-* If you have numeric fields displayed as range slider controls in your dashboard, you will now be able to decide what is the step that you want to be displayed between your values. 
-
-[role="screenshot"]
-image::images/controls-edit.png[An image of the range slider settings.]
-
-[discrete]
-=== ResponseOps
-
-[discrete]
-==== Alert delay
-
-In order to reduce noise for alerting rules with low sensitivity and ensure created alerts will be actionable and reasonable, we want to allow users to define how many rule runs should match before creating the alert. For example, "Generate the alert after 3 threshold matches in a row".
-
-[role="screenshot"]
-image::images/alert-delay.png[]
-
-[discrete]
-==== Slack action message templating using Slack Block Kit
-
-By supporting the link:https://api.slack.com/reference/surfaces/formatting#rich-layouts[Slack Block Kit] with our {kibana-ref}/slack-action-type.html[Slack connector (Web API)], we unlock new message templates to allow users to enrich and format the messages that are sent to Slack channels. Read more about link:https://app.slack.com/block-kit-builder/T0CUZ52US#%7B%22blocks%22:%5B%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22You%20have%20a%20new%20request:%5Cn*%3Cgoogle.com%7CFred%20Enriquez%20-%20Time%20Off%20request%3E*%22%7D%7D,%7B%22type%22:%22section%22,%22text%22:%7B%22type%22:%22mrkdwn%22,%22text%22:%22*Type:*%5CnPaid%20time%20off%5Cn*When:*%5CnAug%2010-Aug%2013%5Cn*Hours:*%2016.0%20(2%20days)%5Cn*Remaining%20balance:*%2032.0%20hours%20(4%20days)%5Cn*Comments:*%20%5C%22Family%20in%20town,%20going%20camping!%5C%22%22%7D,%22accessory%22:%7B%22type%22:%22image%22,%22image_url%22:%22https://api.slack.com/img/blocks/bkb_template_images/approvalsNewDevice.png%22,%22alt_text%22:%22computer%20thumbnail%22%7D%7D,%7B%22type%22:%22actions%22,%22elements%22:%5B%7B%22type%22:%22button%22,%22text%22:%7B%22type%22:%22plain_text%22,%22emoji%22:true,%22text%22:%22Approve%22%7D,%22style%22:%22primary%22,%22value%22:%22click_me_123%22%7D,%7B%22type%22:%22button%22,%22text%22:%7B%22type%22:%22plain_text%22,%22emoji%22:true,%22text%22:%22Deny%22%7D,%22style%22:%22danger%22,%22value%22:%22click_me_123%22%7D%5D%7D%5D%7D[templates options with Slack Block Kit].
-
-[role="screenshot"]
-image::images/slack-api.png[An image of the Slack web api connector, width =60%]
-
-[role="screenshot"]
-image::images/slack-block.png[An image of the block kit builder.]
-
-[discrete]
-=== Machine Learning 
-
-[discrete]
-==== Unified inference API now integrates Cohere embeddings
-
-We continue enhancing Elastic's unified inference API which supports both internal and external models for seamless easy integration with the LLM ecosystem.
-
-In 8.13 we add support for Cohere embeddings. This enhances our offering which supports OpenAI and HuggingFace embeddings since 8.12.
-
-The power of the inference API lies in its simple, unified syntax that abstracts away the underlying complexity of using different internal and external models. 
-
-So, in 8.13 we also added support for inference against the E5 multilingual embeddings that were offered through the Trained Models UI since 8.12.
-
-As a reminder, performing inference on the newly supported models and services is as simple as a call with the simple syntax introduced in 8.11:
-
-[source,bash]
-----
-PUT /_inference/<task_type>/<model_id>
-----
-
-To start using Cohere embeddings with your Elastic deployment using the new inference API, https://www.elastic.co/guide/en/elasticsearch/reference/current/semantic-search-inference.html[please see this tutorial]. This functionality is in Technical Preview in 8.13.
-
-[discrete]
-==== {esql} support in the Data Visualizer
-
-preview:[] The Data Visualizer now supports {esql}, Elastic’s new piped query language that simplifies data investigation. Run your {esql} queries in the Data Visualizer to easily explore your datasets. Choose to explore and apply your query to the entire dataset or a subset of it for speed. In 8.13 this functionality is in Technical Preview and supports keyword, text, numeric, boolean, date, and ip fields.
-
-[role="screenshot"]
-image::images/data-visualizer.png[An image of the new Data Visualizer for {esql}.]
-
-[discrete]
-==== Embed Anomaly Detection Single Metric Viewer in Dashboards
-
-You can now easily add single metric anomaly detection charts to dashboards. Under the *Add panel* option in Dashboard’s edit mode, select Machine Learning and then the *Single metric viewer* option from the menu.
-
-[role="screenshot"]
-image::images/embed-detection.png[An image of the single metric viewer.]
-
-[discrete]
-==== AIOps: Usability enhancements 
-
-We have enhanced Pattern Analysis in AIOps so that you can expand a row and see the tokens, the regex and a few examples that give you a better sense of the pattern. In addition the syntax highlighting (font color) reflects the detected pattern. When you choose to filter a pattern in Discover’s main view, the highlighting is now consistent between the Pattern Analysis feature and Discover. 
-
-[role="screenshot"]
-image::images/usability.png[An image of expanded rows.]
-
-You can now run Log Rate Analysis from the Anomaly Explorer and the Single Metric Viewer. Click on the *Actions* cog and select *Run log rate analysis* from the menu. You will be directed to the Log Rate analysis UI in Machine Learning. 
-
-[role="screenshot"]
-image::images/usability-1.png[An image of the actions menu with an arrow to run log rate analysis.]
-
-From 8.13, you can achieve the same from the anomaly markers in the Single Metric Viewer. Click on them and the actions menu will appear.
-
-[role="screenshot"]
-image::images/single-metric-viewer.png[An image of the single metric viewer.]
-
-[discrete]
-==== Grok highlighting in the File Data Visualizer
-
-Uploading a file through the File Data Visualizer will display the first 5 lines with inline highlighting. Hovering the mouse over displays a tooltip with the field name and type. 
-
-[role="screenshot"]
-image::images/grok-highlighting.png[An image of grok highlighting.]
-
-[discrete]
-=== Security
-
-[discrete]
-==== UI improvements for managing API keys
-
-We’ve made some improvements to the API keys management page. Going forward, you can now easily sort columns in the API keys table, making it easier for you to navigate through API keys, especially when volume is high.
-
-We’ve also made the API key name a required field. This reduces previous bugs on this page that resulted in errors or inconsistent API key names being displayed.
-
-[discrete]
-=== Global Experience
-
-[discrete]
-==== Discover using {esql} without data views
-
-You can now utilize {esql} without creating a data view. To expand, if you have ingested data but have not created a data view, when you navigate to *Discover* you will see a screen prompting you to create a data view, as before. Now an option is available on that screen to *Try ES|QL*. 
-
-[discrete]
-==== Improved UX for Setup Guides
-
-The setup guides you see in Kibana are now organized by solution, making it much easier to find the guide you’re looking for. Additionally, a new card was added to *Connect to the Elasticsearch API* so users with API use cases don’t have to dig through the navigation and documentation to find the connection info.
-
-[role="screenshot"]
-image::images/set-up-improvements.png[A screenshot of the home page cards.]
-
-[discrete]
-==== Live chat available from the Elastic Cloud console
-
-You will now find live chat functionality available in the top right corner in the Elastic Cloud console, completing seamlessly availability from all locations within both Elastic Cloud and Kibana experiences. 
-
-[discrete]
-==== Faster deployment creation times
-
-In particular regions, deployment of a new cluster is nearly instantaneous, saving you about 3-5 minutes of waiting time to get started. The number of regions that provide this performance boost has increased by 5x as we strive to make this the standard experience. 
-
-
-
-
-
+image::images/jira-connector.png[An image of the new jira connector panel, width =60%]