## Summary
This PR modifies the `recall` function from this:
- Grab user prompt and/or queries
- Pass input to Elasticsearch for semantic search using ELSER to find at
most 5 hits
- Return hits
to this:
- Grab user prompt and/or queries
- Pass input to Elasticsearch for semantic search using ELSER to find at most 20 hits
- Take hits and pass them to the LLM with a new system prompt asking it
to score the documents based on relevancy (by forcing it to call a
function)
- Sort the ELSER hits by LLM score and grab the top 5
### How to test
_It might be easier to test this once it has been merged with
https://github.com/elastic/kibana/pull/171933_
_Or simply edit
[this](2abfa54acc/x-pack/plugins/observability_ai_assistant/public/functions/recall.ts (L107))
line to inject your own list of documents_
- Put a large number of entries into the Knowledge base (ideally more
than 20)
- Ask the Assistant questions that relate to some of those entries more
than others
- Check that the initial recall HTTP request returns hits
- Check that the result of recall given to the LLM seem relevant
Fixes#171575
## Summary
Let's see.
1. Using `within` in '`should render the activity content and main
components'` improves the execution time.
2. Removed some `expect`s in other tests that were already tested
elsewhere.
## Summary
Closes https://github.com/elastic/kibana/issues/172061
Fix for update agent policy with optional secret.
Steps to verify:
- change a secret variable in a package to optional, used
`universal_profiling_agent` package
```
- name: profiler.secret_token
title: Authorization
description: Token used to authenticate the agent.
required: false
show_user: true
type: text
secret: true
```
- build a new package version and upload zip to fleet api locally
- add integration policy by keeping the secret_token empty
- edit the integration policy, and verify that the edit works
- there is no secret value saved in `.fleet-secrets`
<img width="1190" alt="image"
src="fd8e7a0d-199d-4dd3-bf2b-b887f48848a3">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## [Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts
This PR implements _Retrieval Augmented Generation_ (RAG) for Alerts in the Security Solution. This feature enables users to ask the assistant questions about the latest and riskiest open alerts in their environment using natural language, for example:
- _How many alerts are currently open?_
- _Which alerts should I look at first?_
- _Did we have any alerts with suspicious activity on Windows machines?_
### More context
Previously, the assistant relied solely on the knowledge of the configured LLM and _singular_ alerts or events passed _by the client_ to the LLM as prompt context. This new feature:
- Enables _multiple_ alerts to be passed by the _server_ as context to the LLM, via [LangChain tools](https://github.com/elastic/kibana/pull/167097)
- Applies the user's [anonymization](https://github.com/elastic/kibana/pull/159857) settings to those alerts
- Only fields allowed by the user will be sent as context to the LLM
- Users may enable or disable anonymization for specific fields (via settings)
- Click the conversation's `Show anonymized` toggle to see the anonymized values sent to / received from the LLM:
### Settings
This feature is enabled and configured via the `Knowledge Base` > `Alerts` settings in the screenshot below:
- The `Alerts` toggle enables or disables the feature
- The slider has a range of `10` - `100` alerts (default: `20`)
When the setting above is enabled, up to `n` alerts (as determined by the slider) that meet the following criteria will be returned:
- the `kibana.alert.workflow_status` must be `open`
- the alert must have been generated in the last `24 hours`
- the alert must NOT be a `kibana.alert.building_block_type` alert
- the `n` alerts are ordered by `kibana.alert.risk_score`, to prioritize the riskiest alerts
### Feature flag
To use this feature:
1) Add the `assistantRagOnAlerts` feature flag to the `xpack.securitySolution.enableExperimental` setting in `config/kibana.yml` (or `config/kibana.dev.yml` in local development environments), per the example below:
```
xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts']
```
2) Enable the `Alerts` toggle in the Assistant's `Knowledge Base` settings, per the screenshot below:
## How it works
- When the `Alerts` settings toggle is enabled, http `POST` requests to the `/internal/elastic_assistant/actions/connector/{id}/_execute` route include the following new (optional) parameters:
- `alertsIndexPattern`, the alerts index for the current Kibana Space, e.g. `.alerts-security.alerts-default`
- `allow`, the user's `Allowed` fields in the `Anonymization` settings, e.g. `["@timestamp", "cloud.availability_zone", "file.name", "user.name", ...]`
- `allowReplacement`, the user's `Anonymized` fields in the `Anonymization` settings, e.g. `["cloud.availability_zone", "host.name", "user.name", ...]`
- `replacements`, a `Record<string, string>` of replacements (generated on the server) that starts empty for a new conversation, and accumulates anonymized values until the conversation is cleared, e.g.
```json
"replacements": {
"e4f935c0-5a80-47b2-ac7f-816610790364": "Host-itk8qh4tjm",
"cf61f946-d643-4b15-899f-6ffe3fd36097": "rpwmjvuuia",
"7f80b092-fb1a-48a2-a634-3abc61b32157": "6astve9g6s",
"f979c0d5-db1b-4506-b425-500821d00813": "Host-odqbow6tmc",
// ...
},
```
- `size`, the numeric value set by the slider in the user's `Knowledge Base > Alerts` setting, e.g. `20`
- The `postActionsConnectorExecuteRoute` function in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts` was updated to accept the new optional parameters, and to return an updated `replacements` with every response. (Every new request that is processed on the server may add additional anonymized values to the `replacements` returned in the response.)
- The `callAgentExecutor` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts` previously used a hard-coded array of LangChain tools that had just one entry, for the `ESQLKnowledgeBaseTool` tool. That hard-coded array was replaced in this PR with a call to the (new) `getApplicableTools` function:
```typescript
const tools: Tool[] = getApplicableTools({
allow,
allowReplacement,
alertsIndexPattern,
assistantLangChain,
chain,
esClient,
modelExists,
onNewReplacements,
replacements,
request,
size,
});
```
- The `getApplicableTools` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/index.ts` examines the parameters in the `KibanaRequest` and only returns a filtered set of LangChain tools. If the request doesn't contain all the parameters required by a tool, it will NOT be returned by `getApplicableTools`. For example, if the required anonymization parameters are not included in the request, the `open-alerts` tool will not be returned.
- The new `alert-counts` LangChain tool returned by the `getAlertCountsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_tool.ts` provides the LLM the results of an aggregation on the last `24` hours of alerts (in the current Kibana Space), grouped by `kibana.alert.severity`. See the `getAlertsCountQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_query.ts` for details
- The new `open-alerts` LangChain tool returned by the `getOpenAlertsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_tool.ts` provides the LLM up to `size` non-building-block alerts generated in the last `24` hours (in the current Kibana Space) with an `open` workflow status, ordered by `kibana.alert.risk_score` to prioritize the riskiest alerts. See the `getOpenAlertsQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_query.ts` for details.
- On the client, a conversation continues to accumulate additional `replacements` (and send them in subsequent requests) until the conversation is cleared
- Anonymization functions that were only invoked by the browser were moved from the (browser) `kbn-elastic-assistant` package in `x-pack/packages/kbn-elastic-assistant/` to a new common package: `x-pack/packages/kbn-elastic-assistant-common`
- The new `kbn-elastic-assistant-common` package is also consumed by the `elastic_assistant` (server) plugin: `x-pack/plugins/elastic_assistant`
## Summary
This PR add the new timeline changes tour.Below is the demo.
@nastasha-solomon , could you please help check the copy and let me know
if its looks okay or it needs change.
Translation messages can be found in below files:
1.
`x-pack/plugins/security_solution/public/timelines/components/timeline/tour/step_config.tsx`
2.
`x-pack/plugins/security_solution/public/timelines/components/timeline/tour/translations.ts`
3ba1a984-e0b5-41c1-8c6e-3d35f50f7c66
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Steph Milovic <stephanie.milovic@elastic.co>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
## Summary
This PR adds a bunch of plugins to help manage AI Assistant Management
settings.
It offers a 'selection' plugin inside Stack Management where a user can
select which AI Assistant she wants to manage.
The Security team can hook into this one, so settings for both AI
Assistants can be accessed from inside one place inside Stack
Management.
This PR also adds the plugin to manage settings for the AI Assistant for
Observability, including Knowledge Base management. This plugin is
available both in Stack Management (stateful) and Project Settings
(serverless).
## What it looks like
51392ec5-05c9-4947-9bf2-810d8d0b7525
## Detailed
1. **Adds a Stack Management plugin**
(`/src/plugins/ai_assistant_management/selection`). Its primary function
is to render a selection screen to help users navigate to the settings
plugin for the AI Assistant for a specific solution. This plugin is
displayed in Stack Management, which is only available in stateful
versions of Kibana.
2. **Adds a AI Assistant for Observability Settings plugin**
(`/src/plugins/ai_assistant_management/observability`). This plugin
allows management of specific Observability AI Assistant settings. It is
available in stateful versions of Kibana (via the aforementioned Stack
Management plugin) or in serverless versions via Project Management.
3. **Knowledge Base management for Observability AI Assistant**: The AI
Assistant for Observability Settings plugin has a Knowledge Base tab,
which allows users to add / read / update / delete and bulk import
entries into the Knowledge Base of the Observability AI Assistant.
4. **Moving of KB endpoints in Observability AI Assistant plugin**: KB
endpoints and functions were located in the same folder. As this PR adds
new endpoints for the KB for CRUD operations, it also moves the existing
ones from the function folder into a dedicated one so there's a clearer
distinction between kb and functions.
5. **Adding of GenAI Connector inside Chat Flyout**: If the user has
admin rights, it is possible to set up a GenAI connector from within the
Observability AI Assistant Chat Flyout. This provides a faster and more
seamless onboarding experience. If the user does not, she will be
redirected to the Settings page.
## Bug fixes
* Fixes chat item styling issues (padding, background color).
## How to test
* Check if the Stack Management plugin works on stateful
* Check if the AI Assistant Settings plugin works on stateful +
serverless
* Check if CRUD operations on KB work
* Check if searching on KB entries work
* Check if its possible to navigate to KB tab directly
(`app/management/kibana/aiAssistantManagementObservability?tab=knowledge_base`)
## Todo
- [x] Add sorting to getEntries
- [x] Add params for tab routing
- [x] Add unit tests
- [ ] Add API tests
- [ ] Add fallback for already indexed entries when searching
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR updates the callouts in the Endpoint Policy where we mention the
detection rules associated with Endpoint. This makes it more current and
provides a docs link with more details.
- Banner is added to the top of the Policy page
- Individual banners are removed from the individual cards
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
1. **This PR adds sampled % of documents & cardinality for text fields
for Data visualizer/Field stats**. Previously, text fields do not show
any computed % or cardinality. This is because text fields are not
aggregatable in Elasticsearch.
This PR fetches a sample of 1000 documents in Elasticsearch, and compute
the approximate count % and cardinality based on that sample.
<img width="1480" alt="image"
src="8a4e5ddf-36b8-4ca2-90a2-f67ad4a7822c">
It also shows a tooltip message indicating that text fields are using a
much smaller sample:
<img width="1137" alt="image"
src="666e53be-19d8-4eaf-b946-997f3c30b33f">
2. **This PR also fixes an issue with the first bucket "missing" in the
doc count chart**. See https://github.com/elastic/kibana/issues/172355.
This happens if the time selected is slightly different from the first
timestamp. This PR changes so that we don't filter out that data if it's
partial.
For example, selecting date to be 18:04 will wipe out the first bucket
at 18:00.
Before:
<img width="1137" alt="image"
src="3dd4a2b7-84f6-40bb-aa77-a8eae14ba8bb">
After:
<img width="1137" alt="image"
src="00c29100-f90e-4477-9374-e1366cea5b7c">
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
This PR sets the "budget consumed" mode as the default mode for
configuring the SLO Burn Rate Rule. This PR also adds a time table to
help the user understand when they can expect their SLO to fire based on
the burn rate windows and sample error rates.
<img width="549" alt="image"
src="01035b5d-60b9-40c8-9034-7ecd6a904bd5">
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
b749781c-1941-40d5-8dc8-094659fba9e5
<img width="674" alt="Screenshot 2023-12-05 at 10 31 10"
src="dc0dc39e-6ac1-47e6-b608-ec6667be251b">
* Remove the `firstLastSeen` call from managed user data because it can
be inferred from the event timestamp.
* Updated the managed data API to return Okta and Azure data.
* Create a Flyout asset document details panel
* Create a cell action that add/remove fields from the asset table
* Persist selected field on the redux store
* Persist the selected fields on local storage
* [] TODO update query match to use e-mail field
### How to test it?
* Enable the experimental flag
`xpack.securitySolution.enableExperimental: ['newUserDetailsFlyout']`
* Start an elastic cluster with fleet and elastic agent
* Follow this steps to setup a cluster with elastic-package
* Install https://github.com/elastic/elastic-package
* `elastic-package stack up -vd --version 8.12.0-SNAPSHOT`
* connect your local kibana instance to the cluster ES instance
* configured `server.port`, `elasticsearch.hosts`,
`elasticsearch.ssl.certificateAuthorities`,
`elasticsearch.serviceAccountToken`
* Install Okta and Entra integrations (ask @machadoum for credentials)
* Create a rule that generates alerts for every event
* Go to the alerts table and click on the username to open the flyout
**Tip:** You can open your Docker application and explore the files to
copy the token and the certificate.
<img width="1420" alt="Screenshot 2023-12-04 at 16 29 53"
src="60032e34-6f50-4316-ad88-2a13109a5622">
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
This PR populates the existing `kibana.alert.workflow_user` field in the
alerts-as-data mappings with the `profile_uid` of the last user to
modify the status of the alert. It also adds a new field,
`kibana.alert.workflow_status_updated_at`, to track the last time the
workflow status was updated and populates it with a timestamp.
Similar to the alert assignment PR, `workflow_user` renders in the table
with a user avatar instead of the raw `profile_uid` value stored in the
alert. The filter in/out buttons on the row cell automatically add a
filter that uses the raw value so that filtering works correctly.
Due to limitations of Kibana's user profile implementation,
`workflow_user` is only populated if a user changes the alert status
using the alert status route (`POST
/api/detection_engine/signals/status`) within an interactive session,
i.e. logs in rather than passes credentials with each API request
([related issue](https://github.com/elastic/kibana/issues/167459)).
## Alerts table
## Alert details
### Checklist
- [ ] Functional changes are hidden behind a feature flag. If not
hidden, the PR explains why these changes are being implemented in a
long-living feature branch.
- [x] Functional changes are covered with a test plan and automated
tests.
- [x] Stability of new and changed tests is verified using the [Flaky
Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner).
- Flaky test run:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/4130
- [ ] Comprehensive manual testing is done by two engineers: the PR
author and one of the PR reviewers. Changes are tested in both ESS and
Serverless.
- [x] Mapping changes are accompanied by a technical design document. It
can be a GitHub issue or an RFC explaining the changes. The design
document is shared with and approved by the appropriate teams and
individual stakeholders.
- https://github.com/elastic/security-team/issues/4820
- [x] Functional changes are communicated to the Docs team. A ticket or
PR is opened in https://github.com/elastic/security-docs. The following
information is included: any feature flags used, affected environments
(Serverless, ESS, or both).
- https://github.com/elastic/security-docs/issues/4325
## Summary
Closes#136391
Uses a description of the anomaly for the alert message for anomaly
detection alerting rules with the `record` result type. This messages is
used for example in the `Reason` field in the alert table and details
flyout.
<img width="753" alt="image"
src="072fe833-204b-4d38-bd3d-50d00015a43f">
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
This adds an API key panel to the Search Hub.
8d0458bd-26e1-40c4-9643-1a91f3924911
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
It sets the `perPage` value to 1000 (overrides the default one set to
`20`) in order to display more policies in Artifacts form and also in
the policy filter for artifacts list.
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
## Summary
This PR re-opens https://github.com/elastic/kibana/pull/171679 which had
to be reverted due to CI instability at the time it was originally
mergted.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Add a mute/unmute action component to be used in alerts table row
actions, move all common alerts table row actions to triggersActionsUI
and re-export them for consumers to compose their table actions.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
---------
Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>
Co-authored-by: Maryam Saeidi <maryam.saeidi@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/172500
PR resolves the issue by not re-throwing errors in GeoJsonVectorLayer
and MvtVectorLayer syncData methods. Error state is stored in data
request and displayed to users. Errors are thrown to stop the flow of
execution for syncData.
Adds the ability to run pattern analysis on the index used for an
anomaly detection job.
The pattern analysis is run for the whole time range selected in the
page's time picker, but also adds a sub aggregation to focus on the
bucket time range for the selected anomaly.
This allows the user to view either patterns found in the bucket or the
full time range.
If partition field has been used in the detector, this is also added as
a filter in the sub agg, so the patterns shown are only ones where the
doc also matches the partition field value.
A sub agg was used rather than just running the whole analysis on the
bucket to ensure we get a good analysis and good patterns. The more data
we see, the more accurate the patterns.
This way we can find all of the patterns and then find which of those
patterns match the bucket.
The pattern analysis action item is added to the menu if the data view
has a `message` or `error.message` field.
3e8295a0-5c7e-4ba6-b260-13c158d32a29
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR fixes an issue with data drift not showing numeric fields
correctly for datasets that automatically uses random sampling.
<img width="1720" alt="image"
src="65e5efce-e428-40fc-8b53-c373c5771dd0">
This PR:
* extracts `enterprise_search` index documents component to common
package.
* Uses EUI pagination, converting Elasticsearch`from` & `size` to EUI
pagination standard for documents list
### Screen Recording
f585d9cc-f92c-44f4-aead-23c75c107a0f
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Adds optional `minBarHeight` option to the Lens embeddable args, defaults to `1px`. Adds back min bar height of `2px` in discover histogram for large document count variances per https://github.com/elastic/kibana/issues/68436.
## Summary
Closes https://github.com/elastic/kibana/issues/172546
### Testing
1. Add the following to your Kibana config file and start Kibana:
```yml
xpack.fleet.outputs:
- id: my-remote-es-output-with-a-secret
name: preconfigured remote elasticsearch output with a secret
type: remote_elasticsearch
hosts: ['http://localhost:9997']
secrets:
service_token: secretRemoteESServiceToken
```
2. Head to Fleet UI and check that the output was correctly created.
3. Go to the Dev Tools Console and query Fleet secrets: `GET
.fleet-secrets/_search`. Check that the service token secret was
created.
4. Edit the output configuration, e.g. the value of the service token,
and wait for Kibana to restart.
5. Check that the value(s) have been correctly updated.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/169523
The `categorize_text` agg uses the `ml_standard` tokenizer by default
which produces slightly different tokens compared to the `standard`
tokenizer, which is the default used for search.
This means the category key (which is comprised of these tokens) will
occasionally not match any documents when it is used as a filter in
Discover to find docs in a category.
This PR ensures the `standard` tokenizer is always used in the pattern
analysis query.
A future enhancement would be to check which analyzer is specified in
the mappings for the source field and to use that instead of
unconditionally using `standard`. However for an initial fix, using the
`standard` analyzer will be more likely to match the results from the
majority of searches.
## Summary
Relates https://github.com/elastic/fleet-server/issues/3116
Relates https://github.com/elastic/kibana/issues/104986
Reading latest output health state from
`logs-fleet_server.output_health-default` data stream by output id, and
displaying error state on UI - Edit Output flyout.
Steps to verify:
- enable feature flag `remoteESOutput`
- add `remote_elasticsearch` output, can be a non-existent host for this
test
- add the output as monitoring output of an agent policy
- run fleet-server with the changes
[here](https://github.com/elastic/fleet-server/issues/3116)
- enroll an agent
- wait until fleet-server starts reporting degraded state in the output
health data stream
- open edit output flyout on UI and verify that the error state is
visible
- when the connection is back again (update host to a valid one, or
remote es was temporarily down), the error state goes away
<img width="568" alt="image"
src="46d0cf95-6aa4-4f7c-8608-4362ada4eb6c">
The UI was suggested in the design doc:
https://docs.google.com/document/d/19D0bX7oURf0yms4qemfqDyisw_IYB-OVw4oU-t4lf18/edit#bookmark=id.595r8l91kaq8
### Notes/suggestions:
- We might want to add the output state to the output list as well
(maybe as badges like agent health?) as it's not too visible in the
flyout (have to scroll down).
- Also the error state will be reported earliest when an agent is
enrolled and fleet-server can't create api key, so not immediately when
the output is added. It would be good to show the time of the last state
(e.g. how we display on agents last checkin x minutes ago)
- I think it would be beneficial to display the healthy state too.
Added badges to output list:
<img width="1233" alt="image"
src="07ff06ec-b778-4420-975b-b46a0a18c7cc">
Added healthy state UI to Edit output:
<img width="627" alt="image"
src="4222d849-c957-41d7-9606-b58493264115">
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Closes#172046
## Summary
This PR adds a control_title component inside infra controls usage to
give us the flexibility to use help messages in the controls.
### Solution
Inside the control_title component, there is a definition of
`helpMessages` used to add a help message in the infra control title
matched by the control id. The message could be a string or string &
link:
- "Message"
<img width="428" alt="image"
src="5199bfb5-c676-42ae-85cd-78d1e26a6adc">
- "Message" + Link
<img width="408" alt="image"
src="d15a838c-ba59-4f10-82c6-0bedab33feb3">
## Testing
- Go to Infrastructure -> Hosts
- Syntrace can be used to get some services, for example, use: `node
scripts/synthtrace --clean simple_trace.ts`
- Check the service name control and interact with it
- Check help message
- Filter by service
e6aed1ca-29f1-47c8-9589-bea26a791598
Closes#158256, #155394
## Summary
- Hides the Logs tab in the Stack Management > Rules page when the user
lacks the necessary permissions to avoid error messages (as shown in
https://github.com/elastic/kibana/issues/158256)
- Switches old `useLoadRuleTypes` hook usages to the new
`useLoadRuleTypesQuery` hook
- Assigns a staleTime to the rule types query to avoid duplicated
requests (as shown in https://github.com/elastic/kibana/issues/155394)
---------
Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>
The user may select a time range that falls into a warm storage tier.
When a request hits the warm tier, it may run into timeouts. To give
some feedback to the user that this may be happening, display a toast
notification letting the user know. The toast will show up after `15s`
for any query that is still `Loading`.
<img width="926" alt="Screenshot 2023-12-04 at 10 39 17"
src="e4a51576-92a3-4edd-8145-feee574397de">
Update routes timeout to 5 min.
## How to test it:
Select a time range greater than 30d.
## Summary
[Test
env](https://p.elstc.co/paste/Y83spa-G#0BRN87aMZxLJfbVok3W0U-7D/sss9OGfH9IIqw4oHL6)
1. When landing on the page the first time, all the tasks are collapsed.
If visited before, it keeps the last expanded task.
2. When clicking on the task, the url has the task id appended as
`#{taskId}`
3. When visiting the page, if url has `#{taskId}`, the relevant task
should be expanded. If no `#{taskId}` in the url, it expands the last
expanded task according to **local storage**.
4. Tasks completion are checked automatically, users are not able to
undo any tasks unless the **local storage** is cleanned.
5. Task completion criteria:
https://github.com/elastic/security-team/issues/8032 `Onboarding tasks
with success criteria`:
- [x] "create first project" -> ~unexpandable~, already complete when
user arrives
- [x] "watch overview video" -> user clicks to expand (success == users
opens section/clicks "Start")
- [x] "add integrations" -> users clicks to expand and goes to
integration page (success == query fleet to confirm an agent exists with
an integration) `Update we use **indicesExist** from sourcerer to
replace fleet api as some performance issue found when running locally,
not sure how it will affect the production.`
- [x] - after user clicks "Start" and expands, if success criteria is
already met (e.g. agent is installed with an integration), notify user
agent is installed and mark step as complete.
- [x] "view and analyze dashboards" -> users clicks to expand (success
== click action)
- [x] "enable prebuilt rules" -> clicks to expand (success == at least
one rule **enabled**, show enabled rules like integrations above)
- [x] "view alerts" -> user clicks to expand (success == click action)
6. Design:
https://github.com/elastic/kibana/pull/171078#issuecomment-1828562066https://github.com/elastic/kibana/issues/170643
<img width="2556" alt="Screenshot 2023-12-04 at 16 47 48"
src="c98eb2f5-6b31-43f9-9db5-0829e78bb3c2">
<img width="2548" alt="Screenshot 2023-12-04 at 16 29 50"
src="41cb329c-417f-4b95-b19b-67663380a9a3">
<img width="2546" alt="Screenshot 2023-12-04 at 16 30 47"
src="daa3cf3b-9d2e-4c69-83f1-fcc4817f6b8f">
- Integration added:
1f9aefe4-c20b-4d46-b8b0-1aabf8bd7091
- Integration not added:
8b0d6c6b-0bae-4857-aeb1-715f9f4080b8
29432bfe-f270-4e5e-a1c9-86ad806ea5bb
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Sergi Massaneda <sergi.massaneda@gmail.com>
The empty state will show up when `has_setup` from the Profiling Status
API returns `false`.
<img width="1276" alt="Screenshot 2023-11-30 at 15 06 30"
src="97a313be-db4f-4a5a-af36-df574f9793d5">
<img width="1036" alt="Screenshot 2023-11-30 at 14 47 48"
src="2622cad6-6763-4abc-9469-fa292137efda">
