## [Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts
This PR implements _Retrieval Augmented Generation_ (RAG) for Alerts in the Security Solution. This feature enables users to ask the assistant questions about the latest and riskiest open alerts in their environment using natural language, for example:
- _How many alerts are currently open?_
- _Which alerts should I look at first?_
- _Did we have any alerts with suspicious activity on Windows machines?_
### More context
Previously, the assistant relied solely on the knowledge of the configured LLM and _singular_ alerts or events passed _by the client_ to the LLM as prompt context. This new feature:
- Enables _multiple_ alerts to be passed by the _server_ as context to the LLM, via [LangChain tools](https://github.com/elastic/kibana/pull/167097)
- Applies the user's [anonymization](https://github.com/elastic/kibana/pull/159857) settings to those alerts
- Only fields allowed by the user will be sent as context to the LLM
- Users may enable or disable anonymization for specific fields (via settings)
- Click the conversation's `Show anonymized` toggle to see the anonymized values sent to / received from the LLM:
### Settings
This feature is enabled and configured via the `Knowledge Base` > `Alerts` settings in the screenshot below:
- The `Alerts` toggle enables or disables the feature
- The slider has a range of `10` - `100` alerts (default: `20`)
When the setting above is enabled, up to `n` alerts (as determined by the slider) that meet the following criteria will be returned:
- the `kibana.alert.workflow_status` must be `open`
- the alert must have been generated in the last `24 hours`
- the alert must NOT be a `kibana.alert.building_block_type` alert
- the `n` alerts are ordered by `kibana.alert.risk_score`, to prioritize the riskiest alerts
### Feature flag
To use this feature:
1) Add the `assistantRagOnAlerts` feature flag to the `xpack.securitySolution.enableExperimental` setting in `config/kibana.yml` (or `config/kibana.dev.yml` in local development environments), per the example below:
```
xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts']
```
2) Enable the `Alerts` toggle in the Assistant's `Knowledge Base` settings, per the screenshot below:
## How it works
- When the `Alerts` settings toggle is enabled, http `POST` requests to the `/internal/elastic_assistant/actions/connector/{id}/_execute` route include the following new (optional) parameters:
- `alertsIndexPattern`, the alerts index for the current Kibana Space, e.g. `.alerts-security.alerts-default`
- `allow`, the user's `Allowed` fields in the `Anonymization` settings, e.g. `["@timestamp", "cloud.availability_zone", "file.name", "user.name", ...]`
- `allowReplacement`, the user's `Anonymized` fields in the `Anonymization` settings, e.g. `["cloud.availability_zone", "host.name", "user.name", ...]`
- `replacements`, a `Record<string, string>` of replacements (generated on the server) that starts empty for a new conversation, and accumulates anonymized values until the conversation is cleared, e.g.
```json
"replacements": {
"e4f935c0-5a80-47b2-ac7f-816610790364": "Host-itk8qh4tjm",
"cf61f946-d643-4b15-899f-6ffe3fd36097": "rpwmjvuuia",
"7f80b092-fb1a-48a2-a634-3abc61b32157": "6astve9g6s",
"f979c0d5-db1b-4506-b425-500821d00813": "Host-odqbow6tmc",
// ...
},
```
- `size`, the numeric value set by the slider in the user's `Knowledge Base > Alerts` setting, e.g. `20`
- The `postActionsConnectorExecuteRoute` function in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts` was updated to accept the new optional parameters, and to return an updated `replacements` with every response. (Every new request that is processed on the server may add additional anonymized values to the `replacements` returned in the response.)
- The `callAgentExecutor` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts` previously used a hard-coded array of LangChain tools that had just one entry, for the `ESQLKnowledgeBaseTool` tool. That hard-coded array was replaced in this PR with a call to the (new) `getApplicableTools` function:
```typescript
const tools: Tool[] = getApplicableTools({
allow,
allowReplacement,
alertsIndexPattern,
assistantLangChain,
chain,
esClient,
modelExists,
onNewReplacements,
replacements,
request,
size,
});
```
- The `getApplicableTools` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/index.ts` examines the parameters in the `KibanaRequest` and only returns a filtered set of LangChain tools. If the request doesn't contain all the parameters required by a tool, it will NOT be returned by `getApplicableTools`. For example, if the required anonymization parameters are not included in the request, the `open-alerts` tool will not be returned.
- The new `alert-counts` LangChain tool returned by the `getAlertCountsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_tool.ts` provides the LLM the results of an aggregation on the last `24` hours of alerts (in the current Kibana Space), grouped by `kibana.alert.severity`. See the `getAlertsCountQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_query.ts` for details
- The new `open-alerts` LangChain tool returned by the `getOpenAlertsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_tool.ts` provides the LLM up to `size` non-building-block alerts generated in the last `24` hours (in the current Kibana Space) with an `open` workflow status, ordered by `kibana.alert.risk_score` to prioritize the riskiest alerts. See the `getOpenAlertsQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_query.ts` for details.
- On the client, a conversation continues to accumulate additional `replacements` (and send them in subsequent requests) until the conversation is cleared
- Anonymization functions that were only invoked by the browser were moved from the (browser) `kbn-elastic-assistant` package in `x-pack/packages/kbn-elastic-assistant/` to a new common package: `x-pack/packages/kbn-elastic-assistant-common`
- The new `kbn-elastic-assistant-common` package is also consumed by the `elastic_assistant` (server) plugin: `x-pack/plugins/elastic_assistant`
Consolidates UI elements and backend code to create/delete data views
and destination indices related to transforms and data frame analytics.
We ended up with two different approaches for creating data views in the
wizards for transforms and data frame analytics, the original reason was
we were not aware of the `allowNoIndex: true` setting and worked around
that in different ways.
This PR aligns UI workflows and moves related code to a new package
`@kbn/ml-data-view-utils` for data views and
`@kbn/ml-creation-wizard-utils` for the destination index form. The
latter might be used for other shared components across wizard..
In Data Frame Analytics, the checkbox to create a data view was removed
from the last "Create" step, instead the option to create a data view
was moved to the "Details" step.
In Transforms, the UI component to create the destination index was
brought over from DFA where there is a switch option to automatically
use the job ID as the name for the destination index by default.
Moves the categorize field uiAction trigger and action and related items
to the AIOps/ML uiActions package.
ML and AIOps are adding more and more uiActions, and so it's nicer to
have them all in one package.
Also cleans up the registration of the uiActions in the AIOps plugin
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Related to https://github.com/elastic/kibana/issues/169759.
## Summary
This PR creates the initial plugin for Dataset Quality.
The plugin has the basic boilerplate that allows us to continue working
on the UI and Server sides in parallel.
Further configs are to be added along with upcoming commits like:
1. Storybook
2. FTR configs
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/166834
Moves the following components out of infra plugin so we can remove the
apm->infra dependency
- `/infra/metrics_explorer` route
- (Host|Pod|Container)MetricsTable components
- InfraAppId
- InfraLocators
### Testing
We should focus the testing on the metrics_explorer route which is used
by 1. Tables in the Infrastructure section of apm and 2. Infrastructure
> Metrics explorer app. No functionality was added so these apps should
have the same existing behavior. Easiest way to get these views loaded
is by connecting kibana to an edge-oblt cluster, alternatively load
service and metrics with a data loader.
### Follow up
- (Host|Pod|Container)MetricsTable components are part of the
metricsDataAccess plugin contract but should be moved to stateless
package
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Jason Rhodes <jason.rhodes@elastic.co>
Closes https://github.com/elastic/kibana/issues/161906
Related to:
https://github.com/elastic/observability-dev/discussions/2787
(_internal_)
This add a new plugin `apm_data_access` that contains the APM query
targets (indices to query for APM data).
This plugin can be consumed by apm and any other plugin, making it
possible for other plugins to know about the configured APM query
targets.
## Example:
APM query targets can be specified in kibana[.dev].yml using
`xpack.apm.indices.{dataset}: some-index-*` for instances:
```yml
xpack.apm.indices.transaction: apm-*
```
See all config options on:
https://www.elastic.co/guide/en/kibana/current/apm-settings-kb.html#general-apm-settings-kb
Query targets can also be specified via the UI (and persisted in a saved
object) via the settings page: `/app/apm/settings/apm-indices`
**Retrieving the query targets**
Query targets can be retrieved from other plugins via `getApmIndices`:
```ts
const apmIndices = await plugins.apmDataAccess.setup.getApmIndices(savedObjects.client);
```
TODO:
- [x] Add SO client and fetch space aware index config (see
4d7f626da0/x-pack/plugins/apm/server/routes/settings/apm_indices/get_apm_indices.ts (L32-L44))
- [ ] Add simple APM client for querying apm data
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR makes the following changes:
- Update look & feel of contextual insights (previously called prompts)
according to the new design that is being developed. Some things might
still change, but hopefully not too much.
- Move all the Observability AI Assistant (previously called CoPilot)
code into a separate plugin for better isolation, more specific code
ownership and to solve some circular dependency issues
- Use connectors instead of a kibana.yml setting
Note: for OpenAI, the model is currently hardcoded to `gpt-4` until
https://github.com/elastic/kibana/issues/162204 has been addressed.
557676b6-065a-4b6f-86b2-1f0c2fd5e07e
---------
Co-authored-by: Coen Warmer <coen.warmer@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Moves the server and client side code which performs analysis on data to
see whether it is suitable for categorization.
This is currently only used by the categorization job wizard to display
this callout:
However this analysis will be useful for the Log Pattern Analysis
feature and so moving the code to a package allows easier sharing
between ML and AIOPs plugins.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
closes: https://github.com/elastic/kibana/issues/159685
- Renaming _x-pack/plugins_:
`serverless_security` -> `security_solution_serverless`
`ess_security` -> `security_solution_ess`
- All the related configurations and types have also been renamed.
- i18n translation prefixes updated
- relocation of internal `security_solution_serverless` directories to
be consistent with `security_solution_ess`
### Eslint
I also added the plugins in the `.eslintrc` configuration, defining the
same rules as the `security_solution` plugin.
All eslint errors have been addressed (mainly _type_ imports errors)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Closes#159128
Due to a dependencies issue when disabling a plugin in serverless mode,
the LogStream feature and related logic were disabled for every
consumer.
We decided to split this shared component and endpoint into their own
plugin of shared logs utilities, reducing to the minimum the required
dependency that could disable the plugin.
What we moved can be summarized with:
- `infrastructure-monitoring-log-view` saved object definition and
registration
- LogViews server/client services (exposed with start contract) +
related endpoints
- LogEntries server service + related endpoints
- LogEntriesDomain logic (exposed with start contract)
- `<LogStream />` component
- `<ScrollableLogTextStreamView />` component and related logic
- LogView state machine
- Containers/Hooks to consume the moved APIs.
- Common types/utils definition, now exported and consumed as a
dependency from the `infra` plugin.
## 🤓 Review hints
Most of the changes are just renaming and moving stuff into the new
plugin, but for some operations was required to implement new logic,
which may deserve a more critical review:
- server/public `plugin.ts` files for the `infra` and `logs_shared`
plugins. The new plugin now registers the fallback actions to retrieve a
source configuration if there's no stored log view. It also set the
configuration for the message field and registers the log view saved
object.
- the `logEntriesDomain` has also been moved inside the new plugin, but
is also used by the logs-analysis endpoints, so it is exposed by the
logs_shared plugin and consumed by `infra`.
## 👣 Following steps
We currently are still using the `observability` plugin for consuming
the CoPilot feature on our LogsStream flyout.
The plugin dependency is marked as optional, so disabling the
`observability` plugin in a serverless environment won't disable also
the exposed features in this new plugin, but it'll affect only the
CoPilot feature, which won't be loaded.
In future, would be nice to extract the CoPilot feature into its own
package/plugin, so that also serverless projects can consume it without
depending on `observability.
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Closes https://github.com/elastic/observability-dev/issues/2655
This PR introduces a customized log consumption experience in the
Discover plugin. By leveraging the new `discover_log_explorer` plugin
and utilizing the `discover.customize` functionality, we have curated a
more tailored user experience.
The key feature of this implementation is the `DatasetSelector`
component, which replaces the original Discover `DataViewPicker`. It
handles the retrieval, rendering, and navigation of integrations and
data streams related to logs, providing an improved user interface.
This PR involves significant development efforts, including the creation
of the `discover_log_explorer` plugin, implementation of services, state
machines, custom hooks, and enhancements to presentational components.
The following overview will help reviewers understand the
responsibilities of each component in this implementation.
d725b699-452e-4718-8189-8dc1fab4d044
## DatasetsService & DatasetsClient
The DatasetsService is introduced, a crucial component that mediates
access to the newly implemented DatasetsClient. During the plugin's
lifecycle, the DatasetsService exposes a client property through its
start() method, providing convenient access to a DatasetsClient
instance.
The DatasetsClient is responsible for abstracting the data fetching
process for two endpoints: the integrations endpoint and the data
streams listing endpoint. These endpoints are utilized to populate the
selector options in the user interface. To facilitate this, the
DatasetsClient exposes the findIntegrations and findDatasets methods,
which handle the respective data fetching.
## Discover Customization
The critical part of this work consists of where the customization is
applied.
Inside the `public/plugin.tsx`, we lazy load and create, injecting the
required dependencies, the `CustomDatasetSelector`, which already
encapsulates all the logic required to make the selector work with the
external APIs.
We kept separating the data fetching logic from how the selector works,
and all the data and events are passed into the UI component with
properties.
```ts
discover.customize(
DISCOVER_LOG_EXPLORER_PROFILE_ID,
({ customizations, stateContainer }) => {
customizations.set({
id: 'search_bar',
CustomDataViewPicker: createLazyCustomDatasetSelector({
datasetsClient: datasetsService.client,
stateContainer,
}),
});
...
```
## Data fetching state machines & custom hooks
To handle the data fetching of integrations and unmanaged data streams,
we created two different state machines to separately handle the related
action for each dataset, such as remote search, in-memory search, error
handling etc.
### Integration machine and useIntegrations
The integrations state machine handles automatic data fetching of the
resources and additionally provides transitions for loading more
integrations, searching integrations by HTTP request, searching locally
into integration streams, and all the related loading and error handling
states.
It is then interpreted inside the `useIntegrations` custom hook, which
exposes the fetched data and handlers for all the above-mentioned
actions.
<img width="1975" alt="Screenshot 2023-05-30 at 09 44 42"
src="6daeca9f-826d-4a0f-bd90-eb4826ed1bde">
### Datasets machine and useDatasets
Similar to the integrations state machine, but simplified since the data
streams search can only happen with HTTP requests and there is no
pagination that requires to handle the load of more entries.
It is interpreted inside the `useDatasets` custom hook, which also
exposes the fetched data and handlers for the available actions.
<img width="1692" alt="Screenshot 2023-05-30 at 09 45 11"
src="5f9690e2-4e8f-439e-9ffd-f3b34cf3eaf5">
## DatasetSelector
The `DatasetSelector` component contains all the logic that manages the
navigation and searches across the different panels that render
integrations, integrations' streams or unmanaged streams.
As the datasets come from different APIs or are performed in-memory, the
search work follow this logic:
- When listing the integrations list (first level of the
`EuiContextMenu`), the search is done with an HTTP request.
- When listing the data streams list for a specific integration (second
level of the `EuiContextMenu`), the search is done in-memory, filtering
and sorting directly in the client.
- When listing the unmanaged data streams list (second level of the
`EuiContextMenu`), the search is done again with an HTTP request.
To handle these possible user journeys correctly without side effects,
we created another state machine and exposed its actions with an
internal `useDatasetSelector` custom hook.
<img width="1978" alt="Screenshot 2023-05-30 at 09 46 04"
src="84aa4247-c65d-40de-9eb6-6117bee731f8">
## Next steps
This component will change quite a lot until we won't get to a final
design. As soon as a first solid mvp is defined for production, a
complete test for the component will be implemented, among with a more
generic functional test for the core customization features.
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Felix Stürmer <weltenwort@users.noreply.github.com>
## Summary
Original issue: https://github.com/elastic/security-team/issues/6489
Relevant issue: https://github.com/elastic/kibana/pull/158461 |
https://github.com/elastic/kibana/pull/158955
1. Move Landing Cards Component to ess_security plugin
2. Any component render `LandingPageComponent` when no indices exists
will be redirected to get started page.
**Steps to verify:**
1. Start an empty Elastic Search to verify the get started page, then
visit all the pages that render get started page when no indices exist
to see if they are rendered properly.
2. Add some data to Elastic Search and visit each page and see if it's
rendered properly. Open the timeline, and flyout, make sure they are
looking alright on the page.
ESS `yarn start`:
Get started page:
No data view exists:
<img width="2549" alt="ess-get-started"
src="dd560ab3-e3b1-4695-adfa-a1851da04cb0">
Serverless `yarn serverless-security`:
Get started page:
<img width="2553" alt="serverless-get-started"
src="2839c66f-e675-4eb5-b909-1b7af450b6d1">
No data view exists:
<img width="2542" alt="serverless-no-data"
src="d8e3a26b-3d39-442a-98ca-df0831e332ea">
Data exists:
<img width="2539" alt="Screenshot 2023-06-14 at 23 30 34"
src="ab7b413e-f25a-4c5e-9946-f4094a26fcdb">
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Sergi Massaneda <sergi.massaneda@gmail.com>
Refactors outdated approach of sharing the custom data grid component
from the `ml` plugin to the `transform` plugin to use packages instead.
Creates the following packages:
- `@kbn/ml-data-grid`
- `@kbn/ml-date-utils`
- `@kbn/ml-runtime-field-utils`
----
- Imports have been refactored so the `transform` plugin no longer
imports code from packages `@kbn/ml-anomaly-utils` and
`@kbn/ml-data-frame-analytics-utils`.
- Removed again auto-generated `@type` related JSDoc comments. Within
the code those annotations are quite redundant, would be cumbersome to
maintain and they are not necessary to satisfy the "missing comments"
check.
- The `renderCellPopover` callback has been refactored out of the
`DataGrid` component and can now be passed in as an optional prop. It is
only used for Data Frame Analytics and this way we can avoid some
dependency of the transform plugin on DFA related code.
- Some more code has been moved to `@kbn/ml-anomaly-utils` to make
available via packages what's needed for the data grid.
## [Security Solution] Elastic Security Assistant
The _Elastic Security Assistant_ has entered the chat, integrating generative AI and large language models (LLMs) into the workflows of Elastic Security users.
Bring your alerts, events, rules, and data quality checks into the conversation.
<31d65c78-5692-4817-b726-820c5df0801c>
This PR merges a feature branch developed by @spong and @andrew-goldstein , seeded by @jamesspi 's prototype of the assistant. Connectivity to LLMs is provided the [Generative AI Connector](<https://github.com/elastic/kibana/pull/157228>) , developed by @stephmilovic . This PR includes:
- A new reusable Kibana package containing the assistant: `x-pack/packages/kbn-elastic-assistant`
- See the `How to embed the Assistant in other parts of Kibana` for details
- Assistant integration into Elastic Security Solution workflows (e.g. alerts, cases, Timeline, rules, data quality)
### An assistant trained on the Elastic stack and Elastic Security
The [Generative AI Connector](<https://github.com/elastic/kibana/pull/157228>) connects the assistant to OpenAI and Azure OpenAI models trained with knowledge of the Elastic stack and the Elastic Security solution, including:
- The Elastic open [Detection Rules](https://github.com/elastic/detection-rules)
- The [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html)
- Elastic query languages, including [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), [EQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html), and the [Elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)
- [Elasticsearch API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rest-apis.html)
This training enables the assistant to offer fully interactive chat experiences that include:
- alert summarization
- interactive query generation
- workflow suggestions
- generating ingestion configurations that conform to the Elastic Common Schema
- your imagination
using context from Elastic Security.
### Take action from your conversations
The Actions (from assistant response):
- Send KQL to Timeline
- Send EQL to Timeline
- Send Elasticsearch DSL to Timeline
- Send Note to timeline
- Create new case
- Add to existing case
- Copy to clipboard
### Components architecture diagram
### How to embed the Assistant in other parts of Kibana
Follow the general instructions in `x-pack/packages/kbn-elastic-assistant/index.ts` to integrate the assistant into a Kibana app.
#### Step 1 - Wrap your Kibana app in the `AssistantProvider` component
```ts
// Step 1: Wrap your Kibana app in the `AssistantProvider` component. This typically
// happens in the root of your app. Optionally provide a custom title for the assistant:
/** provides context (from the app) to the assistant, and injects Kibana services, like `http` */
export { AssistantProvider } from './impl/assistant_context';
```
#### Step 2: Add the `AssistantOverlay` component to your app
```ts
// Step 2: Add the `AssistantOverlay` component to your app. This component displays the assistant
// overlay in a modal, bound to a shortcut key:
/** modal overlay for Elastic Assistant conversations */
export { AssistantOverlay } from './impl/assistant/assistant_overlay';
// In addition to the `AssistantOverlay`, or as an alternative, you may use the `Assistant` component
// to display the assistant without the modal overlay:
/** this component renders the Assistant without the modal overlay to, for example, render it in a Timeline tab */
export { Assistant } from './impl/assistant';
```
#### Step 3: Wherever you want to bring context into the assistant, use the any combination of the following
```ts
// Step 3: Wherever you want to bring context into the assistant, use the any combination of the following
// components and hooks:
// - `NewChat` component
// - `NewChatById` component
// - `useAssistantOverlay` hook
/**
* `NewChat` displays a _New chat_ icon button, providing all the context
* necessary to start a new chat. You may optionally style the button icon,
* or override the default _New chat_ text with custom content, like `🪄✨`
*
* USE THIS WHEN: All the data necessary to start a new chat is available
* in the same part of the React tree as the _New chat_ button.
*/
export { NewChat } from './impl/new_chat';
/**
* `NewChatByID` displays a _New chat_ icon button by providing only the `promptContextId`
* of a context that was (already) registered by the `useAssistantOverlay` hook. You may
* optionally style the button icon, or override the default _New chat_ text with custom
* content, like {'🪄✨'}
*
* USE THIS WHEN: all the data necessary to start a new chat is NOT available
* in the same part of the React tree as the _New chat_ button. When paired
* with the `useAssistantOverlay` hook, this option enables context to be be
* registered where the data is available, and then the _New chat_ button can be displayed
* in another part of the tree.
*/
export { NewChatById } from './impl/new_chat_by_id';
/**
* `useAssistantOverlay` is a hook that registers context with the assistant overlay, and
* returns an optional `showAssistantOverlay` function to display the assistant overlay.
* As an alterative to using the `showAssistantOverlay` returned from this hook, you may
* use the `NewChatById` component and pass it the `promptContextId` returned by this hook.
*
* USE THIS WHEN: You want to register context in one part of the tree, and then show
* a _New chat_ button in another part of the tree without passing around the data, or when
* you want to build a custom `New chat` button with features not not provided by the
* `NewChat` component.
*/
export { useAssistantOverlay } from './impl/assistant/use_assistant_overlay';
```
Co-authored-by: Garrett Spong <garrett.spong@elastic.co>
Co-authored-by: Andrew Macri <andrew.macri@elastic.co>
Creates Package `@kbn/ml-anomaly-utils`.
- This moves some of the utility functions, constants and types related
to Anomaly Detection we previously exposed from the `ml` plugin itself
to a package `@kbn/ml-anomaly-utils`, resulting in ~5.9KB reduction of
the plugin's page load bundle size.
- To reduce increases in async bundle size for consuming plugins the
utils have been refactored into individual files to allow deep imports
and get some optimization through that. Some previously duplicated code
in consuming plugins has been deleted and replaced with deep imports of
the corresponding original code in the package.
- Types have been prefixed with `Ml`.
- Constants have been prefixed with `ML_`.
Created package via `node scripts/generate package @kbn/ml-anomaly-utils
--web --dir ./x-pack/packages/ml/anomaly_utils`.
Closes#154733
Creates a new plugin for logs onboarding with wizard to organize steps
into discrete views.
#### TODO:
- [x] rename plugin to observability_onboarding
- [x] configure: UI and server plugin
- [x] enable/disable new plugin
- [x] remove the link to it from Observability nav
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Yngrid Coello <yngrid.coello@elastic.co>
Co-authored-by: Yngrid Coello <yngrdyn@gmail.com>
Creates a package for sharing general trained model util code.
Initially only contains the constants which had been copied in the
enterprise search plugin.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/149344
This PR migrates all plugins to packages automatically. It does this
using `node scripts/lint_packages` to automatically migrate
`kibana.json` files to `kibana.jsonc` files. By doing this automatically
we can simplify many build and testing procedures to only support
packages, and not both "packages" and "synthetic packages" (basically
pointers to plugins).
The majority of changes are in operations related code, so we'll be
having operations review this before marking it ready for review. The
vast majority of the code owners are simply pinged because we deleted
all `kibana.json` files and replaced them with `kibana.jsonc` files, so
we plan on leaving the PR ready-for-review for about 24 hours before
merging (after feature freeze), assuming we don't have any blockers
(especially from @elastic/kibana-core since there are a few core
specific changes, though the majority were handled in #149370).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Moves duplicates of `DatePickerWrapper` and related code to package
`@kbn/ml-date-picker`. The duplicated components across the code base
have been consolidated and diverging features combined. Each duplicate
has been checked with a diff against the package before deletion.
- Moves duplicates of `query_utils.ts` to a package
`@kbn/ml-query_utils`.
- Some jest test were migrated from enzyme to react-testing-lib.
- `i18n` strings and data-test-subjects have been updated to be prefixes
in line with package names.
- Replaces custom code related to the `compact` flag with EUI's
breakpoints.
* wip file upload component
* early version of upload component, ui only
* compressed buttons
* retry button and restructure some more stuff about default args for the stories
* added first iteration of upload state with tests
* do not self-unsubscribe
* remove unused import
* added simple state store observable and ability to abort uploads
* rename file_upload => file_upload_ui
* rename fileupload -> uploadfile and created new upload_files.ts stateful component, wip
* move file kinds registry to common
* hook up files registry to public side, also delete server side implementation
* implemented upload file ui, changed files$ to be a list of objects, not an array of observables
* did a bunch of stuff hooking up state to component
* added public side mock for filesclient
* rather use existing state to derive values, some other stuff
* added comment
* added files context
* throw first error
* check max file size and added abort state tets
* use i18n
* check mime type too and handle retry state correctly
* handle immediate upload
* update error message
* actually give files back after done
* upload files => upload file and move some stuff around
* map instead of tap
* use form row for showing the error message
* minor refactor
* export everything
* move some files around
* added some react jest tests too
* actually add the test file (previous commit was for a fix
* added processing of file names
* try fix import path
* remove stories for UI component
* type lints
* added i18n.json
* reverse direction again
* kibana utils bundle added
* type lint
* remove unnecessary variable
* updated where route registration happens
* remove _
* removed an option (compressed) also enabled repeated uploads and updated how the error message is laid out
* put upload file component behind lazy laoding
* added export of component back in
* typo
* do not show messages while uploading
* added a test case for the display of error messages
* remove unused import
* expand comment
* use the correct file name parser!
* updated story name and how long error message is generated
* rename inner render method
* fix types
* upload_file_ui -> upload_file.component
* address a11y violation and use inline css only
* updated copy per feedback
* refactor state class per feedback
* refactor to use context per vadims recommendation
* added some more tests
* added a comment and allowed passing through of kind type
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
* removed props from .component since we now have context
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Adds the code for the dual brush component for users to be able select log rate spikes in histogram charts.
For this PR a new package @kbn/aiops-components was created that also includes ProgressControls from @kbn/aiops-utils now. The reason for this is: The brush component includes code from d3 that cannot be imported on the server side which aiops-utils was also used for.
Updates the API endpoint "/internal/aiops/explain_log_rate_spikes" to work with real data instead of the dummy example.
The analysis now includes identifying field candidates and statistically significant field value pairs. Because there is no UI in place yet to allow a user to select custom time ranges for baseline and deviation, for now we fetch a date histogram of the selected index pattern and identify the parameters to get passed as WindowParameters automatically by picking the part of the date histogram with the highest doc count.
This PR is more about getting the API endpoint into shape and updates integration tests too. The UI code should be considered temporary. At the moment it just outputs the returned analysis state as raw JSON in a EUI code block.
* Add kubernetes link on nav sidebar
* Create new plugin for k8s vis page
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Paulo Henrique <paulo.scape@gmail.com>
Sets up the boilerplate code for the aiops plugin and adds a demo page within the ML app to demonstrate single API request data streaming from Kibana server to UI client.
