kibana/x-pack/plugins/security_solution/common/experimental_features.ts
Andrew Macri 3f0fa7d245
[Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts (#172542)
## [Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts

This PR implements _Retrieval Augmented Generation_ (RAG) for Alerts in the Security Solution. This feature enables users to ask the assistant questions about the latest and riskiest open alerts in their environment using natural language, for example:

- _How many alerts are currently open?_
- _Which alerts should I look at first?_
- _Did we have any alerts with suspicious activity on Windows machines?_

### More context

Previously, the assistant relied solely on the knowledge of the configured LLM and _singular_ alerts or events passed _by the client_ to the LLM as prompt context. This new feature:

- Enables _multiple_ alerts to be passed by the _server_ as context to the LLM, via [LangChain tools](https://github.com/elastic/kibana/pull/167097)
- Applies the user's [anonymization](https://github.com/elastic/kibana/pull/159857) settings to those alerts
  - Only fields allowed by the user will be sent as context to the LLM
  - Users may enable or disable anonymization for specific fields (via settings)
  - Click the conversation's `Show anonymized` toggle to see the anonymized values sent to / received from the LLM:
  ![show_anonymized](7db85f69-9352-4422-adbf-c97248ccb3dd)

### Settings

This feature is enabled and configured via the `Knowledge Base` > `Alerts` settings in the screenshot below:
![rag_on_alerts_setting](9161b6d4-b7c3-4f37-bcde-f032f5a02966)

- The `Alerts` toggle enables or disables the feature
- The slider has a range of `10` - `100` alerts (default: `20`)

When the setting above is enabled, up to `n` alerts (as determined by the slider) that meet the following criteria will be returned:

- the `kibana.alert.workflow_status` must be `open`
- the alert must have been generated in the last `24 hours`
- the alert must NOT be a `kibana.alert.building_block_type` alert
- the `n` alerts are ordered by `kibana.alert.risk_score`, to prioritize the riskiest alerts

### Feature flag

To use this feature:

1) Add the `assistantRagOnAlerts` feature flag to the `xpack.securitySolution.enableExperimental` setting in `config/kibana.yml` (or `config/kibana.dev.yml` in local development environments), per the example below:

```
xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts']
```

2) Enable the `Alerts` toggle in the Assistant's `Knowledge Base` settings, per the screenshot below:

![alerts_toggle](07f241ea-af4a-43a4-bd19-0dc6337db167)

## How it works

- When the `Alerts` settings toggle is enabled, http `POST` requests to the `/internal/elastic_assistant/actions/connector/{id}/_execute` route include the following new (optional) parameters:
  - `alertsIndexPattern`, the alerts index for the current Kibana Space, e.g. `.alerts-security.alerts-default`
  - `allow`, the user's `Allowed` fields in the `Anonymization` settings, e.g.  `["@timestamp", "cloud.availability_zone", "file.name", "user.name", ...]`
  - `allowReplacement`, the user's `Anonymized` fields in the `Anonymization` settings, e.g. `["cloud.availability_zone", "host.name", "user.name", ...]`
  - `replacements`, a `Record<string, string>` of replacements (generated on the server) that starts empty for a new conversation, and accumulates anonymized values until the conversation is cleared, e.g.

```json
"replacements": {
    "e4f935c0-5a80-47b2-ac7f-816610790364": "Host-itk8qh4tjm",
    "cf61f946-d643-4b15-899f-6ffe3fd36097": "rpwmjvuuia",
    "7f80b092-fb1a-48a2-a634-3abc61b32157": "6astve9g6s",
    "f979c0d5-db1b-4506-b425-500821d00813": "Host-odqbow6tmc",
    // ...
},
```

- `size`, the numeric value set by the slider in the user's `Knowledge Base > Alerts` setting, e.g. `20`

- The `postActionsConnectorExecuteRoute` function in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts` was updated to accept the new optional parameters, and to return an updated `replacements` with every response. (Every new request that is processed on the server may add additional anonymized values to the `replacements` returned in the response.)

- The `callAgentExecutor` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts` previously used a hard-coded array of LangChain tools that had just one entry, for the `ESQLKnowledgeBaseTool` tool. That hard-coded array was replaced in this PR with a call to the (new) `getApplicableTools` function:

```typescript
  const tools: Tool[] = getApplicableTools({
    allow,
    allowReplacement,
    alertsIndexPattern,
    assistantLangChain,
    chain,
    esClient,
    modelExists,
    onNewReplacements,
    replacements,
    request,
    size,
  });
```

- The `getApplicableTools` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/index.ts` examines the parameters in the `KibanaRequest` and only returns a filtered set of LangChain tools. If the request doesn't contain all the parameters required by a tool, it will NOT be returned by `getApplicableTools`. For example, if the required anonymization parameters are not included in the request, the `open-alerts` tool will not be returned.

- The new `alert-counts` LangChain tool returned by the `getAlertCountsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_tool.ts` provides the LLM the results of an aggregation on the last `24` hours of alerts (in the current Kibana Space), grouped by `kibana.alert.severity`. See the `getAlertsCountQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_query.ts` for details

- The new `open-alerts` LangChain tool returned by the `getOpenAlertsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_tool.ts` provides the LLM up to `size` non-building-block alerts generated in the last `24` hours  (in the current Kibana Space) with an `open` workflow status, ordered by `kibana.alert.risk_score` to prioritize the riskiest alerts. See the `getOpenAlertsQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_query.ts` for details.

- On the client, a conversation continues to accumulate additional `replacements` (and send them in subsequent requests) until the conversation is cleared

- Anonymization functions that were only invoked by the browser were moved from the (browser) `kbn-elastic-assistant` package in `x-pack/packages/kbn-elastic-assistant/` to a new common package: `x-pack/packages/kbn-elastic-assistant-common`
  - The new `kbn-elastic-assistant-common` package is also consumed by the `elastic_assistant` (server) plugin: `x-pack/plugins/elastic_assistant`
2023-12-06 00:56:04 -05:00

191 lines
5.6 KiB
TypeScript

/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/
export type ExperimentalFeatures = { [K in keyof typeof allowedExperimentalValues]: boolean };
/**
* A list of allowed values that can be used in `xpack.securitySolution.enableExperimental`.
* This object is then used to validate and parse the value entered.
*/
export const allowedExperimentalValues = Object.freeze({
tGridEnabled: true,
tGridEventRenderedViewEnabled: true,
// FIXME:PT delete?
excludePoliciesInFilterEnabled: false,
kubernetesEnabled: true,
chartEmbeddablesEnabled: true,
donutChartEmbeddablesEnabled: false, // Depends on https://github.com/elastic/kibana/issues/136409 item 2 - 6
alertsPreviewChartEmbeddablesEnabled: false, // Depends on https://github.com/elastic/kibana/issues/136409 item 9
/**
* This is used for enabling the end-to-end tests for the security_solution telemetry.
* We disable the telemetry since we don't have specific roles or permissions around it and
* we don't want people to be able to violate security by getting access to whole documents
* around telemetry they should not.
* @see telemetry_detection_rules_preview_route.ts
* @see test/detection_engine_api_integration/security_and_spaces/tests/telemetry/README.md
*/
previewTelemetryUrlEnabled: false,
/**
* Enables the insights module for related alerts by process ancestry
*/
insightsRelatedAlertsByProcessAncestry: true,
/**
* Enables extended rule execution logging to Event Log. When this setting is enabled:
* - Rules write their console error, info, debug, and trace messages to Event Log,
* in addition to other events they log there (status changes and execution metrics).
* - We add a Kibana Advanced Setting that controls this behavior (on/off and log level).
* - We show a table with plain execution logs on the Rule Details page.
*/
extendedRuleExecutionLoggingEnabled: false,
/**
* Enables streaming for Security AI Assistant - non-langchain only (knowledge base off)
*/
assistantStreamingEnabled: false,
/**
* Enables the SOC trends timerange and stats on D&R page
*/
socTrendsEnabled: false,
/**
* Enables the automated response actions in rule + alerts
*/
responseActionsEnabled: true,
/**
* Enables the automated endpoint response action in rule + alerts
*/
endpointResponseActionsEnabled: true,
/**
* Enables the `upload` endpoint response action (v8.9)
*/
responseActionUploadEnabled: true,
/**
* Enables top charts on Alerts Page
*/
alertsPageChartsEnabled: true,
/**
* Enables the alert type column in KPI visualizations on Alerts Page
*/
alertTypeEnabled: false,
/**
* Enables expandable flyout in create rule page, alert preview
*/
expandableFlyoutInCreateRuleEnabled: false,
/*
* Enables new Set of filters on the Alerts page.
*
**/
alertsPageFiltersEnabled: true,
/**
* Enables the Assistant Model Evaluation advanced setting and API endpoint, introduced in `8.11.0`.
*/
assistantModelEvaluation: false,
/**
* Enables Retrieval Augmented Generation (RAG) on Alerts in the assistant
*/
assistantRagOnAlerts: false,
/*
* Enables the new user details flyout displayed on the Alerts page and timeline.
*
**/
newUserDetailsFlyout: false,
/**
* Enable risk engine client and initialisation of datastream, component templates and mappings
*/
riskScoringPersistence: true,
/**
* Enables experimental Entity Analytics HTTP endpoints
*/
riskScoringRoutesEnabled: true,
/**
* disables ES|QL rules
*/
esqlRulesDisabled: false,
/**
* Enables Protection Updates tab in the Endpoint Policy Details page
*/
protectionUpdatesEnabled: true,
/**
* Enables alerts suppression for threshold rules
*/
alertSuppressionForThresholdRuleEnabled: false,
/**
* Disables the timeline save tour.
* This flag is used to disable the tour in cypress tests.
*/
disableTimelineSaveTour: false,
/**
* Enables the risk engine privileges route
* and associated callout in the UI
*/
riskEnginePrivilegesRouteEnabled: true,
/*
* Enables experimental Entity Analytics Asset Criticality feature
*/
entityAnalyticsAssetCriticalityEnabled: false,
/**
* Enables SentinelOne manual host manipulation actions
*/
sentinelOneManualHostActionsEnabled: false,
});
type ExperimentalConfigKeys = Array<keyof ExperimentalFeatures>;
type Mutable<T> = { -readonly [P in keyof T]: T[P] };
const allowedKeys = Object.keys(allowedExperimentalValues) as Readonly<ExperimentalConfigKeys>;
/**
* Parses the string value used in `xpack.securitySolution.enableExperimental` kibana configuration,
* which should be a string of values delimited by a comma (`,`)
*
* @param configValue
* @throws SecuritySolutionInvalidExperimentalValue
*/
export const parseExperimentalConfigValue = (
configValue: string[]
): { features: ExperimentalFeatures; invalid: string[] } => {
const enabledFeatures: Mutable<Partial<ExperimentalFeatures>> = {};
const invalidKeys: string[] = [];
for (const value of configValue) {
if (!allowedKeys.includes(value as keyof ExperimentalFeatures)) {
invalidKeys.push(value);
} else {
enabledFeatures[value as keyof ExperimentalFeatures] = true;
}
}
return {
features: {
...allowedExperimentalValues,
...enabledFeatures,
},
invalid: invalidKeys,
};
};
export const getExperimentalAllowedValues = (): string[] => [...allowedKeys];