Resolves https://github.com/elastic/kibana/issues/161393
## Summary
This PR is the first of a series to implement the summary and search
improvement feature on SLO.
Every PR will be merged against the feature branch:
`slo/feature-branch`. And we'll update this feature branch with main as
often as possible to keep conflict as a minimum.
This PR changes the SLO rollup index mapping, adding more fields to it,
that we are going to use to summarize the SLO rollup data later.
It also includes the SLO summary index templates (mappings, settings,
template) and install them with the other templates.
Since this is a **breaking change**, any SLO running will be shown as
`NO_DATA`. You can remove the SLO through the API or the UI, or update
them (make sure you change something significant (time window, indicator
params, ...) in order to induce a revision bump) so the underlying
transform is recreated using the new index structure.
Uses the recently created [category validation
package](https://github.com/elastic/kibana/pull/161261) to perform
validation on the field selected for pattern analysis.
If the field is considered unsuitable for categorization, a warning
callout is displayed which lists the reasons it is unsuitable.
If the field is suitable, no callout is displayed.
Other changes:
- Adds the selected field to the URL state, so it is remembered on page
refresh.
- If no field is in the URL, it will look for a field called `message`
in the data view and auto select it.
- renames the ML route `/jobs/categorization_field_examples` to
`/jobs/categorization_field_validation` as it is a more accurate name
and it's consistent with the newly added route in AIOPs.
**Log Pattern Analysis page in ML**
**Log Pattern Analysis flyout in Discover**
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Originally Kibana's `http` service did not support receiving streams,
that's why we used plain `fetch` for this. This has been fixed in
#158678, so this PR updates the streaming helpers to use Kibana's `http`
service from now on.
- The PR also breaks out the response stream code into its own package
and restructures it to separate client and server side code. This brings
down the `aiops` bundle size by `~300KB`! 🥳
- The approach to client side throttling/buffering was also revamped:
There was an issue doing the throttling inside the generator function,
it always waited for the timeout. The buffering is now removed from
`fetchStream`, instead `useThrottle` from `react-use` is used on the
reduced `data` in `useFetchStream`. Loading log rate analysis results
got a lot snappier with this update!
## Summary
Implementation of serverless-specific pages within the Unified IA
Navigation.
#### Links implemented:
- `Machine Learning`
- Landing page created on serverless only
- All links in the landing page go to `/ml` app
- `Dev Tools`
- Links directly to `/dev_tools` app
#### Links not implemented:
```// TODO: in a follow-up PR```
- Project Settings
- Change the _Settings_ name by _Project Settings_
- Modify the landing page items according to the design
## Changes
### Plugin contract changes
The Machine Learning landing page is the first page that is only available on serverless and should not exist in ess (there are more of this kind in the pipeline), so this PR implements the foundations to enable the _security_solution_serverless_ plugin to implement its own page components, configure the link definition and create new routes to render them in the Security Solution application.
These new APIs can be called from either `security_solution_serverless` or `security_solution_ess`, allowing those plugins to have their own offering-specific pages.
The new APIs exposed in the security_solution public contract are the following:
- `extraAppLinks$`: Observable to add extra app_links into the application links configuration, so they are stored and included in the SecuritySolution plugin `deepLinks` registry, to make them accessible from anywhere in the application using the `chrome.navLinks` API.
- `extraRoutes$`: Observable to add extra routes into the main Router, so it can render the new page components. These additional routes are appended after the "sub-plugin" (_alerts_, _timeline_, ...) routes, so it is not possible to override an existing route path.
### New `security-solution-navigation` package
Since now we need to use the same navigation components and hooks in different plugins, these functionalities have been extracted to the `@kbn/security-solution-navigation` package, which all Security plugins will depend on (generic, serverless, and ess).
The modules exposed by this package have been extracted from the main security_solution plugin and standardized. They include the Landing pages components (new [storybook](https://ci-artifacts.kibana.dev/storybooks/pr-161667/394abe76676c6a76b2982c1d3f5bb675739c3477/security_solution_packages/index.html?path=/story/landing-links-landing-links-icons-categories--landing-links-icons-categories) available), navigation hooks, and link utilities. Also, some types and constants have been moved to this package.
A new context provider has also been created, which needs to be in place in order to use this package. The `<NavigationProvider core={core}>` is required for the package functionalities to have access to the Kibana core navigation APIs: `navigateToUrl`, `navigateToApp`, and `getUrlForApp`.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: YulNaumenko <jo.naumenko@gmail.com>
## Summary
It fixes#161458 by adding API integration tests for the Threshold rule,
with many scenarios (file per scenario), and each scenario has a
complete life-cycle
### The scenario life-cycle
- Generating data using the `fake_host` dataset from the high-card
- Create a DataView based on the generated data
- Create the rule and wait to be active
- Get the fired alert and matches its value
- Clean up
### The covered scenarios
- Avg. percentage, fires alert
- Avg. percentage, fires alert with no data
- Custom equation on bytes filed, fires alert
- Doc count, fires alert
- Group by two fields, fires alert.
---------
Moves the server and client side code which performs analysis on data to
see whether it is suitable for categorization.
This is currently only used by the categorization job wizard to display
this callout:
However this analysis will be useful for the Log Pattern Analysis
feature and so moving the code to a package allows easier sharing
between ML and AIOPs plugins.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Resolves System Prompt not sending issues:
https://github.com/elastic/kibana/issues/161809
Also resolves:
- [X] Not being able to delete really long Conversation, System Prompt,
and Quick Prompt names
- [X] Fix user/all System Prompts being overridden on refresh
- [X] Conversation without default System Prompt not healed if it is
initial conversation when Assistant opens (Timeline)
- [X] New conversation created from Conversations Settings not getting a
connector by default
- [X] Current conversation not selected by default when settings gear is
clicked (and other assistant instances exist)
- [X] Sent to Timeline action sends anonymized values instead of actual
plaintext
- [X] Clicking Submit does not clear the text area
- [X] Remove System Prompt Tooltip
- [X] Fixes confusion when System or Quick Prompt is empty by adding a
placeholder value
- [X] Shows (empty prompt) in System Prompt selector when the Prompt
content is empty
- [X] Fixes connector error callout flashing on initial load
- [X] Shows `(empty prompt)` text within Prompt Editor when prompt
content is empty to prevent confusion
### Checklist
Delete any items that are not applicable to this PR.
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
This PR add a new indicator to support histogram fields. This will allow
you to either use a `range` aggregation or `value_count` aggregation for
the good and total events; including support for filtering with KQL on
both event types. When using a `range` aggregation, both the `from` and
`to` thresholds are required for the range and events will be to total
number of events within that range.[ Keep in mind, with the `range`
aggregation, the range includes the `from` value and excludes the `to`
value.](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-range-aggregation.html)
This PR also includes support for using the histogram field for a
"Custom Metric" indicator, `sum` is calculated on the values and not the
counts. If you need it calculated on the counts then you have to use the
histogram indicator.
<img width="776" alt="image"
src="1d46b722-df13-417e-bf3b-b3c450933da2">
---------
Co-authored-by: Kevin Delemme <kdelemme@gmail.com>
## Summary
Handles elastic/security-team#6971
This PR mainly resolved below 3 issues:
### Rename to `Add To Timeline` control in conversation code blocks to
`Investigate in Timeline`
- `Add to Timeline` according to existing Security Solution actions
means, adding a condition to the timeline with an `OR` clause without
affecting the existing Timeline.
- But the `Add to Timeline` control in the Security Assistant, creates a
new timeline on each action by the user, which contradicts the above
workflow. Hence, it might confuse user.
- `Investigate in Timeline` already means that a new timeline will be
created.
### `Add To Timeline` control was visible on types of codeblock. For
example, it does not make sense for a `Query DSL` to have an `Add to
Timeline` control.
- This PR adds the list of eligible types of queries/code blocks on
which `Add To Timeline` action can be added.
- Currently, that list only contains `kql`, `dsl` and `eql`. Below is
the complete list of types of query that can occur in code blocks.
- Please feel free to suggest a change.
```
'eql' | 'kql' | 'dsl' | 'json' | 'no-type';
```
### Lazy calculation of CodeBlockPortals and CodeBlock Action container
- To add controls to the conversation code blocks, we need to follow
below 2 steps.
1. get the codeBlock containers on which the controls can be added.
2. create portals in the HTML container with our `Add to Timeline`
control.
- Below are issues these steps sometime created.
1. We get codeBlock container in the `useLayoutEffect` but at the time,
all conversations might not have loaded because of which containers are
returns as the undefined.
2. Then, we try to create portal in the `undefined` container, which
fails and hence, `Add to Timeline` controls are not visible.
- Solution:
1. Instead of getting the codeblock container in useLayoutEffect, we get
the function which will eventually return that container, whenever we
are creating the portal.
2. Converted codeBlock Portal to a callback such that callback can be
called during the rendering which makes sure that all needed
conversations are available and using above step we can easily get the
portal containers.
Feel free to let me know if there are any issues with above strategy.
### Better Pattern matching.
- Currently, when we are trying to identify the type of codeblock it
might result in unexpected output because of below reason.
1. Let say, we are trying to identify KQL Query and for that we use
below phrases to match in the `OpenAI` response.
`'Kibana Query Language', 'KQL Query'`
2. Because of this, if the `OpenAI` response contains the phrase `KQL
query` or `kql query`, that fails because of case senstivity when
searching the above phrases.
3. This PR makes that part of pattern matching case insensitive
### Before
b472178a-0145-42d8-8fb9-ab107915086a
### After
b499f099-a7a1-435f-99b2-ab27ee1f5680
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
EPIC: https://github.com/elastic/kibana/issues/144943
## Summary
Update Events/alerts table to provide `CellActions` with a complete
`FieldSpec`object from DataView
### Affected pages:
* Alerts page
* Security Dashboards
* Rule preview
* Host events
* Users events
### How to test it
Use CellActions on one of the affected pages.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR:
- Adds the ability to create system action types
- Creates system connectors on Kibana `start` from the system action
types
- Prevents system action to be created/updated/deleted
- Return system actions from the get/getAll endpoints
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR handles bugs
- elastic/security-team#6977
- https://github.com/elastic/security-team/issues/6978
- elastic/security-team#6979.
Currently, below operations between System Prompts and Conversarions do
not work.
1. When a prompt is set as default for all conversation, it should be
automatically selected for any new conversation user creates.
2. When a new prompt is creates and set as default for all conversation,
it should be automatically selected for any new conversation user
creates.
3. When a prompt is edited such that, it is default for only certain
conversation, it should be automatically selected for that conversation.
4. When a prompt is edited such that conversations are removed to have
that default prompt, it should be automatically removed from
conversation default system prompt list.
In addition to above scenarios, this PR also handles one more bug.
Consider below interface of Conversation which has a property
`apiConfig.defaultSystemPrompt` is of type Prompt. It has been changed
from `defaultSystemPrompt?: Prompt` to `defaultSystemPrompt?: string`
where it will store `promptId` instead of complete prompt.
The current model was posing a problem where, if a prompt was updated,
all its copies in `Conversation` were needed to be updated leading to
inconsistencies. This is now resolved.
```typescript
export interface Conversation {
apiConfig: {
connectorId?: string;
defaultSystemPrompt?: Prompt;
provider?: OpenAiProviderType;
};
id: string;
messages: Message[];
replacements?: Record<string, string>;
theme?: ConversationTheme;
isDefault?: boolean;
}
```
## Summary
This PR adds support for applying a KQL filter to the good/total
metrics.
<img width="858" alt="image"
src="c271352c-10fd-49f1-89b8-a352b69f7f7c">
Original issue: https://github.com/elastic/kibana/issues/144943
## Summary
* Update CellActions value to be `Serializable`.
* Update Default Actions and SecuritySolution Actions to allowlist the
supported Kibana types.
* Add an extra check to Action's `execute` to ensure the field value is
compatible.
### How to test it?
* Open Discover and create a saved search with many different field
types
* Go to Security Solutions dashboards
* Create a new dashboard and import the saved search
* Test the created dashboard inside Security Solutions
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
The random payload for the proxy flushing fix was regenerated for every
push to the stream. This turned out to be quite slow. This PR updates
the logic to create the payload only once and reuse it for every push.
Note: Further testing in Cloud showed the differences there are not as
big as in my local testing, might be related to entropy for
`crypto.randomBytes`.
## Summary
Resolves https://github.com/elastic/kibana/issues/157385
Hides inference stats for the PyTorch models.
- The salient information (`inference_count`, `timestamp`) is a repeat
of what is already displayed in the Deployment Stats section.
- `missing_all_fields_count` is confusing as the PyTorch models take a
single input field rather than multiple fields as DFA models do, hence
omitted.
- The deployment stats have an
[error_count](https://www.elastic.co/guide/en/elasticsearch/reference/current/get-trained-models-stats.html)
field, hence it has been added to the Deployment Stats and
`failure_count` has been removed.
- Displays the stats tab by default for expanded rows if the model has
started deployments
## Summary
This PR fixes#160320 by changing the chart from the `CriterionPreview`
chart, borrowed from the Log Threshold Rule, to an embedded Lens
visualization that represents the correct document count in one chart. I
also took the liberty of changing the ratio chart to use the same
technique for consistency sake.
## Count with multiple conditions
### Before
<img width="736" alt="image"
src="6c6a27ea-f8e4-491f-8a12-261d0ed13dcb">
### After
<img width="736" alt="image"
src="9b18ebe9-e911-4e40-8911-bee55cd7d245">
## Count with multiple conditions and a group by
### Before
<img width="736" alt="image"
src="7b9462da-55b2-4f54-ba09-3c55b372ae2c">
### After
<img width="736" alt="image"
src="b268caed-242f-430a-ade0-14bf491ec899">
## Ratio with multiple conditions
### Before
<img width="736" alt="image"
src="55b8dfa2-7789-433b-bffd-e412bdb08b3f">
### After
<img width="736" alt="image"
src="a029bf8a-3ba1-4e16-87bd-097ebc526a4e">
## Ratio with multiple conditions and a group by
### Before
<img width="736" alt="image"
src="61ddf1e9-c5ad-4546-a539-15a51ee563c0">
### After
<img width="736" alt="image"
src="15b0aaa3-4ef9-47f6-baba-24869feae77e">
Resolves https://github.com/elastic/kibana/issues/156145
Resolves https://github.com/elastic/kibana/issues/160293
## 📝 Summary
This PR fixes a bug related to `shouldUnregister` used on controlled
fields which removes part of the form state while submitting the form,
due to the components unmounting. This is a weird issue between React
Hook Form and React Query, as if we were not using optimistic update
with RQ, we won't notice the problem in the first place.
I have done a few things in this PR:
1. I've introduced a `CreateSLOForm` type to decouple what the API is
expecting (CreateSLOInput) and how we structure the form and store the
form state. This is particularly useful when building a partial
`CreateSLOForm` from a partial url state `_a`.
2. I've introduced a hook that handles the change of indicator
correctly, resetting the default value as we change. The default values
are typed with each indicator schema type, and the hook will throw when
a new indicator is introduced but not handled there.
3. I've removed the custom metric manual useEffect and instead rely on
hidden registered inputs.
4. The time window type handles correctly the initial state, and reload
the default values when we change from rolling <-> calendar aligned.
5. I've grouped some code from the main form component into their own
hook to 1. add a layer of abstraction and 2. make the code more cohesive
6. When the create SLO call fails, we redirect the user to the form with
the previously entered values.
## 🧪 Testing
I would suggest to create and edit some SLOs, playing with the different
time window, budgeting method, indicators.
The main thing to look for are:
1. Switching indicator reset the form as expected
2. When editing an SLO, all the form fields are populated correctly with
the initial SLO values.
3. Creating an SLO with a bad indicator, for example with an invalid KQL
query, will redirect to the form with the previous value filled.
https://www.loom.com/share/516c3d5a1fa74db6bf839cfa0cf41f5d?sid=f0a1a33f-eb29-4b8f-b65f-ffce2313bad8
---------
Co-authored-by: Coen Warmer <coen.warmer@gmail.com>
## Summary
closes: https://github.com/elastic/kibana/issues/157191
Enables Discover DataGrid to use registered cell actions instead of the
default static actions.
### New `cellActionsTriggerId` prop
This PR introduces a new `cellActionsTriggerId` _optional_ prop in the
DataGrid component:
98c210f9ec/src/plugins/discover/public/components/discover_grid/discover_grid.tsx (L198-L201)
When this prop is defined, the component queries the trigger's registry
to retrieve the cellActions attached to it, using the CellActions
package' `useDataGridColumnsCellActions` hook. This hook returns the
cellActions array ready to be passed for each column to the EuiDataGrid
component.
When (non-empty) actions are found in the registry, they are used,
replacing all of the default static Discover actions. Otherwise, the
default cell actions are used.
This new prop also allows other instances of the Discover DataGrid to be
configured with custom cell actions, which will probably be needed by
Security Timeline integration with Discover.
### New `SEARCH_EMBEDDABLE_CELL_ACTIONS_TRIGGER` Trigger
Along with the new `cellActionsTriggerId` prop the plugin also registers
a new trigger for "saved search" embeddable:
055750c8dd/src/plugins/discover/public/plugin.tsx (L387)
And it gets passed to the DataGrid component on the Embeddable creation:
055750c8dd/src/plugins/discover/public/embeddable/saved_search_embeddable.tsx (L403)
Having this new trigger available allows solutions to attach custom
actions to it, in order to be displayed in the saved search embeddables.
Each action will be able to implement its `isCompatible` check to
determine if they are going to be displayed in the embedded saved search
DataGrid field, or not. If no compatible actions are found, DataGrid
will render the default static actions.
ℹ️ In this implementation, the actions registered to this new
"embeddable trigger" need to check if they are being rendered inside
Security using the `isCompatible` function, to prevent them from being
displayed in other solutions, resulting in a non-optimal architecture.
This approach was needed since there's no plausible way to pass the
`cellActionsTriggerId` property from the Dashboard Renderer used in
Security, all the way down to the specific Discover "saved search"
embeddable. However, the Dashboards team is planning to enable us to
pass options to nested embeddables using a registry
(https://github.com/elastic/kibana/issues/148933). When this new tool is
available we will be able to delegate the trigger registering to
Security and configure the "saved search" embeddables to use it.
Therefore, the trigger will only be used by Security, so we won't have
to worry about Security actions being rendered outside Security.
## Videos
before:
de92cd74-6125-4766-8e9d-7e0985932618
after:
f9bd597a-860e-4572-aa9d-9f1c72c11a4b
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Pablo Neves Machado <pablo.nevesmachado@elastic.co>
issue: https://github.com/elastic/kibana/issues/150347
## Context
Some Actions need to access `FieldSpec` data, which is not present on
the `CellActions` API (`subType`and `isMapped`). So we are updating the
`CellActions` `field` property to be compatible with `FieldSpec`.
## Summary
This PR is the first step to fix
https://github.com/elastic/kibana/issues/150347.
* Updates the `CellActions` to support an array of data objects, each
containing field (`FieldSpec`) and value
* Create a new `SecurityCellActions` component that accepts a field name
and loads `FieldSpec` from the Dataview.
## Examples
Before:
```tsx
<SecurityCellActions
value={'admin'}
field={{
name: 'user.name',
type: 'keyword',
searchable: true,
aggregatable: true,
...
}} />
```
After:
```tsx
<SecurityCellActions data={{ field: 'user.name', value: 'admin' }}/>
```
`SecurityCellActions` will load the spec from the Dataview and provide
it to `CellActons`.
`CellActons` now also support an of fields instead of only one field. It
will be useful when rendering cell actions for aggregated data like on
the Entity Analytic page. But for now, the actions are not supporting
multiple values, we need to rewrite them
https://github.com/elastic/kibana/issues/159480.
### Next steps
We must refactor the Security Solution to get `FieldSpec` from the
`DataView` instead of using BrowserFields. Ideally, we have to do that
for every `CellAction` call so the actions can access the `subType`
property.
- [x] ~Refactor the Security Solution CellActions calls to get
`FieldSpec` from the `DataView`~
- [x] Refactor data grid cell actions to get `FieldSpec` from the
`DataView`
- [ ] Rewrite actions to support multiple fields and use them on the
investigation in timeline (.andFilters)
- [ ] Fix https://github.com/elastic/kibana/issues/150347 using
`subType` from `fieldSpec`
- [ ] Fix https://github.com/elastic/kibana/issues/154714 using
`isMapped` from `fieldSpec`
### Extra information
*** Previously we were mixing `esTypes` and `kbnTypes`. For example, if
the `esType` is a keyword the `kbnType` has to be a `string`.
[Here](9799dbba27/packages/kbn-field-types/src/types.ts (L22))
you can check all possible ES and KBN types and
[here](9799dbba27/packages/kbn-field-types/src/kbn_field_types_factory.ts)
you can see the mapping between esType and kbnType
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
This PR adds a burn rate visualization to the overview tab of the SLO
Detail page. This PR also includes a fix for fetching the index pattern
fields hook; it uses the DataViews service to fetch the fields instead
of the internal API.
<img width="1170" alt="image"
src="41057791-880e-4cc8-a0c7-02a0f18aaeca">
### All good
<img width="1141" alt="image"
src="3ec07efa-e35a-4251-87f3-7ddc836171b7">
### Degrading
<img width="1141" alt="image"
src="a6d347be-7b55-404e-99a1-14ad4a38ad36">
### EVERYTHING IS BURNING 🔥
<img width="1141" alt="image"
src="9ed05875-b907-4a57-8387-a094876dd35e">
### Recovering in the dark
<img width="1151" alt="image"
src="f2999c7a-f97b-474c-8146-4565445df892">
### No data
<img width="1141" alt="image"
src="675a65a4-91b1-4de3-9f51-b65760efbb66">
Resolves https://github.com/elastic/kibana/issues/159948
## 📝 Summary
This PR updates the SLO form to support the calendar aligned time
windows for both create and edit flow.
I've also moved the budgeting method selector down, so when selecting
"timeslices", the timeslices related inputs are shown next to it on the
same line.
| Screenshot | Screenshot |
|--------|--------|
|
|
|
PR https://github.com/elastic/kibana/pull/155372 moved our error utils
to a package and also made a few small code changes, one of which added
`isPopulatedObject` to the error object type guards.
`isPopulatedObject` uses `Object.keys` under the hood which cannot be
used to access the non-enumerable properties of an object, like Error's
`message`.
This PR reverts all of these functions back to their original versions
which had existed in ML for a while without issue.
This was change had been causing error messages to not display
correctly.
vs
## [Security Solution] [Elastic AI Assistant] Data anonymization
The PR introduces the _Data anonymization_ feature to the _Elastic AI Assistant_:
_Above: Data anonymization in the Elastic AI Assistant_
_Above: Viewing the anonymized `host.name`, `user.name`, and `user.domain` fields in a conversation_
Use this feature to:
- Control which fields are sent from a context to the assistant
- Toggle anonymization on or off for specific fields
- Set defaults for the above
### How it works
When data anonymization is enabled for a context (e.g. an alert or an event), only a subset of the fields in the alert or event will be sent by default.
Some fields will also be anonymized by default. When a field is anonymized, UUIDs are sent to the assistant in lieu of actual values. When responses are received from the assistant, the UUIDs are automatically translated back to their original values.
- Elastic Security ships with a recommended set of default fields configured for anonymization
- Simply accept the defaults, or edit any message before it's sent
- Customize the defaults at any time
### See what was actually sent
The `Show anonymized` toggle reveals the anonymized data that was sent, per the animated gif below:
_Above: The `Show anonymized` toggle reveals the anonymized data_
### Use Bulk actions to quickly customize what's sent
_Above: bulk actions_
Apply the following bulk actions to customize any context sent to the assistant:
- Allow
- Deny
- Anonymize
- Unonymize
### Use Bulk actions to quickly customize defaults
_Above: Customize defaults with bulk actions_
Apply the following bulk actions to customize defaults:
- Allow by default
- Deny by default
- Anonymize by default
- Unonymize by default
### Row actions
_Above: The row actions overflow menu_
The following row actions are available on every row:
- Allow
- Deny
- Anonymize
- Unonymize
- Allow by default
- Deny by default
- Anonymize by default
- Unonymize by default
### Restore the "factory defaults"
The _Anonymization defaults_ setting, shown in the screenshot below, may be used to restore the Elastic-provided defaults for which fields are allowed and anonymized:
_Above: restoring the Elastic defaults_
See epic <https://github.com/elastic/security-team/issues/6775> (internal) for additional details.
## Summary
<p align="center">
<img width="700" src="3edf2101-718c-4716-80f6-c8377e66f0b9" />
</p>
Adds the following new abilities to the Security Assistant:
- Adds ability to create/delete custom SystemPrompts
- Configurable `Name`, `Prompt`, `Default Conversations`, and `Default for New Conversations`
- Introduces `System Prompt` setting within `Conversation Settings`
- Adds ability to create/delete custom Conversations
- Create conversation in-line within the Conversation selector by just typing the new conversation name and pressing enter
- Applies configured SystemPrompt and default connector on conversation creation
- Extracts `baseSystemPrompts` so they can be provided to the AssistantContextProvider on a per solution basis. The consolidates assistant dependency defaults to the `x-pack/plugins/security_solution/public/assistant/content` and `x-pack/packages/kbn-elastic-assistant/impl/content` directories respectively.
- All Security SystemPrompts now organized in `BASE_SECURITY_SYSTEM_PROMPTS`
- All Security Conversations organized in `BASE_SECURITY_CONVERSATIONS`
See epic https://github.com/elastic/security-team/issues/6775 (internal) for additional details.
### Checklist
Delete any items that are not applicable to this PR.
- [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
https://github.com/elastic/security-team/issues/6479
In this PR we're separating the types for the `Timeline` saved object
and the type of it's API response equivalent. Doing so will allow us to
make changes to either representations individually which is necessary
for versioning the SO and routes (individually) in the future.
The saved object definition of timeline and its API equivalent now live
in `*/saved_object.ts` and `*/api.ts` respectively. A clean cut of these
two, now distinct types. You will encounter a lot of duplication of
types in these files which is unavoidable. In the future, depending on
how both representations evolve when versioned, these two definitions
will diverge.
You will notice that only few types (and values) defined in
`saved_object.ts` are exported. They are only used for the conversion
logic. They are not exported so they're not accidentally required by
frontend or server code that is not dealing with the conversion. The
exported types all start with `SavedObject*` to clearly mark them as SO
representations.
The conversion files (`convert_saved_object_to_savedtimeline.ts` and
`security_solution/server/lib/timeline/saved_object/timelines/index.ts`)
have been updated to use the new representations and there is no
implicit conversion between them (e.g. through spreading or rest
parameters). In some places, an explicit conversion of fields was
necessary (e.g. to translate between timeline types).
The bulk of the changes are updates of `import` statements to change the
import to `**/api.ts`. If you are on a security team other than the
investigations team, you're most likely only required to look at those
import changes :)
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
To allow for more fine grained control of the baseline and deviation
time ranges for `autoAnalysisStart` this PR allows to pass in a
`WindowParameters` object as an alternative to the plain timestamp. When
more useful metadata is available this might lead to better selections
then the default one provided by `getWindowParameters`.
## Summary
Makes version headers required for internal endpoints. We also require
version headers for public endpoints when in dev mode.
### Note to reviewers
This PR is a re-revert of the original
https://github.com/elastic/kibana/pull/158667 with some minor additions
(see comments).
The original was reverted due to failing Cypress tests blocking Kibana
promotion for 8.8.1 (CC @stephmilovic,
https://github.com/elastic/kibana/pull/158961)
Not sending headers to versioned, internal endpoints will return 400!
Due to the somewhat sensitive nature of this change, I went through all
of the existing `.versioned` endpoints and tried to ensure that for
_internal_ endpoints we send through a version as this is now
**required**.
I would greatly appreciate it if code owners could check their code,
think of any existing consumers of your versioned endpoints and ensure
they are sending a version.
Closes https://github.com/elastic/kibana/issues/158722
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
