# Backport
This will backport the following commits from `main` to `8.8`:
- [[Security Solution] getDataViewStateFromIndexFields was using wrong
type as part of a cast
(#158594)](https://github.com/elastic/kibana/pull/158594)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Kevin
Qualters","email":"56408403+kqualters-elastic@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-05-31T21:13:36Z","message":"[Security
Solution] getDataViewStateFromIndexFields was using wrong type as part
of a cast (#158594)\n\n## Summary\r\n\r\nFixes an issue with the field
browser where all types currently display\r\nas unkown, this was because
in a code path where a type cast happens, we\r\nwere using the wrong
type. To see this, remove the as unknown from the\r\ncast, and the
typescript compiler will show the problem:\r\n```\r\n'BrowserField' is
deprecated.ts(6385)\r\nindex.ts(70, 4): The declaration was marked as
deprecated here.\r\nConversion of type 'DataViewField' to type
'BrowserField' may be a mistake because neither type sufficiently
overlaps with the other. If this was intentional, convert the expression
to 'unknown' first.\r\n Type 'DataViewField' is missing the following
properties from type 'BrowserField': category, description, example,
fields, and 2 more.ts(2352)\r\n```\r\nDataViewField actually only has
spec and kbnFieldType properties, spec\r\nis of type FieldSpec which is
basically the same type as BrowserField,\r\nand has sufficient overlap
for the (still unsafe, but more safe than as\r\nunknown) cast to
occur.\r\n\r\nBefore:\r\n<img width=\"338\"
alt=\"image\"\r\nsrc=\"f31c1f9e-25f0-41ee-9e1c-a70171e41d29\">\r\n\r\n\r\nAfter:\r\n<img
width=\"555\"
alt=\"image\"\r\nsrc=\"8b462477-2dce-41bb-9592-f34b20634b84\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1c75903f92b639e2dcffe76ed8b4ef4d6db3b70d","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Team:Threat
Hunting:Investigations","v8.9.0","v8.8.1"],"number":158594,"url":"https://github.com/elastic/kibana/pull/158594","mergeCommit":{"message":"[Security
Solution] getDataViewStateFromIndexFields was using wrong type as part
of a cast (#158594)\n\n## Summary\r\n\r\nFixes an issue with the field
browser where all types currently display\r\nas unkown, this was because
in a code path where a type cast happens, we\r\nwere using the wrong
type. To see this, remove the as unknown from the\r\ncast, and the
typescript compiler will show the problem:\r\n```\r\n'BrowserField' is
deprecated.ts(6385)\r\nindex.ts(70, 4): The declaration was marked as
deprecated here.\r\nConversion of type 'DataViewField' to type
'BrowserField' may be a mistake because neither type sufficiently
overlaps with the other. If this was intentional, convert the expression
to 'unknown' first.\r\n Type 'DataViewField' is missing the following
properties from type 'BrowserField': category, description, example,
fields, and 2 more.ts(2352)\r\n```\r\nDataViewField actually only has
spec and kbnFieldType properties, spec\r\nis of type FieldSpec which is
basically the same type as BrowserField,\r\nand has sufficient overlap
for the (still unsafe, but more safe than as\r\nunknown) cast to
occur.\r\n\r\nBefore:\r\n<img width=\"338\"
alt=\"image\"\r\nsrc=\"f31c1f9e-25f0-41ee-9e1c-a70171e41d29\">\r\n\r\n\r\nAfter:\r\n<img
width=\"555\"
alt=\"image\"\r\nsrc=\"8b462477-2dce-41bb-9592-f34b20634b84\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1c75903f92b639e2dcffe76ed8b4ef4d6db3b70d"}},"sourceBranch":"main","suggestedTargetBranches":["8.8"],"targetPullRequestStates":[{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/158594","number":158594,"mergeCommit":{"message":"[Security
Solution] getDataViewStateFromIndexFields was using wrong type as part
of a cast (#158594)\n\n## Summary\r\n\r\nFixes an issue with the field
browser where all types currently display\r\nas unkown, this was because
in a code path where a type cast happens, we\r\nwere using the wrong
type. To see this, remove the as unknown from the\r\ncast, and the
typescript compiler will show the problem:\r\n```\r\n'BrowserField' is
deprecated.ts(6385)\r\nindex.ts(70, 4): The declaration was marked as
deprecated here.\r\nConversion of type 'DataViewField' to type
'BrowserField' may be a mistake because neither type sufficiently
overlaps with the other. If this was intentional, convert the expression
to 'unknown' first.\r\n Type 'DataViewField' is missing the following
properties from type 'BrowserField': category, description, example,
fields, and 2 more.ts(2352)\r\n```\r\nDataViewField actually only has
spec and kbnFieldType properties, spec\r\nis of type FieldSpec which is
basically the same type as BrowserField,\r\nand has sufficient overlap
for the (still unsafe, but more safe than as\r\nunknown) cast to
occur.\r\n\r\nBefore:\r\n<img width=\"338\"
alt=\"image\"\r\nsrc=\"f31c1f9e-25f0-41ee-9e1c-a70171e41d29\">\r\n\r\n\r\nAfter:\r\n<img
width=\"555\"
alt=\"image\"\r\nsrc=\"8b462477-2dce-41bb-9592-f34b20634b84\">\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"1c75903f92b639e2dcffe76ed8b4ef4d6db3b70d"}},{"branch":"8.8","label":"v8.8.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Original outline: https://github.com/elastic/kibana/issues/138181
Issues outlining the objective of this pr:
https://github.com/elastic/kibana/issues/142903 and
https://github.com/elastic/kibana/issues/142907
#### Overview
Since the data views plugin was introduced, maintaining our own apis for
fetching sourcerer saved objects (data views) and additional types has
become cumbersome and inefficient. The data views plugin provides both
an efficient caching of data view saved objects and a unified interface
for creating ad-hoc data views (see the changes to the `useFetchIndex`
hook in this PR) so that our code can now rely on a single type of saved
object to interface with when fetching data.
This PR is another step towards replacing sourcerer with the data view
picker provided by kibana platform (which benefits users by maintaining
consistency around data source selection UX) and additionally provides
benefits to developers in the security solution by allowing us to reduce
state-management complexity in components that rely on old
`indexPattern` types or data view types.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/151697
## Summary
In a previous [PR](https://github.com/elastic/kibana/pull/145581) we
started installing a context-specific component templates, index
templates and concrete write indices for framework alerts as data when
the `xpack.alerting.enableFrameworkAlerts` config flag is set to true.
In that PR we used a different naming pattern than what is used by the
rule registry for those resources. In this PR, we are aligning the
naming of these resources with the rule registry and installing these
resources on alerting plugin setup when `enableFrameworkAlerts: true`.
If the flag is set to false, the rule registry will continue to handle
this resource installation.
In this PR we are doing the following:
* Registering all rules currently registered with the rule registry with
the alerting framework. This registration allows the alerting framework
to build context specific component templates. Because this PR only
addresses resource installation, rules will continue to be registered
with the rule registry.
* When `enableFrameworkAlerts: true`:
* The framework installs the context specific component template with
the following naming convention: `.alerts-{context}.alerts-mappings`.
This matches what the rule registry currently installs so the transition
should be seamless
* The framework installs the context specific index template for the
`default` space with the following name:
`.alerts-{context}.alerts-default-index-template`. Space awareness will
be addressed in a followup PR. This matches the current rule registry
naming.This index template will reference
(1) ECS component template (if `useEcs: true`),
(2) context-specific component template,
(3) legacy alert component template and
(4) framework component template
where the legacy alert component template + framework component template
= technical component template (from the rule registry).
* The framework creates or updates the concrete write index for the
`default` space with the naming convention:
`.internal.alerts-{context}.alerts-default-000001`. Space awareness will
be addressed in a followup PR. This matches the current rule registry
naming.
* The installation of the index template & write index differs from the
rule registry in that it occurs on alerting plugin start vs the first
rule run.
* We modified the rule registry resource installer to skip installation
of these resources when `enableFrameworkAlerts: true`. In addition, it
will wait for the alerting resource installation promise so if a rule
runs before its resources are fully initialized, it will wait for
initialization to complete before writing.
## To Verify
The following rule registry contexts are affected:
`observability.apm`
`observability.logs`
`observability.metrics`
`observability.slo`
`observability.uptime`
`security`
For each context, we should verify the following:
`Note that if your rule context references the ECS mappings, there may
be differences in those mappings between main and this branch depending
on whether you're running main with enableFrameworkAlerts true or false.
These differences are explained in the summary of this prior PR:
https://github.com/elastic/kibana/pull/150384 but essentially we're
aligning with the latest ECS fields. In the instructions, I suggest
running main with enableFrameworkAlerts: true to minimize the
differences caused by ECS changes`
**While running `main` with `enableFrameworkAlerts: true`:**
1. Get the context specific component template `GET
_component_template/.alerts-{context}.alerts-mappings`
2. Create rule for this context that creates an alert and then
3. Get the index template `GET
_index_template/.alerts-{context}.alerts-default-index-template`
4. Get the index mapping for the concrete index: `GET
.internal.alerts-{context}.alerts-default-000001/_mapping`
**While running this branch with `xpack.alerting.enableFrameworkAlerts:
true` (with a fresh ES instance):**
5. Get the context specific component template `GET
_component_template/.alerts-{context}.alerts-mappings`
6. Get the index template `GET
_index_template/.alerts-{context}.alerts-default-index-template`
7. Get the index mapping for the concrete index: `GET
.internal.alerts-{context}.alerts-default-000001/_mapping`
Note that you should not have to create a rule that generates alerts
before seeing these resources installed.
**Compare the component templates**
Compare 1 and 5. The difference should be:
* component template from this branch should have `_meta.managed: true`.
This is a flag indicating to the user that these templates are system
managed and should not be manually modified.
**Compare the index templates**
Compare 3 and 6. The differences should be:
* index template from this branch should have `managed: true` in the
`_meta` fields
* index template from this branch should not have a `priority` field.
This will be addressed in a followup PR
* index template from this branch should be composed of
`.alerts-legacy-alert-mappings` and `.alerts-framework-mappings` instead
of `.alerts-technical-mappings` but under the hood, these mappings are
equivalent.
**Compare the index mappings**
Compare 4 and 7. The difference should be:
* index mappings from this branch should have `_meta.managed: true`.
### Verify that installed resources templates work as expected
1. Run this branch on a fresh ES install with
`xpack.alerting.enableFrameworkAlerts: true`.
2. Create a rule in your context that generates alerts.
3. Verify that there are no errors during rule execution.
4. Verify that the alerts show up in your alerts table as expected.
5. (For detection rules only): Run this branch with
`xpack.alerting.enableFrameworkAlerts: true` and verify rules in a
non-default space continue to create resources on first rule run and run
as expected.
6. (For detection rules only): Run this branch with
`xpack.alerting.enableFrameworkAlerts: true` and verify rule preview
continue to work as expected
### Verify that installed resources templates work with existing rule
registry resources.
1. Run `main` or a previous version and create a rule in your context
that generates alerts.
2. Using the same ES data, switch to this branch with
`xpack.alerting.enableFrameworkAlerts: false` and verify Kibana starts
with no rule registry errors and the rule continues to run as expected.
3. Using the same ES data, switch to this branch with
`xpack.alerting.enableFrameworkAlerts: true` and verify Kibana starts
with no alerting or rule registry errors and the rule continues to run
as expected.
4. Verify the alerts show up on the alerts table as expected.
5. (For detection rules only): Run this branch with
`xpack.alerting.enableFrameworkAlerts: true` and verify rules in a
non-default space continue to create resources on first rule run and run
as expected.
6. (For detection rules only): Run this branch with
`xpack.alerting.enableFrameworkAlerts: true` and verify rule preview
continue to work as expected
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Resolves [#143152](https://github.com/elastic/kibana/issues/143152)
### Observability changes
This changes is a result of removal some types from `timelines` plugin:
- cleaned up timelines plugin related types,
- replaced `Pick<ActionProps,'data' | 'eventId' | 'ecsData' |
'setEventsDeleted' >` with the props which were actually used:
```
data: TimelineNonEcsData[];
ecsData: Ecs;
eventId: string;
```
In this PR we still have references to `@kbn/timelines-plugin`, which
needs to be changed later.
Threat Hunting team are going to think about replacing
`TimelineNonEcsData` with the other type definition (maybe
`NonEcsData`?) and moving `Ecs` type to the non `timelines` related
plugin/package.
### Security Solution changes
Before the current PR changes the components dependencies around `TGrid`
looked like the image below:
<img width="848" alt="Screen Shot 2022-11-29 at 6 16 14 AM"
src="https://user-images.githubusercontent.com/55110838/204663019-664431fb-f360-4a11-b395-6fa54c35dd6d.png">
After decomposition the `timelines` plugin hosted TGrid HOC and moving
all the data tables related sub-components to `security_solution` plugin
the new components architecture got the next shape:
<img width="842" alt="Screen Shot 2022-11-29 at 6 14 41 AM"
src="https://user-images.githubusercontent.com/55110838/204663068-40897f18-1485-4b59-a71b-ce09e660f7db.png">
`security_solution` plugin changes includes the next things:
- Moved some data table and actions types to
`x-pack/plugins/security_solution/common/types`, which is widely used
across the related components.
- Due to the movement of the data table with the store to from
`timeline` plugin to `security_solution` many test files which had the
reference to `tGridReducer` now cleaned up from the unnecessary logic:
```
- import { tGridReducer } from '@kbn/timelines-plugin/public';
```
and `TableState` references was replaced with the next changes:
```
- import type { TableState } from '@kbn/timelines-plugin/public';
+ import type { TableState } from '../common/store/data_table/types';
```
- Replaced `tGridActions` with `dataTableActions` name.
- Moved `control_columns` to `security_solution` common plugin
components: `RowCheckBox`, `HeaderCheckBox` and
`transformControlColumns`:
`RowActionComponent` moved from `timelines` plugin to
`x-pack/plugins/security_solution/public/common/components/control_columns/row_action`
without changes.
`transformControlColumns` moved from timelines plugin to
`x-pack/plugins/security_solution/public/common/components/control_columns/transform_control_columns.tsx`.
Removed not used property `hasAlertsCrudPermissions`, added unit test.
<img width="1222" alt="Screen Shot 2022-11-29 at 8 59 42 PM"
src="https://user-images.githubusercontent.com/55110838/204711499-9f90fee2-3c2f-4ff6-af28-c324ab1840d8.png">
- Many translation changes as a result of the owner plugin change:
```
- i18n.translate('xpack.timelines....', {
+ i18n.translate('xpack.securitySolution....', {
```
- Moved `useDraggableKeyboardWrapper` to security_solution, added reference to `useAddToTimeline`, by using timelines plugin with kibana services. Added unit tests.
<img width="1112" alt="Screen Shot 2022-11-30 at 9 06 42 AM" src="https://user-images.githubusercontent.com/55110838/204862298-bcd50a52-dbf7-480b-bf13-8e48d6835746.png">
- Replaced the next references:
```
- type: 'x-pack/timelines/t-grid/UPDATE_COLUMN_WIDTH',
+ type: 'x-pack/security_solution/data-table/UPDATE_COLUMN_WIDTH',
```
```
- type: 'x-pack/timelines/t-grid/REMOVE_COLUMN',
+ type: 'x-pack/security_solution/data-table/REMOVE_COLUMN',
```
- moved TGrid store previously hosted in timeline plugin to `security_solution` as `data_table` store:
<img width="1109" alt="Screen Shot 2022-11-29 at 9 24 08 PM" src="https://user-images.githubusercontent.com/55110838/204714668-257a9c50-d722-4a6d-9214-f3ef8a14d0d2.png">
- Migrated TGrid `BodyComponent` to `DataTableComponent`
`x-pack/plugins/security_solution/public/common/components/data_table/index.tsx`
Removed some unused properties: `hasAlertsCrudPermissions, appId, getRowRenderer, isEventViewer, tableView, totalSelectAllAlerts, trailingControlColumns`. Current DataTableComponent is a subset of the previous BodyComponent, which includes only table related functionality:
<img width="1028" alt="Screen Shot 2022-11-30 at 10 44 35 AM" src="https://user-images.githubusercontent.com/55110838/204882561-0950b9ce-5a9f-4bdb-b38f-6ff742fc3f92.png">
- Renamed `TimelineExpandedDetail` to `ExpandedDetail` to make the type more generic for usage.
- BulkActions related changes includes:
<img width="1288" alt="Screen Shot 2022-11-29 at 9 13 32 PM" src="https://user-images.githubusercontent.com/55110838/204713196-409f3d5e-f752-4fe9-9ae9-e752514cbf99.png">
`AlertBulkActionsComponent` moved from timelines plugin to `x-pack/plugins/security_solution/public/common/components/toolbar/bulk_actions/alert_bulk_actions.tsx`, just renaming changes.
Added `x-pack/plugins/security_solution/public/common/components/toolbar/bulk_actions/types.ts` to consolidate types
`useBulkActionItems` moved from timelines plugin to `x-pack/plugins/security_solution/public/common/components/toolbar/bulk_actions/use_bulk_action_items.tsx`. Changed links, renamed `AlertsStatus` to `AlertWorkflowStatus`, removed `in-progress` case handling.
`useUpdateAlertsStatus` moved from timelines plugin to `x-pack/plugins/security_solution/public/common/components/toolbar/bulk_actions/use_update_alerts.ts`. Cleaned up the code from handling Observability API.
- Updated `x-pack/plugins/security_solution/public/common/lib/kuery/index.ts` with the actual implementations of
```
convertKueryToDslFilter,
convertKueryToElasticSearchQuery,
convertToBuildEsQuery,
escapeKuery,
escapeQueryValue,
combineQueries,
```
instead of referencing timelines plugin.
- Moved `EventRenderedView` component to security_solution common components. Later planning to make it as a package.
- `EventsViewer` component became the stateful component which is responsible for the data representation managing. Some part from TGridIntegratedComponent and BodyComponent was merged under its logic:
<img width="1052" alt="Screen Shot 2022-11-30 at 6 22 22 PM" src="https://user-images.githubusercontent.com/55110838/204950708-a8875acd-eb62-4df5-8ac4-613a0a571de6.png">
<img width="242" alt="Screen Shot 2022-11-30 at 6 24 06 PM" src="https://user-images.githubusercontent.com/55110838/204950819-bca194a4-4309-4cb4-a2ba-0176e9fe6c65.png">
- Moved header actions to common components `x-pack/plugins/security_solution/public/common/components/header_actions`
- Renamed component `AlertCount` to `UnitCount`.
- Moved to `security_solution` configuration for `APM_USER_INTERACTIONS`
- changes `createStore` interface by using the direct reference to `dataTableReducer` instead of passing down it's value through the params.
### Timeline plugin changes
- cleaned up timeline plugin interface by removing:
```
getTGrid: <T extends TGridType = 'embedded'>(
props: GetTGridProps<T>
) => ReactElement<GetTGridProps<T>>;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
getTGridReducer: () => any;
getUseDraggableKeyboardWrapper: () => (
props: UseDraggableKeyboardWrapperProps
) => UseDraggableKeyboardWrapper;
```
- renamed embedded store
```
- setTGridEmbeddedStore: (store: Store) => void;
+ setTimelineEmbeddedStore: (store: Store) => void;
```
- removed dependency to triggers_actions_ui plugin
- removed duplicated components and types with `security_solution`:
```
TruncatableText
SubtitleComponent
EventsCountComponent
PopoverRowItems
PagingControlComponent
FooterComponent
SortIndicator
SortNumber
RowRendererContainer
plainRowRenderer
getColumnRenderer
StatefulRowRenderer
getMappedNonEcsValue
InspectButtonComponent
TGridCellAction
useMountAppended
tgrid store
```
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
https://github.com/elastic/kibana/issues/145544 - Suppression icon
should show up in rule name even when
`kibana.alert.suppression.docs_count` column is not included in the
table
https://github.com/elastic/kibana/issues/145669 - Rule name cell popover
formatting
No issue - adds the rule name icon for rule preview table
`props.data` is the fetched columns, `props.ecsData` always has the
fields listed in `requiredFieldsForActions` so we can use
`kibana.alert.suppression.docs_count` even when that column is missing.
## Summary
This pr adds a new parsing plugin to the EuiMarkdownEditor used in
security solution that enables users to create run time queries that can
be parameterized from alert data, or hard coded literal values. A count
of the matching events is displayed in a button that when clicked will
open the same event set in timeline. Markdown is expected to be in the
following format:
`!{insight{"description":"2 top level OR providers, 1 nested
AND","label":"test insight", "providers": [[{ "field": "event.id",
"value": "kibana.alert.original_event.id", "type": "parameter" }], [{
"field": "event.category", "value": "network", "type": "literal" },
{"field": "process.pid", "value": "process.pid", "type":
"parameter"}]]}}`
The 2d array is used to allow nested queries, the top level arrays are
OR'ed together, and the inner array AND'ed together:
<img width="438" alt="image"
src="https://user-images.githubusercontent.com/56408403/201940553-96ab3d39-48fa-404f-ab2e-8946b532567b.png">
Following a prefix of !insight, the configuration object takes optional
description and label strings, along with a 2 dimensional array called
"providers". This value corresponds to what are called data providers in
the timeline view,
and are arrays of filters with 3 fields, "field" which is the field name
for that part of the query clause, "value" which is the value to be
used, and "type" which is either "parameter" or "literal". Filters of
type parameter expect value to be the name of a field present in an
alert document, and will use the value in the underlying document if
found. If the field is not present for some reason, a wildcard is used.
If the markdown is rendered in a context not tied to a specific alert,
parameter fields are treated as a timeline template field.
<img width="632" alt="image"
src="https://user-images.githubusercontent.com/56408403/201940922-7114a75f-0430-4397-8384-59f4e960ec9c.png">
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
## Summary
This PR adds support for an is one of operator allowing users to filter
multiple values for one field.
[Some investigation
](https://discuss.elastic.co/t/passing-multiple-values-in-kibana-add-filter-is-one-of/232694/2)by
@andrew-goldstein revealed that since the underlying engine uses Lucene,
we can add support for multiple values by using an OR query:
`kibana.alert.workflow_status: ("open" OR "closed" OR "acknowledged")`
is equivalent to
```
"terms": {
"kibana.alert.workflow_status": [ "open", "closed", "acknowledged"]
}
```
Where the former is usable in our `DataProviders` used by timeline and
other components that navigate a user to a pre-populated timeline.
As an enhancement to the timeline view, users can also use this `is one
of` operator by interacting with the `Add field` button and selecting
the new operator.
<img width="433" alt="image"
src="https://user-images.githubusercontent.com/28942857/193487154-769005b6-3e5a-40bf-9476-8dd3f3bcb8ee.png">
### Checklist
Delete any items that are not applicable to this PR.
- [X] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [X] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Known issues
This operator does not support timeline templates at this time so usage
there disables the ability for conversion to template field but a better
approach should be implemented to notify users.
https://github.com/elastic/kibana/issues/142437. For now I have added a
template message and prevented users from creating templates with this
operator:
<img width="374" alt="image"
src="https://user-images.githubusercontent.com/28942857/201157676-80017c6c-9f5b-4cd7-ba0b-ee2e43a884cb.png">
## Testing
Create a new timeline or visit an existing one.
Click 'Add field' button on Timeline in OR query section
add any field ( preferably one that can have many values- consider
`kibana.alerts.workflow_status` but this requires alerts.
Select the `is one of` or `is not one of operator`
Add or remove values in the value section.
Click save.
Co-authored-by: Kristof-Pierre Cummings <kristofpierre.cummings@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR contains fixes for the following issues:
#### # 1 Popover overlaps flyout
- https://github.com/elastic/kibana/issues/139280
- https://github.com/elastic/kibana/issues/128235
#### # 2 Popover persists after clicking filter out
- https://github.com/elastic/kibana/issues/115341
#### # 3 Popover persists after clicking a button outside of popover
- https://github.com/elastic/kibana/issues/118844
## Background
Previously, a cell's popover remains open after clicking an action. In
many cases we want the popover to close upon clicking on a cell action.
EUI team addressed this by adding a `closeCellPopover` to a `ref` API.
- https://github.com/elastic/eui/pull/5590
In T-grid, there are 2 types of cell actions:
- Default cell actions such as filter in, filter out, add to timeline
and copy. `closeCellPopover` is not used.
- Formatted fields that have more information in the form of flyouts
(host name, user name, ip, etc.)
`closeCellPopover` prop is passed but currently not working as expected.
This PR contains fixes for:
- Fixing `closeCellPopover` in T-grid body for formatted fields - fixes
# 1
- Adding `closeCellPopover` props in default cell actions - fixes # 2
and # 3
## # 1 - `closeCellPopover` in T-grid
`dataGridRef.current?.closeCellPopover` was added and intended to close
any open popovers when a cell action is clicked. However, because it is
a mutable object, it is not being monitored in `columnsWithCellActions`.
When the page is initially loaded, `dataGridRef.current` remain as null
and it does not update until the page re-renders and `dataGridRef`
becomes non-null.
- After: popover closes properly
https://user-images.githubusercontent.com/18648970/201202326-ec657f78-c425-46a6-9356-f6e9ef1ab798.mov
## # 2 & # 3 Add `closeCellPopover` to default cell actions
- After: upon opening the expansion popover, clicking any options and
the popover will disappear
https://user-images.githubusercontent.com/18648970/201417542-063c514b-5474-4676-a747-a9401627c5e8.mov
- After: upon opening the expansion popover, clicking any options
outside and the popover will disappear
https://user-images.githubusercontent.com/18648970/201417678-7cf0fefa-f4a7-4a70-9a10-76b248323639.mov
Note for UX: although QA only flagged `filter out` and `add to
timeline`, for consistency's sake, the expansion popover will disappear
after clicking any of the cell actions, which includes `filter in` and
`copy`.
* Remove External alert trend table and artifacts, and rename detection… (#136579)
* Remove External alert trend table and artifacts, and rename detections alert
* add test for SignasByCategory
* Update signals_by_category.test.tsx
* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'
Co-authored-by: Kristof-Pierre Cummings <kristofpierre.cummings@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* update event tab to show both alerts and events with toggle. (#136540)
* add test for SignasByCategory
* modify external_alerts_filter to be more efficient
* Update usage across explore views to only use EventsQueryTabBody
* remove unused files and code related to external alerts and move old alerts files to events_tab folder
* test fixes, and more removal of old usage
* update failing snapshots
* last bit of cleanup
* Fix type error
* fix type and translations issue
Co-authored-by: Kristof-Pierre Cummings <kristofpierre.cummings@elastic.co>
* translations fixed
* fix default stackBy value for alerts bug
* memoizations added
Co-authored-by: Kristof-Pierre Cummings <kristofpierre.cummings@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: semd <sergi.massaneda@elastic.co>
* [Timelines T-Grid] Changed formatting of the timeline data to parse all nested objects under the "kibana.alert.rule.parameters"
* Excluded some of the alert params fields from parsing, because this is expected by the design
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* reduce data plugin reexporting of data view exports
* reduce data plugin reexporting of data view exports
* cleanup
* Apply suggestions from code review
Co-authored-by: Dima Arnautov <arnautov.dima@gmail.com>
Co-authored-by: Dima Arnautov <arnautov.dima@gmail.com>
## [Security Solution] Fixes sorting issues related to unmapped fields
This PR fixes the following issues related to sorting unmapped fields in timelines and the events / alerts tables:
- <https://github.com/elastic/kibana/issues/129603>
- <https://github.com/elastic/kibana/issues/123912>
- <https://github.com/elastic/kibana/issues/131625>
The `unmapped_type` property [addition](https://github.com/elastic/kibana/pull/87241/files#diff-52fd5870dcd5f783f9fc8ac3a18a8674d83ac6136e09fe0e0bcae30427d61c3fR55) to the `sort` parameter of requests was using the `type` field metadata from `BrowserFields`, but the `type` metadata (for some fields) contains the value `string`, which is not a [valid field data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html).
The fix for the issues above:
- Populates the `sort` property of requests with values from the `esTypes` `BrowserFields` metadata (instead of `type`)
- The `esTypes` metadata may specify more than one field value type. When `esTypes` contains more than one type, and `keyword` is one of the types, the `sort` property of the request will prefer `keyword` over other the other types
- When the field metadata has an empty `esTypes` collection, the `sort` property of the request will default to using `"unmapped_type": "keyword"`
- The field type displayed in tooltips when hovering over columns in a timeline now displays values from `esTypes` instead of `type`
### Desk testing
To reproduce issue <https://github.com/elastic/kibana/issues/129603> and to verify the fix:
1) Open Kibana `Dev tools`
2) Execute the following query to delete any exiting `logs-ti_test` index:
```
DELETE logs-ti_test
```
3) Execute the following query to create an index named `logs-ti_test`, which has the following properities:
- Dynamic mappings are disabled via `"dynamic": false`
- It does NOT contain a mapping for `event.action` (we will sort by this field in later steps)
- It contains a mapping for the non-ECS `testing` field
```
PUT logs-ti_test
{
"mappings": {
"dynamic": false,
"properties": {
"@timestamp": {
"type": "date"
},
"event": {
"properties": {
"category": {
"type": "keyword"
},
"dataset": {
"type": "keyword"
},
"kind": {
"type": "keyword"
},
"type": {
"type": "keyword"
}
}
},
"host": {
"properties": {
"name": {
"type": "keyword"
}
}
},
"testing": {
"type": "keyword",
"ignore_above": 1024
},
"threat": {
"properties": {
"indicator": {
"properties": {
"file": {
"properties": {
"hash": {
"properties": {
"md5": {
"type": "keyword"
}
}
}
}
}
}
}
}
}
}
}
}
```
4) Execute the following query to add a new document to the `logs-ti_test` index, and note that:
- It does NOT contain a `event.action` field
- It contains a value for the non-ECS `testing` field
```
POST logs-ti_test/_doc/
{
"@timestamp": "2022-05-12T00:00:14.725Z",
"host": {
"name": "foozle"
},
"threat": {
"indicator": {
"file": {
"hash": {
"md5": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f"
}
}
}
},
"event": {
"kind": "enrichment",
"type": "indicator",
"dataset": "ti_*",
"category": "threat"
},
"testing": "simulated threat intel data"
}
```
5) Navigate to the Security > Hosts page
6) Select `Last 1 year` from the date picker
7) Click the `Events` tab
8) Enter the following KQL query in the search bar at the top of the page:
```
host.name: foozle
```
9) Hover over the `foozle` entry in the `host.name` column in the Events table, and click the `Add to timeline investigation` cell action
10) Open the timeline
11) Hover over the `event.action` field
**Expected result**
- The tooltip displays type `keyword` for the `event.action` field
**Actual result**
- The tooltip displays type `string` for the `event.action` field
12) Click the `event.action` column to add a secondary sort
**Expected result**
- The table is sorted by `@timestamp` and `event.action`
- The table contents are (still) visible
**Actual result**
- The table is sorted by `@timestamp` and `event.action`
- The contents of the table are now empty
13) Click the timeline's `Inspect` button
14) In the `Inspect Timeline` dialog, click the `Request` tab
15) Scroll down to the `sort` property of the request
**Expected result**
- The `event.action` field contains a `"unmapped_type": "keyword"` property, per the example below:
```json
"sort": [
{
"@timestamp": {
"order": "desc",
"unmapped_type": "date"
}
},
{
"event.action": {
"order": "desc",
"unmapped_type": "keyword"
}
}
],
```
**Actual result**
- The request's `event.action` field contains a `"unmapped_type": "string"` property, per the example below:
```json
"sort": [
{
"@timestamp": {
"order": "desc",
"unmapped_type": "number"
}
},
{
"event.action": {
"order": "desc",
"unmapped_type": "string"
}
}
],
```
16) In the `Inspect Timeline` dialog, click the `Response` tab
**Expected result**
- The response contains `0` `failed` shards / no failures
**Actual result**
- The response contains failures for the `logs-ti_test` index, with the following reason:
```
"reason": "No mapper found for type [string]"
```
per the example below:
```json
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 4,
"successful": 3,
"skipped": 0,
"failed": 1,
"failures": [
{
"shard": 0,
"index": "logs-ti_test",
"node": "NCRcGeDqSlKQiuPWVFvMEg",
"reason": {
"type": "illegal_argument_exception",
"reason": "No mapper found for type [string]"
}
}
]
},
```
* session tab query modified query all events, not just entry leaders. solves a few problems wrt to query ability. default columns modified and display names provided for each
* snapshot updated
* readded test
* Default sort set to process.entry_leader.start desc
* sessions tab timeline id changed to cache bust localstorage for table column configs
* missed a couple spots for session tab timeline id update
Co-authored-by: mitodrummer <karlgodard@elastic.co>
## [Security Solution] [Investigations] [Tech Debt] removes redundant code from the timelines plugin
This follow-up PR removes redundant code from the `timelines` plugin, identified while implementing https://github.com/elastic/kibana/pull/130740
* Upgrade EUI to v54.0.0
* [Discover] Remove deprecated closePopover call
- for closeCellPopover ref API
* [Lens] Remove deprecated closePopover call
- for closeCellPopover ref API
* [Security/Timelines] Remove deprecated closePopover call
- for closeCellPopover ref API
* [Security/Timeline] Update Timeline datagrid to accept/pass `visibleCellActions` prop
+ update Security to show 3 visible cell actions
* [APM] Account for removed EUI theme avatar sizes
* Update emotion dependencies to latest
* Remove styles from being rendered in emotion serializer
* Update snapshots affected by emotion serializer `includeStyles: false` change
* Update snapshot changes caused by EuiFormControlLayout changes
* Update snapshot changes caused by EuiAvatar CSS-in-JS conversion
* consolidate yarn.lock
* [Spaces] Fix failing test due to new EuiAvatar emotion wrapper
- which, due to mount() causes .first() to no longer work as expected - targeting .last() instead gets the actual div element which works
* [Security] Fix cell expansion popover actions
- EUI added 2 `.euiPopoverFooter`s for overflowing cell actions, and Security's CSS to hide the first 2 cell actions (replaced by their own custom cell actions) was unintentionally affecting other actions
* Clean up spaces test snapshots
* [Security feedback] Revert 793d208 and hard-code visibleCellActions
Co-authored-by: Greg Thompson <thompson.glowe@gmail.com>
Co-authored-by: Joe Portner <joseph.portner@elastic.co>
* Add Events tab to the User page and the User details page
* Add External alerts tab to the User page and the User details page
* Add cypress tests
* Add unit test to EventsQueryTabBody
* Memoize navTabs on Users page
* field browser first revamp implementation
* customize columns for security solution alert tables
* cleaning
* some tests
* clean unused code
* field browser tests created and existing fixed
* security solution test fixes
* translations cleaned
* fix test
* adapt cypress tests
* remove translation
* fix typo
* remove duplicated test
* type error fixed
* enable body vertical scroll for small screens
* fix new field not added to the table bug
* addapt Kevin performance improvement
* fixed linter error
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Bootstrap user page
* Bootstrap user details page
* Delete ueba
* Create User detail flyout
* Add cypress test to User page
* Add Sourcerer to the users page
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Add flattend parameters object and populate it in Security Solution
* Fix severity, risk_score, bugs, tests
* Add ALERT_RULE_PARAMETERS to package
* Skip tightly coupled test
* fix more tests
* Remove unused import
* Fix threat matching API test
* Continue overriding kibana.alert.rule.risk_score and severity for now
* Add ignore_above to ALERT_RULE_PARAMETERS
* Exploratory
* Not pretty
* more garbage
* debugging
* use expandDottedObject for alerts data in UI
* Remove kibana.alert.rule.risk_score and severity
* Fix tests related to risk_score and severity
* Make translation a template
* Can't use expression in template literal
* Remove commented line added by bad merge
* Fix linting
* Fix unflattening of UI data
* Fix mapping
* Remove console logs
* Fix imports
* Clean up, fix dupes
* Remaining test and type errors
* Remove comment
* Fix skip param
* Add backcompat for threshold timeline
* Fix linting
* Use indexNames for threshold timeline instead of data view
* Add tests for threshold timeline action
* Implement suggestion for simplified alertIds initialization
Co-authored-by: Marshall Main <marshall.main@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* chore(NA): splits types from code on @kbn/rule-data-utils
* chore(NA): remove old style imports for this pkg
* chore(NA): eslint fix
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* add kibana.alert.rule.parameters as a flattened type
* temp
* rule_data_formatter
* fix bug in search strategy with flattend field type where prefix was wrong (kibana.alert.rule.parameters was ignored)
* fix inventory rule data formatters
* remove console log
* hack that prepends kibana.alerts.rule.parameters in the nested subfields
* import ALERT_RULE_PARAMETERS from kbn rule data utils
* remove console log
* format custom metric link
* remove ALERT_PARAMS from technical field names
* fix bug in timelines plugin to use dotField instead of prependField & fix failing tests
* remove console log and unused variable
* delete kibana.alert.rule.params from the mapping
* flatten kibana.alert.rule.parameters and add some unit tests
* fix rule_data_formatter
* handle scenario of having multiple items in an array (multiple conditions setup in the rule)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
See: https://github.com/elastic/kibana/issues/110903
This removes all the top level API `export *` spots from:
* `timeline` plugin within both the common and public section
This reduces the number of metrics and warning about undocumented functions.
I also add this text to timeline:
```
// Careful of exporting anything from this file as any file(s) you export here will cause your page bundle size to increase.
// If you're using functions/types/etc... internally or within integration tests it's best to import directly from their paths
// than expose the functions/types/etc... here. You should _only_ expose functions/types/etc... that need to be shared with other plugins here.
// When you do have to add things here you might want to consider creating a package to share with
// other plugins instead as packages are easier to break down and you do not have to carry the cost of extra plugin weight on
// first download since the other plugins/areas of your code can directly pull from the package in their async imports.
// See: https://docs.elastic.dev/kibana-dev-docs/key-concepts/platform-intro#public-plugin-api
```
