# Backport
This will backport the following commits from `main` to `8.8`:
- [[Security Solution] Improve rules exception flyout opening for the
indices with huge amount of fields
(#159216)](https://github.com/elastic/kibana/pull/159216)
<!--- Backport version: 8.9.7 -->
### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)
<!--BACKPORT [{"author":{"name":"Ievgen
Sorokopud","email":"ievgen.sorokopud@elastic.co"},"sourceCommit":{"committedDate":"2023-06-15T12:57:15Z","message":"[Security
Solution] Improve rules exception flyout opening for the indices with
huge amount of fields (#159216)\n\n## Summary\r\n\r\nOriginal
ticket:\r\n[#158751](https://github.com/elastic/kibana/issues/158751)\r\n\r\nThese
changes improve the rule's exceptions flyout opening experience.\r\nWe
had a few complaints that it is very slow to open it and sometimes
it\r\nthrows an exception about the limited response size.\r\n\r\nTo fix
this, we decided to load extended field's data (conflicts
and\r\nunmapped info) only when user selects some field instead of
fetching\r\nthis data for all fields on flyout opening.\r\n\r\n##
NOTES:\r\n\r\nAfter these changes we gonna do next steps related to
fields loading\r\nwhen user creates/edits rule exceptions:\r\n1. We will
call `_fields_for_wildcard` **WITHOUT**\r\n`include_unmapped=true`
parameter to fetch all fields specs on exception\r\nflyout loading\r\n2.
We will call `_fields_for_wildcard` **WITH**
`include_unmapped=true`\r\nfor only one field when user selects it from
the dropdown menu\r\n\r\nWith these changes we will improve slow
exception flyout opening when\r\nuser has lots of fields which are
unmapped in different indices. If for\r\nsome reason user has a lot of
(thousands) conflicting fields around\r\nindices then the loading is
still might be slow as the\r\n`_fields_for_wildcard` call will return
conflicts information even\r\nwithout `include_unmapped=true`
parameter.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"31b34771c5e6f710858a7f617bbca04537cf5c1b","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:fix","Team:
SecuritySolution","ci:cloud-deploy","v8.9.0","Team:Detection
Engine","v8.8.2"],"number":159216,"url":"https://github.com/elastic/kibana/pull/159216","mergeCommit":{"message":"[Security
Solution] Improve rules exception flyout opening for the indices with
huge amount of fields (#159216)\n\n## Summary\r\n\r\nOriginal
ticket:\r\n[#158751](https://github.com/elastic/kibana/issues/158751)\r\n\r\nThese
changes improve the rule's exceptions flyout opening experience.\r\nWe
had a few complaints that it is very slow to open it and sometimes
it\r\nthrows an exception about the limited response size.\r\n\r\nTo fix
this, we decided to load extended field's data (conflicts
and\r\nunmapped info) only when user selects some field instead of
fetching\r\nthis data for all fields on flyout opening.\r\n\r\n##
NOTES:\r\n\r\nAfter these changes we gonna do next steps related to
fields loading\r\nwhen user creates/edits rule exceptions:\r\n1. We will
call `_fields_for_wildcard` **WITHOUT**\r\n`include_unmapped=true`
parameter to fetch all fields specs on exception\r\nflyout loading\r\n2.
We will call `_fields_for_wildcard` **WITH**
`include_unmapped=true`\r\nfor only one field when user selects it from
the dropdown menu\r\n\r\nWith these changes we will improve slow
exception flyout opening when\r\nuser has lots of fields which are
unmapped in different indices. If for\r\nsome reason user has a lot of
(thousands) conflicting fields around\r\nindices then the loading is
still might be slow as the\r\n`_fields_for_wildcard` call will return
conflicts information even\r\nwithout `include_unmapped=true`
parameter.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"31b34771c5e6f710858a7f617bbca04537cf5c1b"}},"sourceBranch":"main","suggestedTargetBranches":["8.8"],"targetPullRequestStates":[{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/159216","number":159216,"mergeCommit":{"message":"[Security
Solution] Improve rules exception flyout opening for the indices with
huge amount of fields (#159216)\n\n## Summary\r\n\r\nOriginal
ticket:\r\n[#158751](https://github.com/elastic/kibana/issues/158751)\r\n\r\nThese
changes improve the rule's exceptions flyout opening experience.\r\nWe
had a few complaints that it is very slow to open it and sometimes
it\r\nthrows an exception about the limited response size.\r\n\r\nTo fix
this, we decided to load extended field's data (conflicts
and\r\nunmapped info) only when user selects some field instead of
fetching\r\nthis data for all fields on flyout opening.\r\n\r\n##
NOTES:\r\n\r\nAfter these changes we gonna do next steps related to
fields loading\r\nwhen user creates/edits rule exceptions:\r\n1. We will
call `_fields_for_wildcard` **WITHOUT**\r\n`include_unmapped=true`
parameter to fetch all fields specs on exception\r\nflyout loading\r\n2.
We will call `_fields_for_wildcard` **WITH**
`include_unmapped=true`\r\nfor only one field when user selects it from
the dropdown menu\r\n\r\nWith these changes we will improve slow
exception flyout opening when\r\nuser has lots of fields which are
unmapped in different indices. If for\r\nsome reason user has a lot of
(thousands) conflicting fields around\r\nindices then the loading is
still might be slow as the\r\n`_fields_for_wildcard` call will return
conflicts information even\r\nwithout `include_unmapped=true`
parameter.\r\n\r\n---------\r\n\r\nCo-authored-by: Kibana Machine
<42973632+kibanamachine@users.noreply.github.com>","sha":"31b34771c5e6f710858a7f617bbca04537cf5c1b"}},{"branch":"8.8","label":"v8.8.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Co-authored-by: Ievgen Sorokopud <ievgen.sorokopud@elastic.co>
## Summary
Adds the following:
- Add the option to duplicate from the shared exception list management
actions dropdowns
- User can select to include exception items with expired TTL
- User can select to not include exception items with expired TTL
- Cypress tests added for both options
👋 Hi all - the biggest breaking change of this PR is around two icon
type changes/renames.
1. ⚠️ **The `alert` icon is now named `warning`**
- <img width="103" alt=""
src="https://user-images.githubusercontent.com/549407/223561599-8913e88c-676f-47cd-aaed-81b64783bd81.png"
align="middle">
- This change should have been automatically converted on your behalf by
the EUI team, **but if for some reason** we missed making this
conversion in this PR and your icon(s) are now broken, please ping us or
let us know in this PR (or fix yourself after this PR merges).
- In some cases, teams were using this icon for error messages,
alongside the `danger` color. In those cases, we opinionatedly changed
those icon usages to the new `error` icon instead of using the old
alert/warning icon.
2. 🛑 **The `crossInACircleFilled` icon has been removed, and a new
`error` icon added**
- <img width="84" alt=""
src="https://user-images.githubusercontent.com/549407/223561892-4406bdf6-1a55-49ac-85ad-3a11eb7c090d.png"
align="middle">
- The conversion for this breaking change was not straightforward. This
was the path we used to determine what to change `crossInACircleFilled`
usages to:
- If the icon was associated with errors or error messages, we changed
it to the new `error` icon.
- If a "delete" action was associated with this icon, we changed it to
the `trash` icon instead.
- If a "clear" action was associated with this icon, we changed it to
just the `cross` icon, or in some cases `minusInCircleFilled` (if used
alongside `plusInCircleFilled`).
- Again, if we made a mistake during this conversion or missed your
plugin, please feel free to ping us.
## Summary
`eui@75.1.2` ⏩ `eui@76.0.2`
## [`76.0.2`](https://github.com/elastic/eui/tree/v76.0.2)
**Bug fixes**
- Added a legacy `alert` alias for the `warning` `EuiIcon` type
([#6640](https://github.com/elastic/eui/pull/6640))
## [`76.0.1`](https://github.com/elastic/eui/tree/v76.0.1)
**Bug fixes**
- Fixed broken icons on all `isInvalid` form controls
([#6629](https://github.com/elastic/eui/pull/6629))
## [`76.0.0`](https://github.com/elastic/eui/tree/v76.0.0)
- Added `pivot` glyph to `EuiIcon`
([#6605](https://github.com/elastic/eui/pull/6605))
- Added the `displayHeaderCellProps` API to `EuiDataGrid`'s columns,
which allows passing custom props directly to column header cells
([#6609](https://github.com/elastic/eui/pull/6609))
- Added the new `headerCellProps`/`footerCellProps` APIs to
`EuiDataGrid`'s control columns, which allows passing custom props
directly to control column header or footer cells
([#6609](https://github.com/elastic/eui/pull/6609))
- Added a new `footerCellRender` API to `EuiDataGrid`'s control columns,
which allows completely customizing control column rendering (previously
rendered an empty cell)
([#6609](https://github.com/elastic/eui/pull/6609))
- Updated the styling of nested ordered lists in `EuiText` to align with
GitHub's list style, which is a popular format used in Markdown or MDX
formatting ([#6615](https://github.com/elastic/eui/pull/6615))
- Added a margin-bottom property exclusively to the direct child `ul`
and `ol` elements of the `EuiText` component
([#6615](https://github.com/elastic/eui/pull/6615))
- Fix issue with badges appearing within an `EuiBadgeGroup`, where the
CSS rule to override the `margin-inline-start` was not being applied
correctly due to the order of appearance in the CSS rules
([#6618](https://github.com/elastic/eui/pull/6618))
**Bug fixes**
- Fixed `EuiDataGrid` footer control columns rendering with cell
expansion popovers when they should not have been
([#6609](https://github.com/elastic/eui/pull/6609))
- Fixed an `EuiSkipLink` bug where main content loading in
progressively/dynamically after the skip link rendered was not being
correctly focused ([#6613](https://github.com/elastic/eui/pull/6613))
**Breaking changes**
- Renamed `EuiIcon`'s `alert` to `warning`
([#6608](https://github.com/elastic/eui/pull/6608))
- Removed `EuiIcon`'s `crossInACircleFilled` in favor of `error`
([#6608](https://github.com/elastic/eui/pull/6608))
---------
Co-authored-by: Davey Holler <daveyholler@hey.com>
Co-authored-by: Constance Chen <constance.chen@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Jon <jon@elastic.co>
## Change validation logic for entry exception field.
Close:
[https://github.com/elastic/kibana/issues/143051](https://github.com/elastic/kibana/issues/143051)
Previously we didn't keep a validation state per field which caused a
reset of validation if we still had invalid fields. Or we can have an
invalid state for the form, but we removed the invalid field. You can
see the videos on the ticket above.
## Solution:
Keep validation state per field, like:
```js
{
[entry.id]: true,
}
```
This state can keep old fields, which already were removed, this is why
we use the selector to get the actual amount of errors.
https://user-images.githubusercontent.com/7609147/220337447-95c1558c-aa85-43d1-87e8-76370aeaf141.mov
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* squashed commit of updates to add/edit flyouts for exception, added cypress tests and unit tests
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Devin W. Hurley <devin.hurley@elastic.co>
## Summary
Adds components shared between new add/edit exception flyouts. Does not yet modify the flyouts themselves. Trying to break down what would be an even larger PR into chunks.
## Summary
**API changes**
- Adds API for determining the list-rule references.
- Updates the exception items find api to include the `search` param which allows for simple search queries - used with the EUI search bar
**UI updates**
- Moved the exception components into new `rule_exceptions` folder per suggested folder structure updates listed [here](https://github.com/elastic/kibana/issues/138600)
- Updates the rule details tabs to split endpoint and rule exceptions into their own tabs
- Updates the viewer utilities header now that these different exception types are split
- Updates exception item UI to match new designs
- Updates the UI for when there are no items
- Removes `use_exception_list_items` hook as it is no longer in use
- Flyouts (add/edit) remain untouched
* Implement wildcard exceptions for detection rules
* Fix index pattern retrieval on edit exceptions flyout
* Fix API integration test logic
* Fix entry_renderer linting
* Remove bad fix idea
* Add 'does not match' operator to UI
* Fix test
* Add unit tests
* Add wildcard exceptions to list of DE exception operators
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Addresses #86258
The variable tracking state needed to be cleared on operator change. If you didn't change operators, then invalidating and then validating an entry worked as expected, but if you switched operators, the error state was not being cleared causing the builder to read that there was an error state.
* Update warning text for event filter matches operator when file path has wildcards
fixes elastic/security-team/issues/3199
* update text
review changes
* Don't show a default value '-' for emoty descriptions on artifacts list. Also removes empty spaces
* Update copy to say 'event filters' instead of 'exceptions'
* Decrease spacing between avatar and comments textbox
* Adds extra spacing between last exception builder field and the buttons group
* Reduces effect scope togle width to by dynamic depending on translations
* Makes effected policy button group persistent across different artifact forms
* Removes unused import
* Center button group for small devices
## Summary
See: https://github.com/elastic/kibana/issues/110903
This removes the `export *` from:
* lists plugin
This also adds `import type` and `export type` in a few areas and fixes the `LicenseType` by changing it from `server` to using the version from `common` to remove the restricted paths. This extra addition prevents more memory leaks when we run jest.
## Summary
This removes all the areas marked as deprecated from `.../src/plugins/data/public` with their `@kbn/es-query` equivalent or it uses the directly exported version from `.../src/plugins/data/public`. Anywhere else this adds the `import type {` where it can to encourage the build system to do more type erasures.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* [eslint] add rule to prevent export* in plugin index files
* deduplicate export names for types/instances with the same name
* attempt to auto-fix duplicate exports too
* capture exported enums too
* enforce no_export_all for core too
* disable rule by default, allow opting-in for help fixing
* update tests
* reduce yarn.lock duplication
* add rule but no fixes
* disable all existing violations
* update api docs with new line numbers
* revert unnecessary changes to yarn.lock which only had drawbacks
* remove unnecessary eslint-disable
* rework codegen to split type exports and use babel to generate valid code
* check for "export types" deeply
* improve test by using fixtures
* add comments to some helper functions
* disable fix for namespace exports including types
* label all eslint-disable comments with related team-specific issue
* ensure that child exports of `export type` are always tracked as types
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes https://github.com/elastic/kibana/issues/105731, by replacing these `any` types:
```json
type IFieldType = any;
type IIndexPattern = any;
type Filter = any;
```
With the types from `es-query` which are:
* IndexPatternFieldBase
* IndexPatternBase
* Filter
Note: I had to do a few creative casting to avoid having to use `FieldSpec` since that is not within the package `es-query` and is not planned to be within that package or another package for at least a while if ever.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Creates an autocomplete package from `lists` and removes duplicate code between `lists` and `security_solutions`
* Consolidates different PR's where we were changing different parts of autocomplete in different ways.
* Existing Cypress tests should cover any mistakes hopefully
Manual Testing:
* Ensure this bug does not crop up again https://github.com/elastic/kibana/pull/87004
* Make sure that the exception list autocomplete looks alright
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* Add event filters filter on exception list to hide it in UI
* Fixes unit test and added more tests for showEventFilters
* fixes test adding showEventFilters test cases
* Pass params as js object instead of individual variables
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Adds boilerplate for new hook-utils package
* Move existing, identified utils into our hook-utils package
Updates references, and fixes a few missing config that were preventing
packages from building.
* Extracts a common type and adds a little more JSdoc for clarity
* Adds new useObservable hook
Similar to useAsync (a nearly identical interface), this is meant to
wrap a thunk returning an observable, allowing conditional invocation
and progressive updates as the observable continues to emit.
* Remove orphaned test
This function (and its tests) were moved to the hook-utils package; this
was simply missed.
* Remove optional chaining from kbn package
The build system does not currently support these typescript features.
While a valid fix would also have been to build separate browser and
node targets a la #99390, the use here was very minimal and so changing
to a supported syntax was the most pragmatic fix.
* Update old reference in test file
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
