* chore(NA): eslint rule for disallowing naked eslint-disable
* chore(NA): export new rule and update docs
* chore(NA): creation of rule in ts
* chore(NA): new corrected rule in ts
* refact(NA): remove old logic from older plugin
* docs(NA): update documentation
* docs(NA): update documentation
* docs(NA): update documentation
* refact(NA): include edge cases for better locating errors
* chore(NA): changed regex name
* docs(NA): correct name rule on docs
* refact(NA): use dedent in the template literals
* refact(NA): check for undefined
* fix(NA): introduces support for eslint-disable-line
* chore(NA): fix extra space
* test(NA): created more test cases
* chore(NA): rename plugin to eslint-plugin-disable
* docs(NA): update nav and operations landing page ids for eslint rule
* test(NA): use messageIds on test
* chore(NA): complete naked eslint disables with specific rules
* chore(NA): specific rules for a few naked eslint disable
* chore(NA): add focused eslint disable on big reindex_operation_with_large_error_message.ts file
* chore(NA): changes according PR feedback
* chore(NA): include specific eslint rules on latest naked eslint disable
* chore(NA): missing eslint disable specific rule
* fix(NA): remove comment for js annotator
* chore(NA): re add eslint focused disable rule to x-pack/plugins/osquery/cypress/support/coverage.ts
* chore(NA): re add eslint focused disable rule to x-pack/plugins/osquery/cypress/support/coverage.ts
* chore(NA): re add eslint focused disable rule to x-pack/plugins/osquery/cypress/support/coverage.ts
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## [Security Solution] Fixes: Queries with `nested` field types fail open with `failed to create query: [nested] failed to find nested object under path [threat.enrichments]` errors for indexes where the nested fields are unmapped
This PR implements a fix for <https://github.com/elastic/kibana/issues/130340>, where queries with [nested](https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html) field types failed open with `failed to create query: [nested] failed to find nested object under path [threat.enrichments]` errors for indexes where the nested fields are unmapped.
The fix uses the new `nestedIgnoreUnmapped` option to the `buildEsQuery` API introduced in <https://github.com/elastic/kibana/pull/134580> as a fix for issue <https://github.com/elastic/kibana/issues/130348>.
Please see <https://github.com/elastic/kibana/issues/130340> for a deep dive on the issue being fixed.
### Before
❌ Before this fix, Timeline queries that used the `nested` query syntax in requests did NOT contain the `ignore_unmapped` option, per the example request below:
```json
"nested": {
"path": "threat.enrichments",
"query": {
"bool": {
"should": [
{
"match": {
"threat.enrichments.matched.atomic": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f"
}
}
],
"minimum_should_match": 1
}
},
"score_mode": "none"
}
```
_Above: Timeline requests for fields with the `nested` query syntax did NOT contain the `ignore_unmapped` option (when inspected)_
When indexes where the nested fields were unmapped were searched:
- Elasticsearch returned a `200` status code
- The response from Elasticsearch included shard failures, per the example response below:
```json
"_shards": {
"total": 5,
"successful": 3,
"skipped": 0,
"failed": 2,
"failures": [
{
"shard": 0,
"index": ".ds-logs-endpoint.events.process-default-2022.06.13-000001",
"node": "3nAChOVOQKy92bhuDztcgA",
"reason": {
"type": "query_shard_exception",
"reason": "failed to create query: [nested] failed to find nested object under path [threat.enrichments]",
```
_Above: Timeline responses contained shard failures (when inspected)_
### After
✅ After this fix, Timeline queries that use the `nested` syntax in requests contain the `"ignore_unmapped": true` option, per the example request below:
```json
"nested": {
"path": "threat.enrichments",
"query": {
"bool": {
"should": [
{
"match": {
"threat.enrichments.matched.atomic": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f"
}
}
],
"minimum_should_match": 1
}
},
"score_mode": "none",
"ignore_unmapped": true
}
```
_Above: Timeline requests with the `nested` query syntax `"ignore_unmapped": true` option (when inspected)_
When indexes where the nested fields were unmapped are searched:
- Elasticsearch (still) returs a `200` status code
- The response from Elasticsearch does NOT include shard failures, per the example response below:
```json
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
```
### A tail of two `convertToBuildEsQuery` functions
While fixing this PR, it was noted that there are two different implementations of the `convertToBuildEsQuery` function in:
- `x-pack/plugins/security_solution/public/common/lib/keury/index.ts`
- `x-pack/plugins/timelines/public/components/utils/keury/index.ts`
The implementations of these functions are not the same. Specifically, the return type of the former implementation is:
```ts
[string, undefined] | [undefined, Error]
```
and the latter is just:
```ts
string
```
- This PR reduces the implementations of `convertToBuildEsQuery` down to a single function exported by the `timelines` plugin in `x-pack/plugins/timelines/public/components/utils/keury/index.ts`
- To minimize the scope of the changes in this PR, the previous Security Solution implementation in `x-pack/plugins/security_solution/public/common/lib/keury/index.ts` re-exports the new `timelines` implementation.
### Desk testing
See the _Reproduction steps_ section of <https://github.com/elastic/kibana/issues/130340> for details
* reduce data plugin reexporting of data view exports
* reduce data plugin reexporting of data view exports
* cleanup
* Apply suggestions from code review
Co-authored-by: Dima Arnautov <arnautov.dima@gmail.com>
Co-authored-by: Dima Arnautov <arnautov.dima@gmail.com>
* Hide empty state for timeline when graph overlay is rendered
* Add unit test to not render empty state when overlay is open
* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## [Security Solution] Fixes sorting issues related to unmapped fields
This PR fixes the following issues related to sorting unmapped fields in timelines and the events / alerts tables:
- <https://github.com/elastic/kibana/issues/129603>
- <https://github.com/elastic/kibana/issues/123912>
- <https://github.com/elastic/kibana/issues/131625>
The `unmapped_type` property [addition](https://github.com/elastic/kibana/pull/87241/files#diff-52fd5870dcd5f783f9fc8ac3a18a8674d83ac6136e09fe0e0bcae30427d61c3fR55) to the `sort` parameter of requests was using the `type` field metadata from `BrowserFields`, but the `type` metadata (for some fields) contains the value `string`, which is not a [valid field data type](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html).
The fix for the issues above:
- Populates the `sort` property of requests with values from the `esTypes` `BrowserFields` metadata (instead of `type`)
- The `esTypes` metadata may specify more than one field value type. When `esTypes` contains more than one type, and `keyword` is one of the types, the `sort` property of the request will prefer `keyword` over other the other types
- When the field metadata has an empty `esTypes` collection, the `sort` property of the request will default to using `"unmapped_type": "keyword"`
- The field type displayed in tooltips when hovering over columns in a timeline now displays values from `esTypes` instead of `type`
### Desk testing
To reproduce issue <https://github.com/elastic/kibana/issues/129603> and to verify the fix:
1) Open Kibana `Dev tools`
2) Execute the following query to delete any exiting `logs-ti_test` index:
```
DELETE logs-ti_test
```
3) Execute the following query to create an index named `logs-ti_test`, which has the following properities:
- Dynamic mappings are disabled via `"dynamic": false`
- It does NOT contain a mapping for `event.action` (we will sort by this field in later steps)
- It contains a mapping for the non-ECS `testing` field
```
PUT logs-ti_test
{
"mappings": {
"dynamic": false,
"properties": {
"@timestamp": {
"type": "date"
},
"event": {
"properties": {
"category": {
"type": "keyword"
},
"dataset": {
"type": "keyword"
},
"kind": {
"type": "keyword"
},
"type": {
"type": "keyword"
}
}
},
"host": {
"properties": {
"name": {
"type": "keyword"
}
}
},
"testing": {
"type": "keyword",
"ignore_above": 1024
},
"threat": {
"properties": {
"indicator": {
"properties": {
"file": {
"properties": {
"hash": {
"properties": {
"md5": {
"type": "keyword"
}
}
}
}
}
}
}
}
}
}
}
}
```
4) Execute the following query to add a new document to the `logs-ti_test` index, and note that:
- It does NOT contain a `event.action` field
- It contains a value for the non-ECS `testing` field
```
POST logs-ti_test/_doc/
{
"@timestamp": "2022-05-12T00:00:14.725Z",
"host": {
"name": "foozle"
},
"threat": {
"indicator": {
"file": {
"hash": {
"md5": "a4f87cbcd2a4241da77b6bf0c5d9e8553fec991f"
}
}
}
},
"event": {
"kind": "enrichment",
"type": "indicator",
"dataset": "ti_*",
"category": "threat"
},
"testing": "simulated threat intel data"
}
```
5) Navigate to the Security > Hosts page
6) Select `Last 1 year` from the date picker
7) Click the `Events` tab
8) Enter the following KQL query in the search bar at the top of the page:
```
host.name: foozle
```
9) Hover over the `foozle` entry in the `host.name` column in the Events table, and click the `Add to timeline investigation` cell action
10) Open the timeline
11) Hover over the `event.action` field
**Expected result**
- The tooltip displays type `keyword` for the `event.action` field
**Actual result**
- The tooltip displays type `string` for the `event.action` field
12) Click the `event.action` column to add a secondary sort
**Expected result**
- The table is sorted by `@timestamp` and `event.action`
- The table contents are (still) visible
**Actual result**
- The table is sorted by `@timestamp` and `event.action`
- The contents of the table are now empty
13) Click the timeline's `Inspect` button
14) In the `Inspect Timeline` dialog, click the `Request` tab
15) Scroll down to the `sort` property of the request
**Expected result**
- The `event.action` field contains a `"unmapped_type": "keyword"` property, per the example below:
```json
"sort": [
{
"@timestamp": {
"order": "desc",
"unmapped_type": "date"
}
},
{
"event.action": {
"order": "desc",
"unmapped_type": "keyword"
}
}
],
```
**Actual result**
- The request's `event.action` field contains a `"unmapped_type": "string"` property, per the example below:
```json
"sort": [
{
"@timestamp": {
"order": "desc",
"unmapped_type": "number"
}
},
{
"event.action": {
"order": "desc",
"unmapped_type": "string"
}
}
],
```
16) In the `Inspect Timeline` dialog, click the `Response` tab
**Expected result**
- The response contains `0` `failed` shards / no failures
**Actual result**
- The response contains failures for the `logs-ti_test` index, with the following reason:
```
"reason": "No mapper found for type [string]"
```
per the example below:
```json
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 4,
"successful": 3,
"skipped": 0,
"failed": 1,
"failures": [
{
"shard": 0,
"index": "logs-ti_test",
"node": "NCRcGeDqSlKQiuPWVFvMEg",
"reason": {
"type": "illegal_argument_exception",
"reason": "No mapper found for type [string]"
}
}
]
},
```
* session tab query modified query all events, not just entry leaders. solves a few problems wrt to query ability. default columns modified and display names provided for each
* snapshot updated
* readded test
* Default sort set to process.entry_leader.start desc
* sessions tab timeline id changed to cache bust localstorage for table column configs
* missed a couple spots for session tab timeline id update
Co-authored-by: mitodrummer <karlgodard@elastic.co>
## [Security Solution] [Investigations] [Tech Debt] removes redundant code from the timelines plugin
This follow-up PR removes redundant code from the `timelines` plugin, identified while implementing https://github.com/elastic/kibana/pull/130740
## [Security Solution] [Investigations] [Tech Debt] removes `deepEqual` checks in column headers and data providers
This tech debt PR is another entry in a series to remove `React.memo` `deepEqual` checks, per the details in <https://github.com/elastic/kibana/issues/124151>
- It removes `deepEqual` checks in Timeline's column headers and data providers
- Files made redundant by the `timelines` plugin adopting `EuiDataGrid` are deleted
### Methodology
The following techniques were used to ensure that removing the `deepEqual` checks did NOT result in unexpected re-renders:
- To understand why components re-rendered, Timeline was profiled with the `Record why each component rendered wile profiling` setting in the React dev tools Profiler enabled, shown in the (illustrative) screenshot below:
- Components were temporarily instrumented with counters that incremented every time the component was rendered. Log statements prefixed with `[pre]` were observed before making changes, per the screenshot below:
- After removing the `deepEqual` checks, the log prefix was updated to `[POST]`, and the log entries were observed again, per the screenshot below:
The `[pre]` and `[POST]` counters were compared to verify removing the `deepEqual` checks did NOT introduce unexpected re-renders.
* Upgrade EUI to v54.0.0
* [Discover] Remove deprecated closePopover call
- for closeCellPopover ref API
* [Lens] Remove deprecated closePopover call
- for closeCellPopover ref API
* [Security/Timelines] Remove deprecated closePopover call
- for closeCellPopover ref API
* [Security/Timeline] Update Timeline datagrid to accept/pass `visibleCellActions` prop
+ update Security to show 3 visible cell actions
* [APM] Account for removed EUI theme avatar sizes
* Update emotion dependencies to latest
* Remove styles from being rendered in emotion serializer
* Update snapshots affected by emotion serializer `includeStyles: false` change
* Update snapshot changes caused by EuiFormControlLayout changes
* Update snapshot changes caused by EuiAvatar CSS-in-JS conversion
* consolidate yarn.lock
* [Spaces] Fix failing test due to new EuiAvatar emotion wrapper
- which, due to mount() causes .first() to no longer work as expected - targeting .last() instead gets the actual div element which works
* [Security] Fix cell expansion popover actions
- EUI added 2 `.euiPopoverFooter`s for overflowing cell actions, and Security's CSS to hide the first 2 cell actions (replaced by their own custom cell actions) was unintentionally affecting other actions
* Clean up spaces test snapshots
* [Security feedback] Revert 793d208 and hard-code visibleCellActions
Co-authored-by: Greg Thompson <thompson.glowe@gmail.com>
Co-authored-by: Joe Portner <joseph.portner@elastic.co>
* Updgraded EUI packages in package.json and src/dev/license_checker/config.js
* Resolved Jest test failures for Jest test suites 1 and 2. Updated snapshots, and updated equality conditions for specific test cases
* Resolve Jest test cases for Jest test suite 3. Updated snapshots for required tests
* Resolved failing Jest test cases in Jest suite 3. Updated tests checking for strict text equality to account for text coming from the EuiScreenReaderOnly component. Also updated tests to account for EuiIcon text that is now rendered when the icon is imported from .testenv (PR 5709 - https://github.com/elastic/eui/pull/5709/).
* type fixes
* eui to 52.2.0
* Resolved test cases for Jest test suites 1 and 2. Updated required snapshots.
Updated tests using getAllByLabelText and getByLabelText to getAllByText and getByText respectively as the former have been deprecated
* Updated Jest tests for Jest test suites 5 and 6. Updated required snapshots. Updated instances of getByLabelText and getAllByLabelText to getByText and getAllByText as the former are now deprecated.
* Updated Jest tests for Jest test suite 7. Updated required snapshots.
* Completed test case revisions for Jest test suites 1, 3, 6, 7, and 8. Updated required snapshots. Updated various tests to account for text rendering of the EuiIcon text.
* removed unused test utils
* use .contains for euiicon content
* storyshots updates
* linting
* Fix failing a11y violations tests
* Fix Jest failures caused by #eui/5709
- these changes should be reverted if we opt to revert the above PR
Co-authored-by: Bree Hall <briannajdhall@gmail.com>
Co-authored-by: Greg Thompson <thompson.glowe@gmail.com>
* Updgraded EUI packages in package.json and src/dev/license_checker/config.js
* Resolved Jest test failures for Jest test suites 1 and 2. Updated snapshots, and updated equality conditions for specific test cases
* Resolve Jest test cases for Jest test suite 3. Updated snapshots for required tests
* type fixes
* Resolved failing Jest test cases in Jest suite 3. Updated tests checking for strict text equality to account for text coming from the EuiScreenReaderOnly component. Also updated tests to account for EuiIcon text that is now rendered when the icon is imported from .testenv (PR 5709 - https://github.com/elastic/eui/pull/5709/).
* eui to 52.2.0
* Resolved test cases for Jest test suites 1 and 2. Updated required snapshots.
Updated tests using getAllByLabelText and getByLabelText to getAllByText and getByText respectively as the former have been deprecated
* Updated Jest tests for Jest test suites 5 and 6. Updated required snapshots. Updated instances of getByLabelText and getAllByLabelText to getByText and getAllByText as the former are now deprecated.
* Updated Jest tests for Jest test suite 7. Updated required snapshots.
* Completed test case revisions for Jest test suites 1, 3, 6, 7, and 8. Updated required snapshots. Updated various tests to account for text rendering of the EuiIcon text.
* eui back to v52.2.0
* removed unused test utils
* use .contains for euiicon content
* storyshots updates
* linting
Co-authored-by: Greg Thompson <thompson.glowe@gmail.com>
The function filterBrowserFieldsByFieldName is being run 4+ times when loading pages in the Security app. With a large number of fields, such as is found in production environment, this function can take 10+ seconds to completed. With this implementation, it should run a bit quicker.
* Add Events tab to the User page and the User details page
* Add External alerts tab to the User page and the User details page
* Add cypress tests
* Add unit test to EventsQueryTabBody
* Memoize navTabs on Users page
* view selected option added
* new header component
* test fixed
* Update x-pack/plugins/timelines/public/components/t_grid/toolbar/fields_browser/field_table_header.test.tsx
use not.toBeInTheDocument
Co-authored-by: Pablo Machado <machadoum@gmail.com>
* pass callback down instead of state setter
Co-authored-by: Pablo Machado <machadoum@gmail.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* eui to v50.0.0
* i18n tokens
* Deprecate EuiDataGrid's `popoverContents` prop for `renderCellPopover`
* [optional ML refactor] Use `renderCellValue.isDetails` to customize numeric popover content instead of `renderCellPopover`
- since no especially custom popover rendering is occuring, just conditional content
* onChangeItemsPerPage update
* storyshots updates
* snapshot updates
* snapshot updates
* snapshot updates
* snapshot updates
* EuiComboBox listbox -> combobox
* remove invalid combobox aria attr
* Revert "onChangeItemsPerPage update"
This reverts commit 127c9e5840.
* eui to v51.0.0
* WIP: schema
* WIP: schema
* EuiSelectable API changes
* WIP: schema
* hidePerPageOptions -> showPerpageOptions
* WIP: schema
* hidePerPageOptions -> showPerpageOptions
* WIP: schema
* breadcrumbs type
* clean up
* snapshot updates
* Fix E2E datagrid cell filter action test
- This changed in 50.0.0 because of https://github.com/elastic/eui/pull/5681
- `await testSubjects.click('filterForButton')` applies to both the cell action button icon and the cell popover button
- The test was trying to click the cell action button icon and not the popover button, which closed the popover and caused nothing to actually get clicked
- the solution I went with was to simply avoid opening the cell popover but instead click the cell action icon directly
* WIP: selectable search
* clean up
* eui to v51.1.0
* i18n tokens
* resolve SharedRenderCellElementProps.schema optionality
* i18n, snapshot updates
* shapshot update
* consolidate url-parse
Co-authored-by: Constance Chen <constance.chen@elastic.co>
* field browser first revamp implementation
* customize columns for security solution alert tables
* cleaning
* some tests
* clean unused code
* field browser tests created and existing fixed
* security solution test fixes
* translations cleaned
* fix test
* adapt cypress tests
* remove translation
* fix typo
* remove duplicated test
* type error fixed
* enable body vertical scroll for small screens
* fix new field not added to the table bug
* addapt Kevin performance improvement
* fixed linter error
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* Probably better
* Make backspace not slow
* Type and prop cleanup
* PR comments, fix failing cypress test
* Update cypress tests to wait for debounced text filtering
* Update cypress test
* Update failing cypress tests by waiting when needed
* Reload entire page for field browser tests
* Skip failing local storage test
* Remove unused import, cleanKibana back to before
* Skip failing tests
* Clear applied filter onHide, undo some cypress changes
* Remove unnecessary wait
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
