Commit graph

55638 commits

Author SHA1 Message Date
Julia Bardi
65c71a78ec
[Fleet] filter out hosted agents when finding common tags for bulk action (#137161)
* filter out hosted agents when finding common tags for bulk action

* fixed checks
2022-07-26 14:18:53 +02:00
Kibana Machine
56a870948e skip failing test suite (#137174) 2022-07-26 08:12:10 -04:00
Byron Hulcher
bb0365e43a
[Enterprise Search] Display most recent crawl request status in Indices and Crawl Request tables (#137128) 2022-07-26 04:50:32 -07:00
Cristina Amico
d96fbc441a
[Fleet] Fix default source_uri host (#137151) 2022-07-26 04:47:18 -07:00
Walter Rafelsberger
bd9955a396
[ML] Explain log rate spikes: Fix data view title. (#137053)
- Use dataView.getName() instead of dataView.title.
- Adds missing scss brought over from Data Visualizer to improve positioning of long Data View names.
2022-07-26 13:44:57 +02:00
Jiawei Wu
cd3d2d79c7
[RAM] Stack management/o11y rule details parity (#136778)
* stack management/o11y rule details parity

* Hide edit button in stack management

* Add tests

* Move fetching summary out of o11y

* Undo changes to hooks in o11y

* Fix test and add new tests

* Remove customLoadExecutionLog prop

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-07-26 13:14:05 +02:00
Pierre Gayvallet
0645a3ba38
Create common and api types packages for the savedObjects domain (#136722)
* Creating empty @kbn/core-saved-objects-common package

* start moving types around

* start fixing imports

* fix entrypoint exports

* fix external import

* create explicit ISavedObjectsRepository interface

* fix another external usage

* rewrite browser exports

* create explicit SavedObjectsClientContract interface

* move client/repository types to @kbn/core-saved-objects-api-server

* start fixing imports

* one more

* fix global re-exports

* fix some browser-side imports

* fix more violations

* prepare the browser-side client

* fix one more usage

* fix external usage

* fix more external usages

* one more

* Create @kbn/core-saved-objects-api-browser package

* fix more usages of error helper

* fix more internal imports

* use interface for SSO

* adapt more imports

* damn those types were a mess

* fix more usages of SSO

* Revert "fix more usages of SSO"

This reverts commit 07a12e5353.

* Revert "use interface for SSO"

This reverts commit 6240fc86c5.

* export the interface with the old name instead.

* adapt tests concrete usages of SSO

* export reference type, more fixes

* this gonna be long

* one more

* other resolve type change

* more usages

* Am I getting close?

* yet more fixes

* back to client impl

* fix bulkGetting undefined

* fix SS mock

* some cleanup

* self-review

* fix new usages
2022-07-26 04:01:33 -07:00
Julia Bardi
b3d7d1288d
added filters to tags query (#137089) 2022-07-26 02:15:21 -07:00
Stratoula Kalafateli
a296e4cc97
[Discover] Supports SQL query language (#134429) (#136702)
* [Discover] Supports SQL query language (#134429)

* Move the add dataview action above the dataview selection panel

* Implements a new selectable on the dataview picker for the text based languages

* Implementation of the transition modal when on SQL mode and select a dataview

* Fix es lint

* Change switch modal button modal icon

* Lazy load components

* Small changes on the styling of the switch without saving button

* Initialization of mocaco editor

* Change to the type

* Fixes types checks

* New submit button for query mode

* Implememtation of the expanded mode of the editor

* Implement documentation

* Implementation of the oneliner mode with ellipsis

* Some  fixes on the resizer

* Implementation of the errors layout, WIP

* Fetch SQL data in Discover

* Fix expression test

* Fix editor zIndex

* Fix types error

* Fix type check in Discover

* Fix more types

* some CI fixes

* Fixes

* Cleanup after merge

* Remove from state

* Connect search errors with the unified search editor

* Add error mrkers in unified search editor

* Save and open saved searches

* Filter out saved searches from text based languages

* Some fixes

* Fix unit tests

* Fix checks

* On save and exit modal implementation

* Add shortcut on the editor for submit query

* Fix wrong condition

* Initial types change

* Use regex to find the index pattern string

* Fix some types and cleanup

* Fix types

* Fix some types

* Further fixes

* More fixes

* More fixes

* Fix visualize types

* more

* More fixes

* Fixes more types

* Fix dashboard types

* Fix dashboard types

* Controls plugin types

* Fix Lens types

* Fix data plugin types

* Fix types in Lens 2

* buildEsConfig type fixes

* Fix observability types

* Fix maps types

* data visualizer types

* Fix ml types

* xpack rest types

* Fix jest test

* Fix

* Move helper functions to es config

* fix bug on breadcrumb click

* Fix time field bug

* Add enableSql advanced setting to discover for enabling the sql mode

* Make the documentation component more dynamic

* Add some comments, improvements

* Enhance storybook with the textbased languages

* Update storybook with the error state of the editor

* Adds a readme for the editor and fixes the modal mobile version

* [Discover] improve test and storybook for new data type

* [Discover] add functional tests

* Add aggregate functions to the documentation

* [Discover] fix tests

* Add some unit tests

* [Discover] fix linting

* [Discover] update linting

* More unti tests

* Dataview picker unit tests

* Fix a bug on the dataview picker

* Add unit tests for the editor

* Fix jest test

* [Discover] apply suggestions

* [Discover] adjust styles

* Fix some bugs and select columns in the sql mode

* [Discover] fix eslint and tests

* [Discover] update unit tests

* Fix bug on transitioning from sql mode to dataview mode

* [Discover] fix tests

* Design fixes on the errors messages

* [Discover] fix ci

* Update the columns only if the query changes

* [Discover] change isPlainRecord retrieval method

* Fix bug on cleanup

* Fix bug on opening a saved search

* [Discover] fix comments

* [Discover] fix bug with browser refresh

* [Discover] fix functional

* [Discover] fix another functional

* Fix ordering lost when the user refreshes the browser

* [Discover] revert use_discover_state

* [Discover] revert functional impl

* Fix security solution types

* Casting dashboard plugin

* Revert change

* type param

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

* Revert types changes

* More reverts

* Types fixes

* Fix Discover jest test

* Fix context app jest test

* Final types changes

* Fixes unit test

Co-authored-by: Dzmitry Tamashevich <diaamnj@mail.ru>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>

* Fix types

* Fix jest test

* More design fixes

* Update advanced setting description

* Further design changes

* [Discover] Remove document explorer header column edit data view field functionality (#136743)

* remove Edit data view field for SQL

* Fix the fix

* [Discover] Implement SQL data fetching for embeddable (#136793)

* remove Edit data view field for SQL

* Fix the fix

* Implement SQL for embeddable

* Fix non-saved-search embeddables

* Fix reporting bundle size

* Allow filters on dashboard level for sql searches

* Fix the radius on the editor

* Add vertical padding on the editor

* Change the theme

* Address PR comments

* Fix types

* Address some of the comments

* Fix bug on transitioning from SQL to dataview mode with the modal dismissed

* More types fixes

* Design review comments

* Discovery team review comments

* Fix jest tests

* Fix bug on navigating from the SQL mode to the dataview mode and back in sql mode by clicking the breadcrumb

* Update src/plugins/discover/public/application/main/hooks/use_discover_state.ts

Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>

* Add padding to the top of the editor without creating any bug

* Add some padding to the bottom without creating any bug

* Fixes undo bug

* Fix confusing naming of variable

* Fix nested selects

* Update texts for transition modal and warning

* Make it work with dashboard Query

* Address some of the comments

Co-authored-by: Dzmitry Tamashevich <diaamnj@mail.ru>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co>
2022-07-26 10:51:31 +03:00
Marshall Main
68162dca7c
Add kibana.alert.new_terms keyword to alerts mapping (#137093) 2022-07-26 00:15:56 -07:00
Julia Bardi
d7415c1f20
simplified logic of showing tags, always displaying tooltip (#137092) 2022-07-26 08:45:22 +02:00
Byron Hulcher
1b4c6e0f0e
[Enterprise Search] Add polling for index data to Search Index view in Content app (#137083)
* Clean-up IndexViewLogic

* Add polling for index data to Search Index view

* Stop flashing UX

* Stop further flashing UX

* Fix TotalStats flashing on poll

* Fix tests

* Fix clearFetchIndexTimeoutId -> clearFetchIndexTimeout
2022-07-26 08:05:17 +02:00
Kibana Machine
0d3c40c04b
[api-docs] Daily api_docs build (#137145) 2022-07-26 14:13:05 +09:30
Candace Park
4227348b95
[Security Solution][Admin][Responder] Disable Reponder option in action menu if processes capabilities are not supported (#136904) 2022-07-25 23:58:44 -04:00
Spencer
0154f75445
[ci-stats] ship buildkiteJobId with metrics and support backfilling (#137123) 2022-07-25 22:24:57 -05:00
Tiago Costa
710432aa0a
skip flaky suite (#137136) 2022-07-26 02:54:45 +01:00
Clint Andrew Hall
b459ffa4c6
[home] Sample Data Tab w/ callout UI (#136790)
* [home] Sample Data Tab w/ callout UI

* Fix tests

* Update packages/home/sample_data_tab_content/src/demo_env_panel.tsx

Co-authored-by: Kelly Murphy <kelly.murphy@elastic.co>

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

* Fixes

* Fixes

* Update test/functional/page_objects/home_page.ts

* Fix tests

* Add telemetry

* Add docs, more telemetry

Co-authored-by: Kelly Murphy <kelly.murphy@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 18:34:32 -07:00
Kevin Qualters
7717e90b4e
Fix typo in events of type analyzer query (#137127) 2022-07-25 17:16:01 -07:00
Tiago Costa
bf12ac2e6c
skip flaky suite (#135341) 2022-07-26 00:52:57 +01:00
Tiago Costa
37831e4e19
skip flaky suite (#137124) 2022-07-26 00:51:41 +01:00
Spencer
20f9cf9fd4
[eslint] add rule for validating cross-boundary imports (#137116) 2022-07-25 18:49:17 -05:00
Byron Hulcher
88bb91f3a0
[Enterprise Search] Remove Ingest Attachment message from App Search Crawler (#137106) 2022-07-25 19:33:59 -04:00
Kevin Qualters
3669e79b82
[Security Solution][Analyzer] Add alerts to analyzer, display alerts by process ancestry in alert flyout (#135340)
* WIP stats appearing in tree api, events api TODO

* All panels work, types/tests TODO

* WIP handle events with only alerts or only events better

* Throw away commit just POC alert ids in tree response

* Remove console.log

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* Fix some tests and 2/3 type errors, still WIP

* Disable tree request until entity request succeeds

* Remove console.log

* Fix remaining types

* Create shared hook for timeline selectors used by analyzer

* Remove reset scroll

* Change type definition for getRacClient

* Address pr comments

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 18:13:23 -04:00
Joey F. Poon
3515e6f22c
[Security Solution] bubble up linux deadlock errors (#136241) 2022-07-25 16:58:24 -05:00
Efe Gürkan YALAMAN
1967117c4f
[Enterprise Search] Document explorer and Index Mappings tabs (#136874)
* Stubbing out the UI for the new result component

* Integrate stub UI to Kibana

* WIP, bind backend with result component

* Change from flex based to table for result components

* Add query to the search field

* Add mappings page

* Update code block

* Review changes

* Add Documents logic tests

* Revert mock folder name change

* Fix server tests

* Add accessibility text on missing buttons

* Review changes

* Update api calls

* Add docLinks for elasticsearch mappings

Co-authored-by: Davey Holler <daveyholler@hey.com>
2022-07-25 14:47:20 -07:00
Angela Chuang
5b465b9f99
Host risk score module UI enhancement (#133708)
* test

* open in dev tool

* Adding comments and removing the space placeholder in some places

* import dashboard

* clean up

* clean up buttons

* isSignalIndexExists

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* getIndexExists

* import dashboard

* clean up

* sync with the main branch

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

* useDashboardButtonHref

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* tooltip

* clean up

* types

* clean up

* clean up

* rename

* fix unit and cypress tests

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* fix type error

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* fix type error

* fixup

* i18n

* update script

* unit tests

* unit tests

* UI review

* fix up

* unit tests

* add trouble shooting hint and update tests

* update cypress tests

* Update x-pack/plugins/security_solution/server/lib/prebuilt_dev_tool_content/routes/read_prebuilt_dev_tool_content_route.ts

Co-authored-by: Sergi Massaneda <sergi.massaneda@gmail.com>

* review

* UI review

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

* update api path

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

* update read_alerts_index_exists_route

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* update dev tool content

* update api path

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

* update snapshot

Co-authored-by: Apoorva <appujo@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Sergi Massaneda <sergi.massaneda@gmail.com>
2022-07-25 14:41:41 -07:00
Jack
141b765568
[8.4] [Kubernetes Security] Enable Kubernetes visualization page on security solution (#137090)
* Toggle securitySolution.kubernetesEnabled to true

* Add beta tag to kubernetes page

* Remove unused color from euiTheme

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 17:41:15 -04:00
Ashokaditya
3dc26b6a75
[Security Solution][Endpoint][Response Actions] Sync action status correctly when API response is slow (#136644)
* sync action details refresh times

fixes elastic/kibana/issues/136098

* mutate local actionId instead

fixes elastic/kibana/issues/136098

* Update API responses

review changes

* Correctly call API and update store

Ensure that we call the action API and also that we update the store only when the console is open

* fix types

* add tests for the fix

fixes elastic/kibana/issues/136098

* fix incorrect imports ++ failing tests

* Fix isolate/release tests

Co-authored-by: Paul Tavares <paul.tavares@elastic.co>
2022-07-25 17:27:31 -04:00
Faisal Kanout
546f2b158b
[Actionable Observability] [ResponseOps] - Create and add Rule Alerts Summary as a sharable component to the O11y Rule Details (#135805)
* Add useLoadRuleAlertsAggs hook

* Update hook

* Make RuleAlertsSummary sharable

* Like RuleAlertsSummary component with useLoadRuleAlertsAggs hook

* Add date_histogram to the hook

* Update layout

* Provide 0 as default value when there is no recovered or active alerts

* Update style

* Fix style

* Fix style rule details page

* Add OBSERVABILITY_SOLUTIONS filter

* Update naming filteredRuleTypes

* Add alerts aggs chart data

* Always return active and recovered

* Update the query/aggs

* pair programing to get the bar series working with date

* Add correct color to the chart

* WIP

* Style the chart correctly

* Update x-pack/plugins/triggers_actions_ui/public/application/sections/rule_details/components/rule_alerts_summary.tsx

Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>

* Update x-pack/plugins/triggers_actions_ui/public/application/sections/rule_details/components/rule_alerts_summary.tsx

Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>

* Update x-pack/plugins/triggers_actions_ui/public/application/sections/rule_details/components/rule_alerts_summary.tsx

Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>

* Update x-pack/plugins/triggers_actions_ui/public/application/sections/rule_details/components/rule_alerts_summary.tsx

Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>

* Update x-pack/plugins/triggers_actions_ui/public/application/sections/rule_details/components/rule_alerts_summary.tsx

Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>

* Remove duplicated copyrights

* Code review update component structure

* Fix import error

* Remove OBSERVABILITY_SOLUTIONS

* Code review

* No more needed as the aggs is changed

* Fix import

Co-authored-by: Xavier Mouligneau <xavier.mouligneau@elastic.co>
2022-07-25 14:14:53 -07:00
JD Kurma
d3ae221bed
[Security Solution] Policy config overlap (#136217)
* policy config overlap

* default advanced settings added to policy config

* add back original logic

* null check + remove unused func

* clean up/rename

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* add unit tests

* add unit tests

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* linting

* linting

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* fix linting

* fix linting

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 14:00:35 -07:00
Nicolas Chaulet
616f7be237
[Fleet] Do not break UI and API on orphaned package policies (#137107) 2022-07-25 16:21:23 -04:00
Georgii Gorbachev
becaec81e1
[Security Solution][Detections] Extended rule execution logging to Event Log (#126063)
**Epics:** https://github.com/elastic/kibana/issues/124947, https://github.com/elastic/kibana/issues/118324
**Fixes:** https://github.com/elastic/kibana/issues/131352

## Summary

Console logs written by rule executors can now be "routed" to the Event Log in addition to the console. A new table UI for viewing plain rule execution logs allows the user to look at all status changes, errors, warnings, info and debug messages on the Rule Details page.

<img width="1502" alt="Screenshot 2022-07-20 at 15 31 54" src="https://user-images.githubusercontent.com/7359339/179995075-24440224-daf9-4e73-bc62-b6ce211052b3.png">

**This feature is hidden under a feature flag and disabled by default** -- it might not be production-ready yet. We will need to work on product and UX/UI design in the subsequent development cycles to make it ready for release. Until then, we can start using it in development: it should make it easier to troubleshoot issues with rule execution. Add this flag to your Kibana config to enable this feature:

```yaml
xpack.securitySolution.enableExperimental: ['extendedRuleExecutionLoggingEnabled']
```

If the flag is enabled:

- Rules will start writing console logs to Event Log as events of a new type `message`, in addition to the existing `status-change` and `execution-metrics` events.
- Rule Details page will show a new tab called `Execution events`. This tab will contain a table with plain execution logs.
- In Stack Management, you will find two new Kibana Advanced Settings for controlling this extended logging.

As for the new Kibana Advanced Settings, by default:

- Extended logging is enabled.
- The minimum console log level to be written to Event Log is `error`. This only affects the new `message` events.

<img width="774" alt="Screenshot 2022-07-20 at 15 41 29" src="https://user-images.githubusercontent.com/7359339/179997070-d86dfc6b-3862-49ff-879d-ecc30bc128d7.png">

## Implementation details

**Important change**: refactored the folder structure as our first step to **domain-driven architecture** and **splitting the Detection Engine into subdomains**.

- Extracted most of the code related to Rule Execution Log and Rule Monitoring in general into a subdomain called `rule_monitoring`. This subdomain now lives in three folders:
  - `security_solution/common/detection_engine/rule_monitoring`
  - `security_solution/public/detection_engine/rule_monitoring`
  - `security_solution/server/lib/detection_engine/rule_monitoring`
- Tried to create a developer-friendly and clear folder structure within the subdomain.

Other changes:

- Changed all rule executors to write console logs via an instance of `IRuleExecutionLogForExecutors` instead of the console `Logger`.
  - `IRuleExecutionLogForExecutors` is passed to rule executors and downstream functions they call.
  - `Logger` is not passed anymore.
  - `buildRuleMessage` and `buildRuleMessageFactory` are deleted.
- Added support for writing console logs to Event Log.
  - Added a new rule execution event type `message` for writing console logs to Event Log.
  - Every rule execution event now has a `log.level` and `event.severity`.
- Improved the format of console logs written by rules.
  - Created a child logger for console logs of rule executors: `plugins.securitySolution.ruleExecution`.
  - Added rule static “signature” ID (`rule.rule_id`) as a correlation id to the logs.
  - Cleaned up the formatting of console logs.
  - Fixed `ExtMeta` to use interfaces instead of type intersection due to a found [bug](https://github.com/microsoft/TypeScript/issues/47935) in TypeScript that affected this type.
- Made changes in the domain model.
  - Renamed the `AggregateRuleExecutionEvent` into the `RuleExecutionResult`.
  - The new plain event is called `RuleExecutionEvent`.
- Finalized the API endpoint for fetching plain execution logs.
- Built a Rule Execution Events Table UI for showing and filtering plain execution logs.
  - Did some refactoring to extract reusable components/hooks to make development of tables easier in the future.

## Execution events table UI

For context, this is how the existing `Execution logs` table looks like when the flag is off (notice the renaming to `Execution results`:

<img width="1506" alt="Screenshot 2022-07-20 at 15 29 04" src="https://user-images.githubusercontent.com/7359339/179994450-45121035-ebb0-4e6f-83c0-9cbbbbd0b598.png">

This is the new `Execution events` table when the flag is on:

<img width="1502" alt="Screenshot 2022-07-20 at 15 31 54" src="https://user-images.githubusercontent.com/7359339/179995075-24440224-daf9-4e73-bc62-b6ce211052b3.png">

Showing only trace and debug events:

<img width="1505" alt="Screenshot 2022-07-20 at 15 33 48" src="https://user-images.githubusercontent.com/7359339/179995484-d97ff7e3-2756-42db-802f-41f11bd37507.png">

Showing only status changes:

<img width="1507" alt="Screenshot 2022-07-20 at 15 35 04" src="https://user-images.githubusercontent.com/7359339/179995804-ca6808b7-3b47-411b-a74e-d141b3fd74e0.png">

Showing only warning and error `message`s:

<img width="1508" alt="Screenshot 2022-07-20 at 15 37 11" src="https://user-images.githubusercontent.com/7359339/179996258-c154b95d-642d-45a6-b19a-7185cd71f295.png">

Expanded rows showing details of the corresponding events:

<img width="1452" alt="Screenshot 2022-07-20 at 15 39 16" src="https://user-images.githubusercontent.com/7359339/179996771-3954ceea-24e9-4760-9103-2daf6cb7b528.png">

<img width="1449" alt="Screenshot 2022-07-20 at 15 39 56" src="https://user-images.githubusercontent.com/7359339/179996805-c866674d-09a1-42ec-b954-58c6829ef19b.png">

## Console logs

Example:

```
[2022-02-23T17:05:09.901+03:00][DEBUG][plugins.securitySolution.ruleExecution] [+] Starting Signal Rule execution [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:09.907+03:00][DEBUG][plugins.securitySolution.ruleExecution] interval: 5m [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:09.908+03:00][INFO ][plugins.securitySolution.ruleExecution] Changing rule status to "running" [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:10.595+03:00][WARN ][plugins.securitySolution.ruleExecution] This rule is attempting to query data from Elasticsearch indices listed in the "Index pattern" section of the rule definition, however no index matching: ["logs-endpoint.alerts-*"] was found. This warning will continue to appear until a matching index is created or this rule is de-activated. If you have recently enrolled agents enabled with Endpoint Security through Fleet, this warning should stop once an alert is sent from an agent. [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:10.595+03:00][WARN ][plugins.securitySolution.ruleExecution] Changing rule status to "partial failure" [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.630+03:00][DEBUG][plugins.securitySolution.ruleExecution] sortIds: undefined [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.634+03:00][DEBUG][plugins.securitySolution.ruleExecution] totalHits: 0 [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.634+03:00][DEBUG][plugins.securitySolution.ruleExecution] searchResult.hit.hits.length: 0 [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.635+03:00][DEBUG][plugins.securitySolution.ruleExecution] totalHits was 0, exiting early [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.636+03:00][DEBUG][plugins.securitySolution.ruleExecution] [+] completed bulk index of 0 [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.636+03:00][DEBUG][plugins.securitySolution.ruleExecution] [+] Signal Rule execution completed. [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.638+03:00][DEBUG][plugins.securitySolution.ruleExecution] [+] Finished indexing 0 signals into .alerts-security.alerts [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
[2022-02-23T17:05:11.639+03:00][DEBUG][plugins.securitySolution.ruleExecution] [+] Finished indexing 0 signals searched between date ranges [
  {
    "to": "2022-02-23T14:05:09.775Z",
    "from": "2022-02-23T13:55:09.775Z",
    "maxSignals": 10000
  }
] [siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]
```

Note that:

- The logger name is now `plugins.securitySolution.ruleExecution`, which allows to turn on _only_ rule execution logs in the config (could be useful when debugging).
- Every log message has a suffix with correlation ids: `[siem.queryRule][Endpoint Security][rule id 825b2fab-8b3e-11ec-a4a0-cf820453283c][rule uuid 9a1a2dae-0b5f-4c3d-8305-a268d404c306][exec id ebb7f713-b216-4c90-a456-6c1a6815a065][space default]`


### Checklist

- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials
  - [x] `x-pack/plugins/security_solution/server/lib/detection_engine/rule_monitoring/logic/rule_execution_log/README.md`
  - [x] Various JSDoc comments
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### For maintainers

- [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2022-07-25 13:09:17 -07:00
JD Kurma
45db88e0e6
[Security Solution] Endpoint/Prebuilt Alert Counter (#136557)
* alert counter

* stub telemetry usage

* accurate count

* accurate count for mock

* handle edge case when agg is undefined

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* clarify counter name

* linting errors for mock data

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* linting

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 13:01:04 -07:00
Tiago Costa
242c91a432
fix(NA): buildkite es_snapshot build job after es default branch rename to main (#137109) 2022-07-25 12:43:35 -07:00
Jiawei Wu
b2825cb9a8
[RAM] Consolidate event logs UI (#136589)
* Move error action log endpoint to a new endpoint

* consolidate run log and error log

* Add new tests and fix existing tests

* Fix lint

* Fix lint

* Add API integration tests

* Remove invalid sorting columns

* Addressed comments

* Fix tests

* Add utility to convert ES sort to event log sort, addressed comments

* Fix jest tests

* Fix type error

* Address design feedback

* Revert testing code to action executor

* Address comments and add tests

* remove unused translations

* Address design feedback

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 12:08:31 -07:00
Khristinin Nikita
8d88c7851c
[Security Solution][Data Views] - Add custom DataView error (#136525)
Update rule error shown when data view not found during execution.
2022-07-25 11:57:38 -07:00
Kristof C
3246ec733e
[Security Solution][Revisit external alerts] [2861] add alerts to explore event tabs and revamp network details (#136913)
* Remove External alert trend table and artifacts, and rename detection… (#136579)

* Remove External alert trend table and artifacts, and rename detections alert

* add test for SignasByCategory

* Update signals_by_category.test.tsx

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

Co-authored-by: Kristof-Pierre Cummings <kristofpierre.cummings@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

* update event tab to show both alerts and events with toggle. (#136540)

* add test for SignasByCategory

* modify external_alerts_filter  to be more efficient

* Update usage across explore views to only use EventsQueryTabBody

* remove unused files and code related to external alerts  and move old alerts files to events_tab folder

* test fixes, and more removal of old usage

* update failing snapshots

* last bit of cleanup

* Fix type error

* fix type and translations issue

Co-authored-by: Kristof-Pierre Cummings <kristofpierre.cummings@elastic.co>

* translations fixed

* fix default stackBy value for alerts bug

* memoizations added

Co-authored-by: Kristof-Pierre Cummings <kristofpierre.cummings@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: semd <sergi.massaneda@elastic.co>
2022-07-25 11:42:51 -07:00
Devin W. Hurley
c2985c4daa
[Security Solution] [Platform] Adds state to remember what was in data view or index pattern selection when switching between the two (#136448)
Co-authored-by: Khristinin Nikita <nikita.khristinin@elastic.co>
2022-07-25 20:42:05 +02:00
Alejandro Fernández Haro
0824234fe1
[EBT] ElasticV3-Server - Check connectivity as soon as optIn(true) is called (#136936)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 10:45:41 -07:00
Gloria Hornero
691e21ec2d
[Security Solution] Adds basic test for rule data view (#136822)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 19:40:19 +02:00
Paulo Henrique
711d337820
[8.4] [Kubernetes Security] - adding Collapse / Expand button to Tree View Navigation (#136782) 2022-07-25 10:29:59 -07:00
Philippe Oberti
19aa51e5a8
[TIP] Add new Threat intelligence plugin (#136479)
* [TIP] Add Threat Intelligence plugin

- create Threat Intelligence plugin and integrate with Security Solution plugin
- setup jest unit tests, i18n, Cypress tests and  Storybook
- fetch Indicator of Compromise, and display in data-grid
- add flyout components to show IOCs details (table and JSON)
- add new threatIntelInt entry to kbn-doc-links package

https://github.com/elastic/security-team/issues/4329
https://github.com/elastic/security-team/issues/4138
https://github.com/elastic/security-team/issues/4241
https://github.com/elastic/security-team/issues/4242
https://github.com/elastic/security-team/issues/4244
https://github.com/elastic/security-team/issues/4245

Co-authored-by: lgmys <lgmys@pm.me>
Co-authored-by: Maxim Kholod <maxim.kholod@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 19:27:05 +02:00
Shahzad
b127231787
[Synthetics] Add availability sparklines on summary page (#135292) 2022-07-25 19:17:13 +02:00
JD Kurma
af2557b31a
[Security Solution] Add credential_access to allow filter list (#136232)
* add crendential_access to allow filter list

* unit test for credential_access
2022-07-25 10:04:54 -07:00
Dima Arnautov
2e6a96701d
[ML] Limit threads per allocation to 8 in the UI control (#137061) 2022-07-25 18:42:41 +02:00
Cristina Amico
f4348e6c9c
[Fleet] Add basic cypress tests for download source section (#137079) 2022-07-25 09:35:49 -07:00
Spencer
0596007f64
[inspector] remove root-level redirect to common (#136999)
* [inspector] remove root-level redirect to common

* autofix all imports
2022-07-25 11:33:51 -05:00
Ievgen Sorokopud
b6b53d152c
[Security Solution][Alerts] Threshold alert history state is not always pruned correctly (#129821) (#136538)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 18:31:16 +02:00
Rickyanto Ang
170fa429ae
[Kubernetes Security][Bug Fix] Container Images widget bug fix (#136696)
* added fix for issue 136575

* padding and font updates

* added copy to clipboard button, pr comments

* fix check type

* pr comments

* PR comments

* fix check fail

* Add more media queries to adjust flex item margins on smaller screens

Co-authored-by: Jack <zizhou.wang@elastic.co>
2022-07-25 09:10:51 -07:00
Christiane (Tina) Heiligers
d80890467f
Migrate core's metrics service to packages (#136909)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-07-25 09:09:33 -07:00