* [type-summarizer] reimplement for broader support
* Enable sourceMaps in all packages
* include naming collision in summarizePackage test
* fix readmes
* remove unnecessary transient dependency
* remove code that was commented out
* remove outdated todo comment
* ensure errors triggered by untyped-exports are ligible
* remove unused import
* break out snippet generation from AstIndexer
* refactor several massive files into smaller pieces and add more inline docs
* fix typos
* update jest snapshots
* add sections to readme that points people to the useful parts of the source code along with a high-level overview of how the type-summarizer works
* remove --dump flag, it doesn't work
* use decName instead of calling names.get a second time
* include `export` as invalid name
* restructure performance tests for per journey config
* add cleanup hook to performance service for browser instance cleanup
* only enable APM in performance configs when TEST_PERFORMANCE_PHASE is set
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Spencer <spencer@elastic.co>
Adds a new experimental Kibana setting called `csp.disableUnsafeEval` which will default to `false`. When set to `true`, it will remove `unsafe-eval` from our CSP.
Also introduces a new module called `@kbn/handlebars` which is a replacement for the official `handlebars` module used in the frontend. This new module is necessary in order to avoid calling `eval`/`new Function` from within `handlebars` which is not allowed once `unsafe-eval` is removed from our CSP.
The `@kbn/handlebars` module is simply an extension of the main `handlebars` module which adds a new compile function called `compileAST` (as an alternative to the regular `compile` function). This new function will not use code-generation from strings to compile the template but will instead generate an AST and return a render function with the same API as the function returned by the regular `compile` function.
This is a little bit slower method, but since this is only meant to be used client-side, the slowdown should not be an issue.
The following limitations exists when using `@kbn/handlebars`:
The Inline partials handlebars template feature is not supported.
Only the following compile options will be supported:
- `knownHelpers`
- `knownHelpersOnly`
- `strict`
- `assumeObjects`
- `noEscape`
- `data`
Only the following runtime options will be supported:
- `helpers`
- `blockParams`
- `data`
Closes#36311
* chore(NA): upgrade bazel into v5
* chore(NA): initial work to upgrade to rules v5
* chore(NA): remove @elastic/datemath jest unit test for eslint plugin imports
* chore(NA): removed packager tsc_wrapped added my mistake
The exclamation mark prefix doesn't have any effect in the `files` list.
You instead need to override the matched path later with a different
rule in order to change the behavior.
* [ts] enable sourcemaps in summarized types of @kbn/crypto
* update snapshots
* remove unnecessary exports of @kbn/type-summarizer package
* remove tsc from the build process
* use `@kbn/type-summarizer` to summarize its own types
* add tests for interface and function
* switch to export type where necessary
* ignore __tmp__ in global jest preset
* ignore __tmp__ globally
* remove `@kbn/crypto` types path
* chore(NA): splits types from code on @kbn/rule-data-utils
* chore(NA): remove old style imports for this pkg
* chore(NA): eslint fix
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* [kbn/rule-data-utils] add submodules and require public use them
* fix lint errors
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes: https://github.com/elastic/kibana/issues/114535
**What this linter rule does:**
* Sets the [@typescript-eslint/no-non-null-assertion](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/docs/rules/no-non-null-assertion.md) linter rule to become an error if seen.
If you try to use the `!` operator you get an error and nice helper message that tries to encourage better practices such as this one:
<img width="1635" alt="Screen Shot 2021-10-07 at 11 26 14 AM" src="https://user-images.githubusercontent.com/1151048/136474207-f38d3461-0af9-4cdc-885b-632cb49d8a24.png">
**Why are we deciding to set this linter rule?**
* Recommended from Kibana [styleguide](https://github.com/elastic/kibana/blob/master/STYLEGUIDE.mdx#avoid-non-null-assertions) for ~2 years now and still recommended.
* A lot of TypeScript has evolved and has operators such as `?` which can replace the `!` in most cases. Other cases can use a throw explicitly or other ways to manage this.
* Some types can change instead of using this operator and we should just change the types.
* TypeScript flows have improved and when we upgrade the linter will cause errors where we can remove the `!` operator which is 👍 better than leaving them when they're not needed anymore.
* Newer programmers and team members sometimes mistake it for the `?` when it is not the same thing.
* We have had past bugs and bugs recently because of these.
* It's easier to use the linter to find bugs than to rely on manual tests.
**How did Frank fix all the 403 areas in which the linter goes off?**
* Anywhere I could remove the `!` operator without side effects or type script errors I just removed the `!` operator.
* Anywhere in test code where I could replace the code with a `?` or a `throw` I went through that route.
* Within the code areas (non test code) where I saw what looks like a simple bug that I could fix using a `?? []` or `?` operator or `String(value)` or fixing a simple type I would choose that route first. These areas I marked in the code review with the words `NOTE:` for anyone to look at.
* Within all other areas of the code and tests where anything looked more involved I just disabled the linter for that particular line. When in doubt I chose this route.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary
Fixes https://github.com/elastic/security-team/issues/1759
Related earlier PR, https://github.com/elastic/kibana/pull/109722, where these were removed to where they could no longer function. This PR adds them back to where they will function for existing users. The end goal is to have users naturally migrate as they update, enable/disable, or create new rules.
What this PR does:
* Adds back the legacy side car actions `siem-detection-engine-rule-actions`
* Adds back the legacy hidden alert of `siem.notifications`
* Adds back unit tests where they existed. Both of these systems did not have existing e2e tests.
* Re-adds the find feature and functionality which should show the rules with legacy and non-legacy notifications/side car actions during a REST find operation.
* Updates the logic for when to show a legacy vs. non-legacy notification/side car action.
* Adds a new route called `/internal/api/detection/legacy/notifications` which is only for developer and tests for us to maintain this system for the foreseeable future.
* Adds script to exercise creating old notifications `detection_engine/scripts/post_legacy_notification.sh`
* Adds a data file for the script to use as an example for ad-hoc testing, `scripts/legacy_notifications/one_action.json`
* Adds within `security_solution/server/types.ts` `ActionsApiRequestHandlerContext` so that if we need to directly access actions within plugins we can. I do not use it here, but it should have been existing there and is good to have it in case we need it at this point within REST routes.
* When adding back the files and changes, I use the kibana-core approach of prefixing files, functions, types, etc... with the words `legacyFoo`. The files are named `legacy_foo.ts`. Everything has `@deprecation` above them as well. The intent here is all of this should hopefully make it unambiguously clear which parts of the notification system are for the new system/existing API and which ones are only for the deprecated legacy system. There exists some parts of the system that are used within _both_ and the hope is that we can keep the legacy pieces separate from the non-legacy pieces for strangling the legacy pieces.
* This adds a new linter rule to prevent users from easily importing files named `legacy_foo.ts` or `foo_legacy.ts` we are using here and can also use for other similar legacy parts of the system we have. This seems to be the established pattern that kibana-core does as well looking through the linters and code base.
* Removes some dead import/export code and types instead of maintaining them since they are no longer used.
What this PR does not do (but are planned on follow ups):
* This PR does not add migration logic in most conditions such as a user enabling/disabling a rule, editing a rule unless the user is explicitly changing the actions by turning off the notification and then re-adding the notification.
* This PR does not log any information indicating to the user that they are running legacy rules or indicates they have that.
* This PR does not allow the executors or any UI/UX, backend to re-add a legacy notification. Instead only the hidden REST route of `/internal/api/detection/legacy/notifications` allows us to do this for testing purposes.
* This PR does not migrate the data structure of actions legacy notification system `siem-detection-engine-rule-actions` to use saved object references.
* If you delete an alert this will not delete the side car if it detects one is present on it.
* If you update an alert notification with a new notification this will not remove the side car on the update.
**Ad-hoc testing instructions**
How to do ad-hoc testing for various situations such as having a legacy notification system such as a user's or if you want to mimic a malfunction and result of a "split-brain" to where you have both notification systems running at the same time due to a bug or regression:
Create a rule and activate it normally within security_solution:
<img width="1046" alt="Screen Shot 2021-09-22 at 2 09 14 PM" src="https://user-images.githubusercontent.com/1151048/134416564-e4e001a7-1086-46a1-aa8d-79880f70cc35.png">
Do not add actions to the rule at this point as we will first exercise the older legacy system. However, you want at least one action configured such as a slack notification:
<img width="575" alt="Screen Shot 2021-09-22 at 2 28 16 PM" src="https://user-images.githubusercontent.com/1151048/134417012-58e63709-5447-4832-8866-f82be1b9596b.png">
Within dev tools do a query for all your actions and grab one of the `_id` of them without their prefix:
```json
# See all your actions
GET .kibana/_search
{
"query": {
"term": {
"type": "action"
}
}
}
```
Mine was `"_id" : "action:879e8ff0-1be1-11ec-a722-83da1c22a481",` so I will be copying the ID of `879e8ff0-1be1-11ec-a722-83da1c22a481`
Go to the file `detection_engine/scripts/legacy_notifications/one_action.json` and add this id to the file. Something like this:
```json
{
"name": "Legacy notification with one action",
"interval": "1m", <--- You can use whatever you want. Real values are "1h", "1d", "1w". I use "1m" for testing purposes.
"actions": [
{
"id": "879e8ff0-1be1-11ec-a722-83da1c22a481", <--- My action id
"group": "default",
"params": {
"message": "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
},
"actionTypeId": ".slack" <--- I am a slack action id type.
}
]
}
```
Query for an alert you want to add manually add back a legacy notification to it. Such as:
```json
# See all your siem.signals alert types and choose one
GET .kibana/_search
{
"query": {
"term": {
"alert.alertTypeId": "siem.signals"
}
}
}
```
Grab the `_id` without the `alert` prefix. For mine this was `933ca720-1be1-11ec-a722-83da1c22a481`
Within the directory of `detection_engine/scripts` execute the script
```bash
./post_legacy_notification.sh 933ca720-1be1-11ec-a722-83da1c22a481
{
"ok": "acknowledged"
}
```
which is going to do a few things. See the file `detection_engine/routes/rules/legacy_create_legacy_notification.ts` for the definition of the route and what it does in full, but we should notice that we have now:
Created a legacy side car action object of type `siem-detection-engine-rule-actions` you can see in dev tools:
```json
# See the actions "side car" which are part of the legacy notification system.
GET .kibana/_search
{
"query": {
"term": {
"type": {
"value": "siem-detection-engine-rule-actions"
}
}
}
}
```
Note in the response:
```json
"siem-detection-engine-rule-actions" : {
"ruleAlertId" : "933ca720-1be1-11ec-a722-83da1c22a481", <--- NOTE, not migrated to references yet
"actions" : [
{
"action_type_id" : ".slack",
"id" : "879e8ff0-1be1-11ec-a722-83da1c22a481", <--- NOTE, not migrated to references yet
"params" : {
"message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
},
"group" : "default"
}
],
"ruleThrottle" : "1m", <--- Should be the same as the interval in "one_action.json" config
"alertThrottle" : "1m" <--- Should be the same as the interval in "one_action.json" config
},
"type" : "siem-detection-engine-rule-actions",
"references" : [ ],
```
Created a `siem.notification` rule instance which you can see in dev tools as well:
```json
# Get the alert type of "siem-notifications" which is part of the legacy system.
GET .kibana/_search
{
"query": {
"term": {
"alert.alertTypeId": "siem.notifications"
}
}
}
```
Take note from the `siem.notifications` these values which determine how/when it fires and if your actions are set up correctly:
```json
"name" : "Legacy notification with one action" <--- Our name from one_action.json
"schedule" : {
"interval" : "1m" <--- Interval should match interval in one_action.json
},
"enabled" : true, <--- We should be enabled
"actions" : [
{
"group" : "default",
"params" : {
"message" : "Hourly\nRule {{context.rule.name}} generated {{state.signals_count}} alerts"
},
"actionTypeId" : ".slack", <--- Our actionID
"actionRef" : "action_0"
}
],
```
And that now there exists a task within task manager that will be executing this:
```json
# Get the tasks of siem notifications to ensure and see it is running
GET .task-manager/_search
{
"query": {
"term": {
"task.taskType": "alerting:siem.notifications"
}
}
}
```
You can double check the interval from the result of the query to ensure it runs as the configuration test file shows it should be:
```json
"schedule" : {
"interval" : "1m"
},
```
Within time you should see your action execute like the legacy notification system:
<img width="876" alt="Screen Shot 2021-09-22 at 2 55 28 PM" src="https://user-images.githubusercontent.com/1151048/134422639-80523abb-f43c-4f7c-abef-a60062bef139.png">
If you go to edit the rule you should notice that the rule now has the side car attached to it within the UI:
<img width="1050" alt="Screen Shot 2021-09-22 at 8 08 54 PM" src="https://user-images.githubusercontent.com/1151048/134445265-fa0a330b-3238-48e2-aef3-6042c7e9aa69.png">
You can also look at your log messages in debug mode to verify the behaviors of the legacy system and the normal rules running.
Compare these data structures to a 7.14.x system in cloud to ensure the data looks the same and the ad-hoc testing functions as expected.
Check the scripts of `./find_rules.sh`, `./read_rules.sh` to ensure that the find REST route returns the legacy actions when they are there.
### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* [eslint] add rule to prevent export* in plugin index files
* deduplicate export names for types/instances with the same name
* attempt to auto-fix duplicate exports too
* capture exported enums too
* enforce no_export_all for core too
* disable rule by default, allow opting-in for help fixing
* update tests
* reduce yarn.lock duplication
* add rule but no fixes
* disable all existing violations
* update api docs with new line numbers
* revert unnecessary changes to yarn.lock which only had drawbacks
* remove unnecessary eslint-disable
* rework codegen to split type exports and use babel to generate valid code
* check for "export types" deeply
* improve test by using fixtures
* add comments to some helper functions
* disable fix for namespace exports including types
* label all eslint-disable comments with related team-specific issue
* ensure that child exports of `export type` are always tracked as types
Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* First pass add to case action in timelines plugin
* Fix fake duplicate import lint rule and some type errors
* Fix some tests
* Remove use_insert_timeline and pass as prop
* Remove unneeded ports, fix types/tests
* Finish fixing types and tests for add to case action
* Remove duplicated security_solution code
* Pass appId as props
* Fix lint and a type error
* Use react-router-dom instead of window.location.search
* Fix broken test
* Remove unused imports
* Remove unused export and related code
* expression_reveal_image skeleton.
* expression_functions added.
* expression_renderers added.
* Backup of daily work.
* Fixed errors.
* Added legacy support. Added button for legacy.
* Added storybook.
* Removed revealImage from canvas.
* Types fixed.
* Fixed test suite error.
* Fixed eslint error.
* Moved UI and elements, related to expressionRevealImage from canvas.
* Fixed unused translations errors.
* Moved type of element to types.
* Fixed types and added service for representing elements, ui and supported renderers to canvas.
* Added expression registration to canvas.
* Fixed
* Fixed mutiple call of the function.
* Removed support of a legacy lib for revealImage chart.
* Removed legacy presentation_utils plugin import.
* Removed useless translations and tried to fix error.
* One more fix.
* Small imports fix.
* Fixed translations.
* Made fixes based on nits.
* Removed useless params.
* fix.
* Fixed errors, related to jest and __mocks__.
* Removed useless type definition.
* Replaced RendererHandlers with IInterpreterRendererHandlers.
* fixed supported_shareable.
* Moved elements back to canvas.
* Moved views to canvas, removed expression service and imported renderer to canvas.
* Fixed translations.
* Moved libs to presentation utils.
* Fixed types and removed function_wrapper.ts
* Fixed types of test helpers.
* Fixed imports.
* One more fix.
* Fixed public API.
* Moved css to component.
* Fixed spaces at element.
* Removed unused plugin.
* Basic setup of error plugin.
* Removed not used `function` files at `error` expression.
* Moved related components from canvas.
* Changed imports of components.
* Removed useless translations and fixed .i18nrc.json
* More fixes of i18nrc.
* Fixed async functions.
Written current code, based on https://github.com/storybookjs/storybook/issues/7745
* Fixed one test with Expression input.
After changing the way of rendering in stories, all elements are mounting and componentDidMount is involved. The previous snapshot was without
mounted `monaco` editor.
* generated plugin and copied code from expression_reveal_image
* fixed double import after merge.
* Changed all names from reveal_image to shape.
* moved shape to plugin and added all necessary configs
* Fixed translations, fixed all imports and debug of svg.
* `function` moved to `server`.
* One shape is rewritten to `React` and rendering is written with passing necessary props.
* changed default width and heigth.
* Added `ShapeHOC`.
* Shapes changed.
* small refactor.
* Removed useless import.
* one more refactor.
* Refactor + fix errors + updated limits.
* Changed ShapePreview from pure js to react and removed `dangerouslySetInnerHTML`
* Fixed types of viewbox.
* Changed types source for Shape components.
* small refactor.
* Fixed imports.
* Removed `shape` from `canvas`
* Updated docs.
* Basic setup of error plugin.
* Removed not used `function` files at `error` expression.
* Changed imports of components.
* Fixed errors, related to shape and autosuggestions.
* Fixed i18n for shape.
* Moved function from public to common and registered at server.
* Fixed types error.
* Fixed snapshots and shape mocks.
* Moved some libs from `presentations_util` to `expression_shape`
* Shape refactored.
* Shape picker fixed.
* Moved `Popover` back to `canvas`
* Removed `Popover` export from presentation_utils components.
* Moved error_component and debug_component from presentation_util to expression_error.
* Removed `.i18nrc.json`.
* Removed `.i18nrc.json`.
* Removed useless scss.
* Fixed color of `error`.
* added fixes of rebase.
* More fixes of rebase error .
* Removed useless .i18nrc.json file.
* More fixes.
* More fixes of rebase.
* One more fix.
* More fixes.
* Fixed limits and translations.
* Added.
* Fixed i18nrc.
* Fixed error..
* Moved shapes to async chunks.
* One more fix.
* Some fixes.
* Trying to fix the typecheck error.
* Added temp of drawer.
* Moved shapes to the async chunk in a less complex way.
* Made `ShapeDrawer` reusable among different `expressions`.
* Changed type of `shapes` from `any` and `Shape` to `string`.
* Made changes, based on nits.
* Removed not necessary changes.
* Moved all reusable libs to `expression_shapes`.
* Reduced the size of the bundle.
* Hope, fixed type check errors.
* Removed getDefaultShapeData.
* Removed `getViewBox` from bundle.
* expression_reveal_image skeleton.
* expression_functions added.
* expression_renderers added.
* Backup of daily work.
* Fixed errors.
* Added legacy support. Added button for legacy.
* Added storybook.
* Removed revealImage from canvas.
* setState while rendering error fixed.
* tsconfig.json added.
* jest.config.js added.
* Demo doc added.
* Types fixed.
* added limits.
* Removed not used imports.
* i18n namespaces fixed.
* Fixed test suite error.
* Some errors fixed.
* Fixed eslint error.
* Removed more unused translations.
* Moved UI and elements, related to expressionRevealImage from canvas.
* Fixed unused translations errors.
* Moved type of element to types.
* Fixed types and added service for representing elements, ui and supported renderers to canvas.
* Added expression registration to canvas.
* Fixed
* Fixed mutiple call of the function.
* Removed support of a legacy lib for revealImage chart.
* Removed legacy presentation_utils plugin import.
* Doc error fixed.
* Removed useless translations and tried to fix error.
* One more fix.
* Small imports fix.
* Fixed translations.
* Made fixes based on nits.
* Removed useless params.
* fix.
* Fixed errors, related to jest and __mocks__.
* Removed useless type definition.
* Replaced RendererHandlers with IInterpreterRendererHandlers.
* fixed supported_shareable.
* Moved elements back to canvas.
* Moved views to canvas, removed expression service and imported renderer to canvas.
* Fixed translations.
* Types fix.
* Moved libs to presentation utils.
* Fixed one mistake.
* removed dataurl lib.
* Fixed jest files.
* elasticLogo removed.
* Removed elastic_outline.
* removed httpurl.
* Removed missing_asset.
* removed url.
* replaced mostly all tests.
* Fixed types.
* Fixed types and removed function_wrapper.ts
* Fixed types of test helpers.
* Changed limits of presentationUtil plugin.
* Fixed imports.
* One more fix.
* Fixed huge size of bundle.
* Reduced allow limit for presentationUtil
* Updated limits for presentationUtil.
* Fixed public API.
* fixed type errors.
* Moved css to component.
* Fixed spaces at element.
* Changed order of requiredPlugins.
* Updated limits.
* Removed unused plugin.
* Added rule for allowing import from __stories__ directory.
* removed useless comment.
* Changed readme.md
* Fixed docs error.
* A possible of smoke test.
* onResize changed to useResizeObserver.
* Remove useless events and `useEffect` block.
* Changed from passing handlers to separate functions.
* `function` moved to `server`.
* Fixed eslint error.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* wip
* First pass at standalone and embedded redux stores and usage
* wip
* First pass at standalone and embedded redux stores and usage
* wip
* clean up
* wip
* refact(NA): remove extra pkg_npm target and add specific target folders on @kbn/i18n
* cleanup
* - fixes type errors in tests
* WIP remove use_manage_timeline
* wip add query + selector
* finishing integrating timeline manage context from redux
* integrating t-grid in security solution
* fix RowRender type
* WIP begin to move components from package to plugin
* integration of t-grid inside of security solution
* wip to make redux work
* little trick to make it render
* - fixes a few type errors
* better integration betwen tgrid and security solutions
* bringing back tsconfig on timeline
* wip integration t-grid in observability
* fix types
* fix type in security solutions
* add type to import + trie dto get the bundle size as small as possible
* fix type in integration test
* fix type in integration test
* - fix tests
* clean up to use technical fields
* - fixes unit tests
* - mocks the `useDateFormat` function of the `useKibana` service to fix unit tests
* fix t-grid settings vs create timeline + fix inspect button
* fix last suites test
* Update unit tests, snapshots and lint
* Fix bad merge
* fix plugin export
* Fix some failing tests
* fix unit tets in timelines plugins
* fix latest test
* fix i18n
* free obs from t-grid
* Fix timeline functional plugin types
* fix store provider
* Update failing defaultHeader test
* Fix i18n usage in security solution
* Fix remaining i18n errors in timelines plugin
* Dedupe common shared types
* move drag and drop utils in package to avoid duplication
* More shared type cleanup
* add feature flag
* review I
* fix merge with master
* fix i18n translation
* More type deduping
* Use @kbn/common-utils, fix remaining types
* fix types
* fix tests
* missing type
* fix cypress tests
Co-authored-by: Kevin Qualters <kevin.qualters@elastic.co>
Co-authored-by: Tiago Costa <tiagoffcc@hotmail.com>
Co-authored-by: Andrew Goldstein <andrew.goldstein@elastic.co>
