Commit graph

102 commits

Author SHA1 Message Date
Kibana Machine
78e9b7652a
[8.x] chore(tsconfig): uncomment option `moduleDetection` (#191834) (#193481)
# Backport

This will backport the following commits from `main` to `8.x`:
- [chore(tsconfig): uncomment option `moduleDetection`
(#191834)](https://github.com/elastic/kibana/pull/191834)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Alejandro Fernández
Haro","email":"alejandro.haro@elastic.co"},"sourceCommit":{"committedDate":"2024-09-19T20:04:14Z","message":"chore(tsconfig):
uncomment option `moduleDetection`
(#191834)","sha":"f8ad0f1ec920600de6ede16e54c2e2aafbe5ce57","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["chore","Team:Operations","release_note:skip","Feature:Drilldowns","Team:Fleet","v9.0.0","Team:Search","backport:prev-minor"],"title":"chore(tsconfig):
uncomment option
`moduleDetection`","number":191834,"url":"https://github.com/elastic/kibana/pull/191834","mergeCommit":{"message":"chore(tsconfig):
uncomment option `moduleDetection`
(#191834)","sha":"f8ad0f1ec920600de6ede16e54c2e2aafbe5ce57"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191834","number":191834,"mergeCommit":{"message":"chore(tsconfig):
uncomment option `moduleDetection`
(#191834)","sha":"f8ad0f1ec920600de6ede16e54c2e2aafbe5ce57"}}]}]
BACKPORT-->

Co-authored-by: Alejandro Fernández Haro <alejandro.haro@elastic.co>
2024-09-20 05:24:14 -05:00
Maxim Palenov
ecbe36de17
[Security Solution] Add missing Endpoint Exceptions API OpenAPI specifications (#186082)
**Resolves:** https://github.com/elastic/kibana/issues/183839

## Summary

This PR adds missing OpenAPI specifications for Endpoint exceptions API which are the following

- `POST /api/endpoint_list`
- `POST /api/endpoint_list/items`
- `GET /api/endpoint_list/items`
- `PUT /api/endpoint_list/items`
- `DELETE /api/endpoint_list/items`
- `GET /api/endpoint_list/items/_find`
2024-07-29 16:19:12 +02:00
Maxim Palenov
4d7c36cee9
[Security Solution] Add missing Exceptions API OpenAPI specifications (#185951)
**Resolves:** https://github.com/elastic/kibana/issues/183837

## Summary

This PR adds missing OpenAPI specifications for Exceptions API which are the following

- `POST /api/exception_lists/_export`
- `POST /api/exception_lists/_import`
- `POST /api/exception_lists`
- `GET /api/exception_lists`
- `PUT /api/exception_lists`
- `DELETE /api/exception_lists`
- `GET /api/exception_lists/_find`
- `POST /api/exception_lists/_duplicate`
- `POST /api/exception_lists/items`
- `GET /api/exception_lists/items`
- `PUT /api/exception_lists/items`
- `DELETE /api/exception_lists/items`
- `GET /api/exception_lists/items/_find`
- `GET /api/exception_lists/summary`
- `POST /api/exceptions/shared` 
- `POST /api/detection_engine/rules/{id}/exceptions`
2024-07-15 13:12:56 +02:00
Maxim Palenov
7a0065d5b6
[Security Solution] Add missing Lists API OpenAPI specifications (#185865)
**Resolves:** https://github.com/elastic/kibana/issues/183821

## Summary

This PR adds missing OpenAPI specifications for Lists API which are the following

- `POST /api/lists`
- `GET /api/lists`
- `PUT /api/lists`
- `DELETE /api/lists`
- `PATCH /api/lists`
- `GET /api/lists/_find`
- `GET /api/lists/privileges`
- `POST /api/lists/items`
- `GET /api/lists/items`
- `PUT /api/lists/items`
- `DELETE /api/lists/items`
- `PATCH /api/lists/items`
- `POST /api/lists/items/_export`
- `POST /api/lists/items/_import`
- `GET /api/lists/items/_find`
- `POST /api/lists/index`
- `GET /api/lists/index`
- `DELETE /api/lists/index`

**Note:** Code generation is enabled for the added specs to verify that it works and produces expected results. Generated Zod schemas and types aren't integrated in the route's code.
2024-06-18 13:22:19 -07:00
Wafaa Nasr
a4ed14bec8
[Security Solution][API testing] Move and restructures Lists APIS (#171992)
## Summary

Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
List APIs tests
- Added a new folder under the `security_solution_api_integration`
called `lists_and_exception_lists` to hold the lists and exception lists
tests, and split the `List` APIs into two groups since the execution
time in Serverless was close to 30 mins
- Modified the
[x-pack/plugins/lists/common/schemas/response/exception_list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-4e3545fdeb8c8d9467cfa1c4aa88194e189193a92fa6f1cf5f859b1ef1beb45c),
[x-pack/plugins/lists/common/schemas/response/exception_list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-c3cc18faf07aab86e307185d41599c3596a3f8b360d3e4829591afa148283238)
,
[x-pack/plugins/lists/common/schemas/response/list_item_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-608579ca5e65da74f41319a58d81ab12cc3d79d389b087806c7b74949fbc6cc3),
[x-pack/plugins/lists/common/schemas/response/list_schema.mock.ts](https://github.com/elastic/kibana/pull/171992/files#diff-efc64eb35937a8da28fc982c527253c0923650ae4163d4bbc203d3ebc2949835)
to accept `elastic user` input because it changes in ESS and Serverless
- Deleted the `x-pack/test/lists_api_integration` folder
- Moved the utility files associated with Basic tests to the new
directory `security_solution_api_integration`. Files not actively used
in the previous folder were moved, while duplicate files remained in
their original positions.
- Updated the below files imports from the old `lists_api_integration`
folder to the new `lists_and_exception_lists`

    ```
      lists_api_integration/  
        - exceptions/operators_data_types/date_numeric_types/date.ts
        - exceptions/operators_data_types/date_numeric_types/double.ts
        - exceptions/operators_data_types/date_numeric_types/float.ts 
        - exceptions/operators_data_types/date_numeric_types/integer.ts
        - exceptions/operators_data_types/ips/ip.ts
        - exceptions/operators_data_types/ips/ip_array.ts
        - exceptions/operators_data_types/keyword/keyword.ts
        - exceptions/operators_data_types/keyword/keyword_array.ts
        - exceptions/operators_data_types/long/long.ts
        - exceptions/operators_data_types/text/text.ts
        - exceptions/operators_data_types/text/text_array.ts
        - exceptions/workflows/create_endpoint_exceptions.ts
        - exceptions/workflows/create_rule_exceptions.ts
        - exceptions/workflows/find_rule_exception_references.ts
        - exceptions/workflows/role_based_add_edit_comments.ts
        - exceptions/workflows/role_based_rule_exceptions_workflows.ts
        - exceptions/workflows/rule_exception_synchronizations.ts
        - rule_execution_logic/execution_logic/esql.ts
        - rule_execution_logic/execution_logic/machine_learning.ts
        - rule_execution_logic/execution_logic/new_terms.ts
        - rule_execution_logic/execution_logic/query.ts
        - telemetry/task_based/all_types.ts
        - telemetry/task_based/detection_rules.ts
        - telemetry/task_based/security_lists.ts
     ```
     ```
These files should be moved too soon to the new `lists_api_integration`
        detection_engine_api_integration/security_and_spaces/group10
            - import_export_rules.ts 
            - import_rules.ts
            - perform_bulk_actions   
     ```

- Updated the below files imports to the
`ftr_provider_context_with_spaces.d.ts`
` - risk_engine/risk_scoring_task/task_execution_nondefault_spaces.ts`

- The QA phase concluded with all tests passing successfully. 🟢  

- Updated the CodeOwner file for the newly moved tests

- Add a new util file to `deleteAllExceptions` under the old
`detection_engine_api` folder since the Rule management related-tests
are still need to be moved over to the new folder

- Old/new group details, decisions, and execution time are mentioned in
this
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-11-28 16:40:09 +01:00
Vitalii Dmyterko
505d8265c8
[Security Solution][Detection Engine] move lists to data stream (#162508)
## Summary

- addresses https://github.com/elastic/security-team/issues/7198
- moves list/items indices to data stream
  - adds `@timestamp` mapping to indices mappings
- migrate to data stream if indices already exist(for customers < 8.11)
or create data stream(for customers 8.11+ or serverless)
- adds
[DLM](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/data-streams-put-lifecycle.html)
to index templates
- replaces update/delete queries with update_by_query/delete_by_query
which supported in data streams
  - fixes existing issues with update/patch APIs for lists/items
    - update/patch for lists didn't save `version` parameter in ES
- update and patch APIs for lists/items were identical, i.e. for both
routes was called the same `update` method w/o any changes

<details>

<summary>Technical detail on moving API to
(update/delete)_by_query</summary>


`update_by_query`, `delete_by_query` do not support refresh=wait_for,
[only false/true
values](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/docs-update-by-query.html#_refreshing_shards_2).
Which might break some of the use cases on UI(when list is removed, we
refetch all lists. Deleted list will be returned for some time. [Default
refresh time is
1s](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/docs-refresh.html)).
So, we retry refetching deleted/updated document before finishing
request, to return reindexed document

`update_by_query` does not support OCC [as update
API](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/optimistic-concurrency-control.html).
Which is supported in both
[list](https://www.elastic.co/guide/en/security/current/lists-api-update-container.html)/[list
item
](https://www.elastic.co/guide/en/security/current/lists-api-update-item.html)updates
through _version parameter.
_version is base64 encoded "_seq_no", "_primary_term" props used for OCC

So, to keep it without breaking changes: implemented check for version
conflict within update method
</details>

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-23 19:42:57 +01:00
Wafaa Nasr
a2275dd346
[Security Solution] [Detection Engine] Fix exception comment flakiness (#162807)
## Summary

- Addresses https://github.com/elastic/kibana/issues/162565 
- Converting the `Comments` related Cypress test to a FTR test according
to these
[observations](https://github.com/elastic/kibana/issues/162565#issuecomment-1653410937)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-01 14:58:40 +01:00
Marshall Main
5da645acee
[Security Solution] Move lists plugin API schemas to /common/api (#162489)
Closes https://github.com/elastic/security-team/issues/7102

Establishes the `/common/api` folder structure for the lists plugin.
This PR simply imports and re-exports the schemas from the schema
package since the schemas are all already separated from the `server`
code. Future PRs will replace these re-exports with schemas generated
from OpenAPI specs.
2023-07-27 15:22:00 -07:00
Wafaa Nasr
005108684b
[Security Solution][Exceptions]- Increase exceptions test coverage (#152757)
## Summary

- Addresses https://github.com/elastic/security-team/issues/5947
- Adding tests to cover the yellow rows in [test
sheet](https://docs.google.com/spreadsheets/d/1Eb_317s7nkQ4axVA270Ja99PRS-NWrYZAEc-1aVuyXg/edit#gid=0)
- Organise the tests to correspond to the following
[structure](https://docs.google.com/spreadsheets/d/14DdtghpxgfEmWoc7kot4XgEva_4GDEC_uTej65MUjV8/edit?pli=1#gid=0)
- Removed the
`x-pack/plugins/security_solution/cypress/e2e/exceptions/alerts_table_flow/add_exception.cy.ts`
as it was duplicated from
`x-pack/plugins/security_solution/cypress/e2e/exceptions/exceptions_management_flow/exceptions_table.cy.ts`
- Skipped the `flyout_validation` tests until resolving this
[ticket](https://github.com/elastic/kibana/issues/154994)
- Regarding `Exception-List`, `Exception-List-Item` and `Rule with
exceptions` migrations test cases are handled by most of our old `FTR`
tests as most of them deal with `Exception List Schema` which doesn't
include the new props, like the `expire_time` that was introduced in
`8.7`, so adding new tests using the new schema can be treated as
testing the new versions against the existing scenarios whereas the
existing tests for the migrations (downgrade) tests
[4a75a51](4a75a51a3e)
- Tests under `x-pack/plugins/security_solution/cypress/upgrade_e2e` are
just POCs can't be used

# New tests folder structure based on workflow

<img width="432" alt="image"
src="https://user-images.githubusercontent.com/12671903/234849016-f6f227d1-fcaf-43cb-abe3-d3fc7f9cee00.png">

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-06-07 15:15:30 +01:00
Davis Plumlee
92a1689e95
[Security Solution][Exceptions] Rule exceptions TTL - Expiration (#145180) 2023-02-07 16:20:39 -05:00
Tiago Costa
016e3e03a4
chore(NA): enables isolatedModules on base tsconfig file (#144841)
This PR enables `isolatedModules` on our `tsconfig.base.json`. Enabling
this means that our codebase is safe for tools that use TypeScript APIs
like `transpileModule` or alternative compilers like Babel. The
requirements introduced by enabling `isolatedModules` were already in
place for every piece of code transpiled with babel so we feel like its
time to make it the default across the board inside our repository.

The DX shouldn't be impacted negatively by these change as we introduced
a lint rule verification for the critical part around `isolatedModules`
which is around `const enums`. The PR also has a couple of `TODOs` to be
removed once we upgrade into typescript v4.8 where we would be able to
say everything that is typescript inside our repo should be consider a
module by default.

More information about `isolatedModules` can be found at
https://www.typescriptlang.org/tsconfig#isolatedModules

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-11-18 19:39:11 +00:00
Davis Plumlee
51699fa21a
[Security Solution] Value list exceptions (#133254) 2022-09-19 22:41:28 +02:00
Marshall Main
aaa3107dbc
[Security Solution][Alerts] Detection engine wildcard exceptions (#136147)
* Implement wildcard exceptions for detection rules

* Fix index pattern retrieval on edit exceptions flyout

* Fix API integration test logic

* Fix entry_renderer linting

* Remove bad fix idea

* Add 'does not match' operator to UI

* Fix test

* Add unit tests

* Add wildcard exceptions to list of DE exception operators

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-07-19 15:03:47 -07:00
spalger
3730dd0779 fix all violations 2022-04-16 01:37:30 -05:00
Madison Caldwell
11bba0a04b
[Security Solution] Consider exceptions when loading threshold alert timelines (#128495)
* Add exceptions to threshold timeline

* Tests and error handling

* Fix unit tests

* Add alias for exceptions filter

* Fix tests

* Type fixes

Co-authored-by: Marshall Main <marshall.main@elastic.co>
2022-03-29 15:09:55 -04:00
Ashokaditya
185570221f
[Security Solution][Endpoint] Update list api summary endpoint to use filter (#123476)
* update summary endpoint to use filters and use that for fleet event filters cards

fixes elastic/security-team/issues/2513

* update tests

fixes elastic/security-team/issues/2513

* update host isolation card to show total as the actual number of artifacts

fixes elastic/kibana/issues/121507

* fix types

missing merge updates

* use named constant for isolation exception list

review changes

* Update fleet_integration_event_filters_card.tsx

review changes

* fix the total on summary api

review suggestions

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-01-26 12:58:53 -07:00
Esteban Beltran
b6753241ed
[Security Solution] host isolation exceptions listing under policy integration details tab (#120361) 2021-12-13 08:46:42 -07:00
Yara Tercero
fccdcb6dae
[Security Solution][Platform] - Exceptions imports (#118816)
## Summary

Addresses https://github.com/elastic/kibana/issues/92613 and https://github.com/elastic/kibana/issues/117399

Goal is to allow users to import their exception lists and items alongside their rules. This PR does not complete all the UI updates needed, but does tackle the majority of use cases. The bulk of the changes occur in `import_rules_route` and the new `import_exceptions_route`.

- Adds exceptions import endpoint in `lists` plugin
- Adds exceptions import logic in import rules route in `security_solution` plugin
- Adds integration tests for exception import endpoint
- Adds integration tests for rules import endpoint to account for new functionality
- Purposely not yet adding an import modal in the exceptions table UI until further list management features added (checked with product on this front)
2021-12-08 12:07:07 -07:00
Frank Hassanabad
915206531b
[Security Solutions] Removes tech debt of exporting all from linter rule for security_solution plugin (#120188)
## Summary

See: https://github.com/elastic/kibana/issues/110903

This removes the top level API `export *` spots from:
* `security_solution` plugin

by removing _all_ the exports from `security_solution/common/index.ts` since non of those were shared outside this plugin. Look at the metrics from the build below and you will see _huge_ drops off numbers across the board for required API documentation to the page load size.

In the file `security_solution/common/index.ts` I now put the advice of:

 ```
// Careful of exporting anything from this file as any file(s) you export here will cause your page bundle size to increase.
// If you're using functions/types/etc... internally it's best to import directly from their paths than expose the functions/types/etc... here.
// You should _only_ expose functions/types/etc... that need to be shared with other plugins here.
```

But really I doubt we will have to share anything from `security_solutions` plugin to another plugin or expose it for anyone else. So I think this is 👍 the way forward to not expose anything directly from `security_solution/common/index.ts` anymore.
2021-12-02 12:10:48 -07:00
Yara Tercero
2f88776eac
[Security Solution][Platform] - Update rule exported counts to include total object count (#116338)
### Summary

Addresses #116330.
2021-11-04 03:00:13 +00:00
Yara Tercero
b52a9aba1f
Exceptions export duplicates (#116698)
## Summary

Addresses https://github.com/elastic/kibana/issues/116329

Removes duplicate exception lists on rule export when multiple rules reference the same list.
2021-11-02 11:55:32 -06:00
Tyler Smalley
4681a80317
[DX] Upgrade prettier to v2.4.0 (#112359)
Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>
2021-09-19 22:34:30 -07:00
Dmitry Shevchenko
c6aa4f625c
Fix exceptions page table pagination (#111000) 2021-09-03 21:38:45 +02:00
Frank Hassanabad
0a5c96b117
[Security Solutions] Fixes exception lists to be able to filter on os type (#106494)
## Summary

Fixes https://github.com/elastic/kibana/issues/102613, and targets `7.14.0` as a blocker/critical

Previously we never fully finished the plumbing for using the `os_types` (operating system type) in the exception lists to be able to filter out values based on this type. With the endpoint exceptions now having specific selections for os_type we have to filter it with exceptions and basically make it work.

Some caveats is that the endpoints utilize `host.os.name.casless` for filtering against os_type, while agents such as auditbeat, winlogbeat, etc... use `host.os.type`. Really `host.os.type` is the correct ECS field to use, but to retain compatibility with the current version of endpoint agents I support both in one query to where if either of these two matches, then that will trigger the exceptions.

* Adds e2e tests
* Enhances the e2e tooling to do endpoint exception testing with `os_types`.
* Adds the logic to handle os_type
* Updates the unit tests

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-07-22 12:54:40 -06:00
Frank Hassanabad
aa2f5b535d
[Security Solution] Utilizes constants package and deletes duplicate code (#100513)
## Summary

Utilizes constants package and deletes duplicate code

* Renames the `securitysolution-constants` to be `securitysolution-list-constants` to be specific
* Deletes duplicated code found during cleanup
* Moves more tests into the packages found along the way with the duplicated code
* Moves `parseScheduleDates` from `@kbn/securitysolution-io-ts-types` to `@kbn/securitysolution-io-ts-utils`

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-24 18:38:14 -06:00
Frank Hassanabad
192c8ba743
Adds two more packages and moves files into the packages (#100375)
## Summary

* Adds package `kbn-securitysolution-list-api`
* Adds package `kbn-securitysolution-list-hooks`
* Moves files into the packages
* Moves a few additional types into the other packages such as the `kbn-securitysolution-io-ts-types` package to remove more things from the shard_export/shared_import between lists and security solution
* Removes more duplicated code

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-20 12:36:07 -06:00
Frank Hassanabad
9bf488e826
[Security Solutions] Re-arranges and adds more packages to remove copied code (#100310)
## Summary

* Creates a `securitysolution-list-utils` packaged and moves the first set of utilities into there
* Fixes a slight bug with `kbn-securitysolution-io-ts-list-types` where the wrong name was used
* Moves _all_ of the lists schemas and types into the package `kbn-securitysolution-io-ts-list-types`
* Removes copied code found in a few places

## Tech debt
* Some spots I have to use an `any` in the package as Kibana kbn packages don't have the types I need
* Some spots I copy constants until we can straighten out those pieces.
* I keep copied mock files until we figure out how to share mocks from these packages without adding weight or we create dedicated mock packages for all of this. 


### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-19 15:59:52 -06:00
Frank Hassanabad
e15b887e43
[Security Solutions] Replaces most deprecated io-ts alerting and list types (#100234)
## Summary

Replaces most of the deprecated io-ts alerting and list types within securitysolution as part of Phase 3 of 4 phases outlined in earlier PR's such as https://github.com/elastic/kibana/pull/99260

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-17 23:28:06 -06:00
Frank Hassanabad
bfe08d25c5
[Security Solutions] Removes deprecation and more copied code between security solutions and lists plugin (#100150)
## Summary

* Removes deprecations 
* Removes duplicated code

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-14 16:56:08 -06:00
Frank Hassanabad
7dd29a56ad
[Security Solutions] Breaks down the io-ts packages to decrease plugin size (#100058)
## Summary

The io-ts package was too large and needed to broken down more by domain to decrease the lists plugin size and any other plugin wanting to use the packages will not incur big hits as well.

Before we had one large io-ts package:

```
@kbn/securitysolution-io-ts-utils
```

Now we have these broken down 4 packages:

```
@kbn/securitysolution-io-ts-utils
@kbn/securitysolution-io-ts-types
@kbn/securitysolution-io-ts-alerting-types
@kbn/securitysolution-io-ts-list-types   
```

Deps between these packages are:

```
@kbn/securitysolution-io-ts-utils (none)
@kbn/securitysolution-io-ts-types -> @kbn/securitysolution-io-ts-utils
@kbn/securitysolution-io-ts-alerting-types -> @kbn/securitysolution-io-ts-types, @kbn/securitysolution-io-ts-utils
@kbn/securitysolution-io-ts-list-types  -> @kbn/securitysolution-io-ts-types, @kbn/securitysolution-io-ts-utils
```

Short description and function of each (Also in each of their README.md):

```
@kbn/securitysolution-io-ts-utils, Smallest amount of utilities such as format, validate, etc...
@kbn/securitysolution-io-ts-types, Base types such as to_number, to_string, etc...
@kbn/securitysolution-io-ts-alerting-types, Alerting specific types such as severity, from, to, etc...
@kbn/securitysolution-io-ts-list-types, list specific types such as exception lists, exception list types, etc...
```

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-13 15:36:06 -06:00
Frank Hassanabad
c092fbfe1a
[Security Solutions] (Phase 3, part 1) Removes dependency on security_solution plugin from lists (#99431)
## Summary

Removes the dependency of security_solution from the lists plugin

* Removes some of the deprecated types in favor of the new kbn package
* Adds a workaround in the kbn packages of removing the ?? and `a?.b?.c` typescript since kbn packages cannot transpile it
* Exposes the test_utils from the kbn package 

### Checklist
- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-10 10:16:19 -06:00
Frank Hassanabad
92da1059bd
Added deprecation to all the io-ts types and copies of them (#99260)
## Summary

Adds deprecation to all the io-ts types and copies of them found in the code base.

Phase 1 (Completed): Copy all the utilities to the `packages/kbn-securitysolution-io-ts-utils`
Phase 2: Add all the deprecation messages about them to the code base
Phase 3+: Teams and others will eventually remove/replace them with the utils from `kbn-securitysolution-io-ts-utils`
2021-05-04 17:00:45 -06:00
Ashokaditya
c93e028e0c
[Security Solution][Endpoint] Allow wildcard in trusted app paths (#97623)
* show operator dropdown for path field

refs elastic/security-team/issues/543

* update translation to use consistent values

refs elastic/security-team/issues/543

* update schema to validate path values

refs elastic/security-team/issues/543

* add tests for field and operator values

refs elastic/security-team/issues/543

* review changes

refs elastic/security-team/issues/543

* update schema to enforce dropdown validation for PATH field

refs elastic/security-team/issues/543

* add tests for schema updates

refs 1deab39453
refs elastic/security-team/issues/543

* optimise dropdown list for re-renders

refs elastic/security-team/issues/543

* align input fields and keep alignments when resized

refs elastic/security-team/issues/543

* correctly enter operator data on trusted app CRUD

refs elastic/security-team/issues/543

* update tests

refs 2ac56ee839
refs elastic/security-team/issues/543

* remove redundant code

review changes

* better type assertion

review changes

* move operator options out of component

- these do not depend on component props and thus no need to have it within a useMemo callback.

- review changes

* derive keys from operator entry field

review changes

* update type

* use custom styles for aligning input fields

review changes

* add a custom type for trusted_apps operator

undo changes from list plugin and server/lib/detection_engine

refs 2ac56ee839
refs elastic/security-team/issues/543

* add wildcard entry type

refs elastic/security-team/issues/543
refs https://github.com/elastic/kibana/pull/97623#pullrequestreview-642618462

* use the new entry type

refs elastic/security-team/issues/543
refs https://github.com/elastic/kibana/pull/97623#pullrequestreview-642618462

* update tests

refs elastic/security-team/issues/543
refs https://github.com/elastic/kibana/pull/97623#pullrequestreview-642618462

* update name for wildcard type so that it can be used also for cased inputs

refs elastic/security-team/issues/543
refs f9cb7eddda

* update artifacts to support wildcard entries

refs elastic/security-team/issues/543

* add tests for list schemas

refs f9cb7eddda
refs elastic/security-team/issues/543

* add placeholders for path values

review changes
elastic/kibana/pull/97623#discussion_r620617999

* ignore type check for now

* add type assertion

refs 284352ec9a

* remove unnecessary test

refs 2ac56ee839

* fix types

refs f9cb7eddda
refs b3f5dc4553

* add a note to entries

review changes

refs dbd3532149

* remove redundant type assertions

review changes
refs bcf615ac98
refs b3f5dc4553

* move placeholder text logic to utils

review changes elastic/kibana/pull/97623#discussion_r621673881

refs 6f2d0d7810

* pass the style as prop

review changes

* update api doc

CI check suggestion

* make placeholderText a function expression

review suggestion

elastic/kibana/pull/97623/commits/2dc4fd390cf5ea0e4fa67b3f5fc2561cbb29555e

* use semantic names for functions

refs 330731ebfc

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-04-29 14:54:19 +02:00
David Sánchez
bc240f0af7
[Security Solution] User can select event from event list and create a filter (#96940)
* Initial version of event filtering form/dialog. Pending to add all redux services

* Uses redux store instead of props to get the form values

* Manage errors on redux

* Creates even filter list on service constructor

* Add os type selector depending on form parent by props. Also added create action

* Allows add exception to an event. This commit has to be reviewed and maybe it will change depending on next changes

* Fix imports because changes on ExceptionBuilder component and add needed type export

* Adds constants. Rename eventFilters to eventFilter. Add http wrapper as a hook to check if the list has been created or not

* Adds missing files on last commit.

* Relocate async resource state to be shared between different pages

* Use async resource state to manage async operations on components. Relocate initial entry status to an utils module instead of hook.

* Adds comments into redux store from component

* Fixes typechecks and wrong imports

* Fixes translations and adds subheader and description modal

* Relocates form description

* Removes unused import

* Sanitize entries before submit to remove entry.id

* Missed file on last commit

* Use specific fields for endpoint_event type builder

* Split error field for each kind of errors to prevent unexpected renders. Adds unit test for event filter form component

* Set event.kind == event by default

* Changes folder names. Add notifications when success. Remove default event.king

* Adds notifications when api error and fixed multiple notifications showed for same error

* Adds new test for event filter modal and changes component name to be consistent

* Adds unit tests for event filter notification

* Adds middleware unit tests. Also isolate common event for all tests

* Adds unit tests for event filter reducer

* Adds unit tests for event filter selector

* Fixes same key on different multilanguages. Fixes naming incoherence

* Adds feature flag for event filtering

* Fixes unit tests and weird behavior when changing items after name or comments on event filter form

* Removes unused import

* Fixes unit tests. Add imports from lists plugin. Add expects on tests. Change some names

* Renames everything from eventFilter to eventFilters (plural)

* Rename state variable

* Create hook for notifications instead of a component. Removes className from modal body.

* Updates available fields for enpoint events builder

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-04-22 10:08:33 +02:00
David Sánchez
21f38afd27
[SECURITY SOLUTION] Add new exception list type and feature flag for event filtering (#96037)
* New exception list type for event filtering
* New feature flag for event filtering
2021-04-07 15:01:54 -04:00
Paul Tavares
9a0c73e515
[Security Solution][Endpoint] Endpoint Event Filtering List, Test Data Generator and Loader (#96263)
* Added new const to List plugin for new Endpont Event Filter list
* Data Generator for event filters ++ script to load event filters (WIP)
* refactor `generate_data` to use `BaseDataGenerator` class
2021-04-07 12:36:28 -04:00
Davis Plumlee
95271bf798
[Security Solution][Exceptions] Fixes OS adding method for exception enrichment (#94343) 2021-03-10 15:09:30 -05:00
Marshall Main
3992ed13db
Move exceptions builder to lists plugin (#94002)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-03-09 16:39:23 -05:00
Yara Tercero
9d2a7b8ece
[Security Solution][Exceptions] - Fixes exceptions builder UI where invalid values can cause overwrites of other values (#90634)
### Summary

This PR is a follow-up to #89066 - which fixed the same issue occurring with indicator match lists UI. The lack of stable ids for exception item entries resulted in some funky business by where invalid values could overwrite other values when deleting entries in the builder.
2021-02-24 18:32:44 -08:00
Davis Plumlee
087449cbf6
[Security Solution][Detections] Adds more granular validation for nested fields (#92041) 2021-02-19 19:16:14 -05:00
Bohdan Tsymbala
e81b5c1e40
[Security Solution][Artifacts] implemented policy specific trusted apps support in the manifest manager (#90991)
* Implemented policy specific trusted apps support in the manifest manager.
2021-02-16 14:31:36 -05:00
Bohdan Tsymbala
e94a164b7e
Initial version of adding artifacts per policy to the manifest. (#89130)
* Initial version of adding artifacts per policy to the manifest.

* Minor renaming to convey the purpose of the variable.

* Added ability to override list item mock data.

* Changed function signature to be more reusable.

* Implementationg of support of artifacts per policy in the manifest data structure.

* Added saved objects migrations.

* Renamed the endpoint to reflect that it's artifacts endpoint.

* Fixed tests.

* Fixed the manifest data.

* Fixed linting errors (result of merge).

* Updated ES mappings for manifest in all test setups.

* Updated hash in the mappings.

* Fixed the typo that lead to failing test.

* Fixed the problem with manifest not being dispatched to policies if there are same artifact names but different content. Artifact name in the ManifestSchema is not unique id, hence added decoded_sha256 to the comparison. Added test case to cover this.

* Fixed the problem with the task flow when failure to dispatch to policies will result in commited manifest and no redispatch on next task run. Changed tests to reflect new flow (actually restored previous flow).

* Forgot to commit changes in mock.

* Made other tests more readable using same varialbe naming pattern.

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-02-10 16:47:56 +01:00
Brandon Kobel
4584a8b570
Elastic License 2.0 (#90099)
* Updating everything except the license headers themselves

* Applying ESLint rules

* Manually replacing the stragglers
2021-02-03 18:12:39 -08:00
Yara Tercero
3dfb1aba2a
[Security Solution][Detections] - Fix export on exceptions list view (#86135)
## Summary

This PR addresses a fix on the exceptions list table export functionality. A dedicated route for exception list export needed to be created. List is exported into an `.ndjson` format. 

Exception lists consist of two elements - the list itself, and its items. The export file should now contain both these elements, the list followed by its items.
2020-12-23 00:27:37 -05:00
Yara Tercero
be055b85b8
[Security Solution][Detections] - Add skeleton exceptions list tab to all rules page (#85465)
## Summary

This PR is the first of 2 to complete the addition of a table displaying all exception lists on the all rules page. This PR focuses on the following:

- all exception lists displayed
- 'number of rules assigned to' displayed
- names and links of rules assigned to displayed
- refresh action button working
- no trusted apps list show
- search by `name`, `created_by`, `list_id`
  - just searching a word will search by list name
  - to search by `created_by` type `created_by:ytercero`
  - to search by `list_id` type `list_id:some-list-id`

#### TO DO (follow up PR)
- [ ] add tests
- [ ] wire up export of exception list
- [ ] wire up deletion of exception list

<img width="1121" alt="Screen Shot 2020-12-09 at 2 10 59 PM" src="https://user-images.githubusercontent.com/10927944/101676548-50498e00-3a29-11eb-90cb-5f56fc8c0a1b.png">

### Checklist
- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials
- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers)

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2020-12-15 22:45:18 -07:00
Garrett Spong
4dccbcad33
[SecuritySolution][Detections] Resolves referential integrity issues when deleting value lists (#85925)
## Summary

Resolves https://github.com/elastic/kibana/issues/77324, https://github.com/elastic/kibana/issues/77325, resolves https://github.com/elastic/kibana/issues/77325, and resolves https://github.com/elastic/kibana/issues/81302


This PR addresses referential integrity issues when deleting value lists. Previously when deleting value lists, any references in Exception Lists/Items would be left behind. This PR introduces a new confirmation modal when deleting value lists that are referenced in either space aware (`simple`) or space `agnostic` exception lists.

Also includes:

* Fixed Lists plugin `quick_start.sh` as it was using endpoint exception list + value lists (unsupported)
* Adds `quick_start_value_list_references.sh` to create exception lists/items, value lists, and references to easily test
* Add support to `findExceptionList` for searching for both `simple` and `agnostic` list types
* Two new query params have been added to the `deleteListRoute`
  * `ignoreReferences` (default:false) when true, maintains pre-7.11 behavior of deleting value list without performing any additional checks. 
    * NOTE: As written, this becomes an API breaking change as existing existing calls to the same API will `409` conflict if references exist. cc @jmikell821 @DonNateR 
  * `deleteReferences` (default:false) to perform dry run and identify referenced exception lists/items

## Testing
To test, run `quick_start_value_list_references.sh` and it will create all the necessary resources/references to easily exercise the above functionality. The below diagram details the resources created and how the references are wired up.

> Creates three different exception lists and value lists, and associates as
> below to test referential integrity functionality.
>
> NOTE: Endpoint lists don't support value lists, and are not tested here
>
> EL: Exception list
> ELI Exception list Item
> VL: Value list
>
>      EL1        EL2 (Agnostic)   EL3
>       |          |                |
>      ELI1       ELI2             ELI3
>       |\        /|                |
>       | \      / |                |
>       |  \    /  |                |
>       |   \  /   |                |
>       |    \/    |                |
>       |    /\    |                |
>       |   /  \   |                |
>       |  /    \  |                |
>       | /      \ |                |
>       |/        \|                |
>      VL1        VL2              VL3        VL4
>      ips.txt  ip_range.txt       text.txt   hosts.txt
>

Corner cases to be aware of:

* An exception item may have multiple value list entries -- only referenced value list entries should be removed
  * There is no API for removing individual entries. If all entries are references the entire item is deleted. If only some entries are references, the item is updated via a `PUT` (no `PATCH` support for exception items)
* It's not possible via the UI to create a space agnostic list that has value list exception items (only agnostic endpoint exception lists can be created and they do not support value lists). Please use above script to exercise this behavior.


Additional notes:
* Once the Exception List table is introduced (https://github.com/elastic/kibana/pull/85465), we can add an enhancement for deeplinking to exception lists from the reference error modal.
* The `deleteListRoute` response has been updated to include the responses from the reference checks to provide maximum flexibility
* There is no bulk API for deleting exception list items, and so they are iterated over via the `deleteExceptionListItem` API.


##### Reference error modal
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/102199153-813e1e80-3e80-11eb-8a9b-af116ca13df9.gif" />
</p>




##### Overflow example
<p align="center">
  <img width="500" src="https://user-images.githubusercontent.com/2946766/102199032-5784f780-3e80-11eb-81c7-17283d002ce4.gif" />
</p>

### Checklist

Delete any items that are not applicable to this PR.

- [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
- [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [X] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))

### For maintainers

- [X] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
2020-12-15 19:50:31 -07:00
Frank Hassanabad
28738e6b4b
[Security Solution] Fixes CIDR, float, long, integer, array, and text based issues when using value lists in exceptions (#85191)
## Summary

Fixes different bugs/issues when using exceptions with value based lists for both the UI, the backend, and the large value based lists. See https://github.com/elastic/kibana/issues/79516, but this also fixes several other bugs found mentioned below.

For the front end UI:
* Adds the ability to specify value based lists that are IP Ranges when the source event is an IP. Before you could only match IP to IP and the IP Ranges lists could not be used. 
* Breaks down a few functions into smaller functions for unit test writing abilities.

You can now add ip ranges as list values for the UI when before it would not show up:
<img width="1035" alt="Screen Shot 2020-12-07 at 2 15 39 PM" src="https://user-images.githubusercontent.com/1151048/101406552-d6819b00-3896-11eb-9fb5-4c7c2ad93b2e.png">

For value based lists:
* Fixes text data type to use "and" between matching using `operator: 'and'` and changes it from a `terms query to a `match` query
* Adds new API for searching against types called `searchListItemByValues ` so that numeric, text, array based, and other non-stringable types can be sent and then the value based lists will push that to ElasticSearch. This shifts as many corner cases and string/numeric coercions to ElasticSearch rather than Kibana client side code.
* Adds ability to handle arrays within arrays through a `flatten` call.
* Utilizes the `named queries` from ElasticSearch for the new API so that clients can get which parts matched and then use that for their exception list logic rather than in-memory string to string checks. This fixes CIDR and ranges as well as works with arrays.

For the backend exception lists that used value based lists:
* Broke down the `filterEventsAgainstList` function into a folder called `filters` and the functions into other files for better unit based testing.
* Changed the calls from `getListItemByValues` to `searchListItemByValues` which can return exactly what it matched against and this will not break anyone using the existing REST API for `getListItemByValues` since that REST API and client side API stays the same.
* Cleaned up extra promises being used in a few spots that async/await automatically will create. 
* Removed the stringabilities and stringify in favor of just a simpler exact check using `JSON.stringify()`

For the tests:
* Adds unit tests to broken down functions
* Adds ip_array, keyword_array, text_array, FTR tests for the backend.
* Adds more CIDR and range based FTR tests for the backend.
* Unskips and fixes all the numeric tests and range tests that could not operate previously from bugs.

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures (run axe in browser: [FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/), [Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] This renders correctly on smaller devices using a responsive layout. (You can test this [in your browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
2020-12-10 18:07:47 -07:00
Yara Tercero
21ea4f7a6f
[Security Solution][Detection Engine] - Improve DE query build times for large lists (#85051)
## Summary

This PR addresses the following issues:
- https://github.com/elastic/kibana/issues/76979
- https://github.com/elastic/kibana/issues/82267
- removal of unused lucene exceptions logic
2020-12-09 13:18:37 -05:00
Bohdan Tsymbala
de5edaa278
Trusted Apps signer API. (#83661)
* Separated out service layer for trusted apps.

* Improved the type structure a bit to avoid using explicit string literals and to add possibility to return OS specific parts of trusted app object in type safe manner.

* Added support for mapping of trusted app to exception item and back.

* Changed schema to support signer in the API.

* Renamed utils to mapping.

* Exported some types in lists plugin and used them in trusted apps.

* Added tests for mapping.

* Added tests for service.

* Switched deletion to use exceptions for not found case.

* Added resetting of the mocks in service layer tests.

* Added handlers tests.

* Refactored mapping tests to be more granular based on the case.

* Restored lowercasing of hash.

* Added schema tests for signer field.

* Removed the grouped tests (they were split into tests for separate concerns).

* Corrected the tests.

* Lowercased the hashes in the service test.

* Moved the lowercasing to the right location.

* Fixed the tests.

* Added test for lowercasing hash value.

* Introduced OperatingSystem enum instead of current types.

* Removed os list constant in favour of separate lists in places that use it (each place has own needs to the ordering).

* Fixed the missed OperatingSystem enum usage.
2020-11-30 15:42:31 +01:00
Mikhail Shustov
95861a0fb0
[DX] Prettier v2.2 (#83899)
* update prettier with ts version support

* mute type-error

* run prettier on codebase

* fix examples

* fix errors after master merged
2020-11-23 13:17:05 +01:00