Commit graph

30 commits

Author SHA1 Message Date
David Sánchez
6a0fa94cbb
[Security Solution][Endpoint] Adapt exception list api calls to versioned router (#165658)
## Summary

- Adds version to http calls for endpoint exceptions at Lists API.
- Fixes unit test.

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2023-09-07 17:35:15 +02:00
Marshall Main
21879be883
[Security Solution] Migrate lists plugin API to versioned router (#165160)
Closes https://github.com/elastic/security-team/issues/7176
2023-08-31 07:48:01 -07:00
Vitalii Dmyterko
505d8265c8
[Security Solution][Detection Engine] move lists to data stream (#162508)
## Summary

- addresses https://github.com/elastic/security-team/issues/7198
- moves list/items indices to data stream
  - adds `@timestamp` mapping to indices mappings
- migrate to data stream if indices already exist(for customers < 8.11)
or create data stream(for customers 8.11+ or serverless)
- adds
[DLM](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/data-streams-put-lifecycle.html)
to index templates
- replaces update/delete queries with update_by_query/delete_by_query
which supported in data streams
  - fixes existing issues with update/patch APIs for lists/items
    - update/patch for lists didn't save `version` parameter in ES
- update and patch APIs for lists/items were identical, i.e. for both
routes was called the same `update` method w/o any changes

<details>

<summary>Technical detail on moving API to
(update/delete)_by_query</summary>


`update_by_query`, `delete_by_query` do not support refresh=wait_for,
[only false/true
values](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/docs-update-by-query.html#_refreshing_shards_2).
Which might break some of the use cases on UI(when list is removed, we
refetch all lists. Deleted list will be returned for some time. [Default
refresh time is
1s](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/docs-refresh.html)).
So, we retry refetching deleted/updated document before finishing
request, to return reindexed document

`update_by_query` does not support OCC [as update
API](https://www.elastic.co/guide/en/elasticsearch/reference/8.9/optimistic-concurrency-control.html).
Which is supported in both
[list](https://www.elastic.co/guide/en/security/current/lists-api-update-container.html)/[list
item
](https://www.elastic.co/guide/en/security/current/lists-api-update-item.html)updates
through _version parameter.
_version is base64 encoded "_seq_no", "_primary_term" props used for OCC

So, to keep it without breaking changes: implemented check for version
conflict within update method
</details>

### Checklist

Delete any items that are not applicable to this PR.

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2023-08-23 19:42:57 +01:00
Georgii Gorbachev
5776e97d9c
[Security Solution] Clean up CODEOWNERS and other files for the Detection Engine team (#159729)
## Summary

This PR replaces any mentions of
https://github.com/orgs/elastic/teams/security-solution-platform team
with https://github.com/orgs/elastic/teams/security-detection-engine
according to the recent restructure.
2023-06-14 10:39:50 -07:00
Yara Tercero
11155329cc
[Security Solution][Exceptions] - Add exception list duplication options with and without expired items (#154991)
## Summary

Adds the following:

- Add the option to duplicate from the shared exception list management
actions dropdowns
  - User can select to include exception items with expired TTL
  - User can select to not include exception items with expired TTL 
  - Cypress tests added for both options
2023-04-21 16:01:43 -07:00
Davis Plumlee
92a1689e95
[Security Solution][Exceptions] Rule exceptions TTL - Expiration (#145180) 2023-02-07 16:20:39 -05:00
Spencer
afb09ccf8a
Transpile packages on demand, validate all TS projects (#146212)
## Dearest Reviewers 👋 

I've been working on this branch with @mistic and @tylersmalley and
we're really confident in these changes. Additionally, this changes code
in nearly every package in the repo so we don't plan to wait for reviews
to get in before merging this. If you'd like to have a concern
addressed, please feel free to leave a review, but assuming that nobody
raises a blocker in the next 24 hours we plan to merge this EOD pacific
tomorrow, 12/22.

We'll be paying close attention to any issues this causes after merging
and work on getting those fixed ASAP. 🚀

---

The operations team is not confident that we'll have the time to achieve
what we originally set out to accomplish by moving to Bazel with the
time and resources we have available. We have also bought ourselves some
headroom with improvements to babel-register, optimizer caching, and
typescript project structure.

In order to make sure we deliver packages as quickly as possible (many
teams really want them), with a usable and familiar developer
experience, this PR removes Bazel for building packages in favor of
using the same JIT transpilation we use for plugins.

Additionally, packages now use `kbn_references` (again, just copying the
dx from plugins to packages).

Because of the complex relationships between packages/plugins and in
order to prepare ourselves for automatic dependency detection tools we
plan to use in the future, this PR also introduces a "TS Project Linter"
which will validate that every tsconfig.json file meets a few
requirements:

1. the chain of base config files extended by each config includes
`tsconfig.base.json` and not `tsconfig.json`
1. the `include` config is used, and not `files`
2. the `exclude` config includes `target/**/*`
3. the `outDir` compiler option is specified as `target/types`
1. none of these compiler options are specified: `declaration`,
`declarationMap`, `emitDeclarationOnly`, `skipLibCheck`, `target`,
`paths`

4. all references to other packages/plugins use their pkg id, ie:
	
	```js
    // valid
    {
      "kbn_references": ["@kbn/core"]
    }
    // not valid
    {
      "kbn_references": [{ "path": "../../../src/core/tsconfig.json" }]
    }
    ```

5. only packages/plugins which are imported somewhere in the ts code are
listed in `kbn_references`

This linter is not only validating all of the tsconfig.json files, but
it also will fix these config files to deal with just about any
violation that can be produced. Just run `node scripts/ts_project_linter
--fix` locally to apply these fixes, or let CI take care of
automatically fixing things and pushing the changes to your PR.

> **Example:** [`64e93e5`
(#146212)](64e93e5806)
When I merged main into my PR it included a change which removed the
`@kbn/core-injected-metadata-browser` package. After resolving the
conflicts I missed a few tsconfig files which included references to the
now removed package. The TS Project Linter identified that these
references were removed from the code and pushed a change to the PR to
remove them from the tsconfig.json files.

## No bazel? Does that mean no packages??
Nope! We're still doing packages but we're pretty sure now that we won't
be using Bazel to accomplish the 'distributed caching' and 'change-based
tasks' portions of the packages project.

This PR actually makes packages much easier to work with and will be
followed up with the bundling benefits described by the original
packages RFC. Then we'll work on documentation and advocacy for using
packages for any and all new code.

We're pretty confident that implementing distributed caching and
change-based tasks will be necessary in the future, but because of
recent improvements in the repo we think we can live without them for
**at least** a year.

## Wait, there are still BUILD.bazel files in the repo
Yes, there are still three webpack bundles which are built by Bazel: the
`@kbn/ui-shared-deps-npm` DLL, `@kbn/ui-shared-deps-src` externals, and
the `@kbn/monaco` workers. These three webpack bundles are still created
during bootstrap and remotely cached using bazel. The next phase of this
project is to figure out how to get the package bundling features
described in the RFC with the current optimizer, and we expect these
bundles to go away then. Until then any package that is used in those
three bundles still needs to have a BUILD.bazel file so that they can be
referenced by the remaining webpack builds.

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-12-22 19:00:29 -06:00
Khristinin Nikita
35e02bcf64
Add sorting for exceptions cards (#145070)
## Add sorting for shared exceptions

Currently is possible to sort by Name, Created At, Created By


https://user-images.githubusercontent.com/7609147/201640150-dc9d53e4-0d34-4da1-8522-9899d35e7359.mov

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Devin W. Hurley <snowmiser111@gmail.com>
2022-11-15 13:55:38 -07:00
spalger
52f2b33a07
[auto] migrate existing plugin/package configs 2022-10-28 14:06:46 -05:00
spalger
42879f7656
[bazel] fix some BUILD.bazel file inconsistencies 2022-10-26 11:07:55 -05:00
Davis Plumlee
51699fa21a
[Security Solution] Value list exceptions (#133254) 2022-09-19 22:41:28 +02:00
Yara Tercero
194e0d7144
[Security Solution][Exceptions] - Initial updates to exceptions viewer UX (#138770)
## Summary

**API changes**
- Adds API for determining the list-rule references. 
- Updates the exception items find api to include the `search` param which allows for simple search queries - used with the EUI search bar

**UI updates**
- Moved the exception components into new `rule_exceptions` folder per suggested folder structure updates listed [here](https://github.com/elastic/kibana/issues/138600)
- Updates the rule details tabs to split endpoint and rule exceptions into their own tabs
- Updates the viewer utilities header now that these different exception types are split
- Updates exception item UI to match new designs
- Updates the UI for when there are no items
- Removes `use_exception_list_items` hook as it is no longer in use
- Flyouts (add/edit) remain untouched
2022-09-08 16:41:52 -04:00
Spencer
32491462a9
add kibana.jsonc files to existing packages (#138965)
* [packages] add kibana.jsonc files

* auto-migrate to kibana.jsonc

* support interactive pkg id selection too

* remove old codeowners entry

* skip codeowners generation when .github/CODEOWNERS doesn't exist

* fall back to format validation if user is offline

* update question style

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2022-09-08 13:31:57 -07:00
Tiago Costa
1cbf83f73c
chore(NA): remove src folder requirement from packages (part 2) (#138476)
* refact(NA): apply root_input_dir=src to each already created pkg

* refact(NA): update package generator

* fix(NA): correctly use rootDir

* fix(NA): use root input dir on latest introduced pkgs for jsts_transpiler macro

* chore(NA): merge with main

* chore(NA): first attempt to complete removal of src folder on a small group of pkgs

* Revert "chore(NA): first attempt to complete removal of src folder on a small group of pkgs"

This reverts commit b6f34b7530.

* chore(NA): remove src folder requirement from xpack pkgs

* chore(NA): remove src folder from analytics pkgs

* chore(NA): remove src folder from home pkgs

* chore(NA): remove src folder from shared_ux pkgs

* fix(NA): remove missing src folder inputs on sharedux pkg

* chore(NA): remove src folder from kbn-a* pkgs

* chore(NA): remove src folder from kbn-b* pkgs

* chore(NA): remove src folder from kbn-c* pkgs

* chore(NA): correct exclude pattern for each changed pkg

* chore(NA): remove src folder from kbn-y* pkgs

* chore(NA): remove src folder from kbn-e* pkgs

* chore(NA): remove src folder from kbn-f* and kbn-g* pkgs

* chore(NA): remove src folder from kbn-f* and kbn-g* pkgs

* chore(NA): remove src folder from kbn-h** pkgs

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* Revert "chore(NA): remove src folder from kbn-h** pkgs"

This reverts commit dcdf72bcc1.

* fix(NA): grammar location

* test(NA): fix tests for kbn/config-schema

* test(NA): fix tests for kbn/config-schema

* chore(NA): multiple errors fixed

* chore(NA): remove kuery grammar fix

* fix(NA): @kbn/ace imports

* fix(NA): grammar location

* fix(NA): add missing files to tsconfigs

* [CI] Auto-commit changed files from 'node scripts/build_plugin_list_docs'

* chore(NA): complete tsconfigs

* Revert "chore(NA): complete tsconfigs"

This reverts commit f48c616864.

* chore(NA): remove src folder from kbn-core* pkgs

* chore(NA): remove src folder from kbn-u* pkgs

* chore(NA): remove src folder from kbn-ui-shared-deps* pkgs

* chore(NA): fix problems on core pkgs

* chore(NA): fix problems on core pkgs

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* fix(NA): shared_built_assets correct location'

* test(NA): update @kbn/optimizer integration snapshots

* chore(NA): remove src folder from kbn-t* p1 pkgs

* chore(NA): remove src folder from kbn-t* p2 pkgs

* chore(NA): update rootDir on utility types pkg

* chore(NA): include missing files on @kbn/test

* chore(NA): include missing files on @kbn/test

* fix(NA): new jest preset paths on @kbn/test

* chore(NA): fix eslint

* fix(NA): new jest preset paths on @kbn/test

* chore(NA): remove exclusion for mocks folde on @kbn/test

* fix(NA): several imports from target_node

* fix(NA): @kbn/test mocha and jest types clash

* chore(NA): remove src folder from kbn-storybook pkg

* chore(NA): remove src folder from kbn-stdio-dev-helpers pkg

* chore(NA): remove src folder from kbn-std pkg

* chore(NA): remove src folder from kbn-sort-pkg-json pkg

* chore(NA): remove src folder from kbn-some-dev-log and kbn-shared-ux-utility pkgs

* chore(NA): remove src folder from kbn-ux-storybook pkg

* chore(NA): remove src folder from kbn-shared-ux-services pkg

* chore(NA): remove src folder from kbn-shared-ux-components pkg

* chore(NA): remove src folder from kbn-shared-svg pkg

* chore(NA): remove src folder from kbn-server-http-tools pkg

* chore(NA): remove src folder from kbn-securitysolution-* pkgs

* chore(NA): remove src folder from kbn-r-* pkgs

* chore(NA): remove src folder from kbn-p* pkgs

* chore(NA): remove src folder from kbn-o* pkgs

* chore(NA): remove src folder from kbn-m* pkgs

* chore(NA): remove src folder from kbn-j,k,l* pkgs

* chore(NA): remove src folder from kbn-j,k,l* pkgs

* chore(NA): remove src folder from kbn-io-ts-utils* pkgs

* chore(NA): remove src folder from kbn-* pkgs except a few

* chore(NA): update @kbn/generate

* fix(NA): wrong exclusion on kbn-storybook

* chore(NA): remove src folder from kbn-monaco pkg

* chore(NA): remove src folder from kbn-interpreter pkg

* fix(NA): wrong exclusion on kbn-storybook

* chore(NA): update every require for target_*/src

* chore(NA): remover src folder from @kbn/handlebars

* fix(NA): license for @kbn/handlebars

* chore(NA): copy templates as part of the jsts_transpiler macro for @kbn/storybook

* chore(NA): update handlebars

* fix(NA): @kbn/plugin-generator import paths

* fix(NA): bundle sizes

* fix(NA): web bundle for @kbn/i18n-react

* Revert "fix(NA): bundle sizes"

This reverts commit 8aefe84fbc.

* Revert "Revert "fix(NA): bundle sizes""

This reverts commit e9d87d72a4.

* fix(NA): @kbn/docs-utils index.ts path expectation

* chore(NA): merge and solve conflicts with main

* fix(NA): relative import to index

* chore(NA): merge and solve conflicts with main

* [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix'

* chore(NA): apply eslint fix

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-08-30 15:57:35 +01:00
Wafaa Nasr
f107c273af
Fix/100018 import value list sorting (#138381)
* add sortField and sortOrder to findLists params

* move tests for list-hooks

* pass sortField and sortOrder from Flyout component + tests + remove obsolete test files

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-08-10 13:07:32 +02:00
Tiago Costa
df8b62aa98
chore(NA): allow packages to not have an obligation for a src folder on packages (#138038)
* refact(NA): apply root_input_dir=src to each already created pkg

* refact(NA): update package generator

* fix(NA): correctly use rootDir

* fix(NA): use root input dir on latest introduced pkgs for jsts_transpiler macro

* chore(NA): merge with main

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-08-08 15:54:05 +01:00
Spencer
a3b2757e4e
[type-summarizer] reimplement for broader support (#135163)
* [type-summarizer] reimplement for broader support

* Enable sourceMaps in all packages

* include naming collision in summarizePackage test

* fix readmes

* remove unnecessary transient dependency

* remove code that was commented out

* remove outdated todo comment

* ensure errors triggered by untyped-exports are ligible

* remove unused import

* break out snippet generation from AstIndexer

* refactor several massive files into smaller pieces and add more inline docs

* fix typos

* update jest snapshots

* add sections to readme that points people to the useful parts of the source code along with a high-level overview of how the type-summarizer works

* remove --dump flag, it doesn't work

* use decName instead of calling names.get a second time

* include `export` as invalid name
2022-07-06 13:48:45 -05:00
David Sánchez
a07c47c652
Set filter to undefined when it's empty on findExceptionLists method (#128455)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2022-03-29 12:44:31 +02:00
Spencer
614139b8e5
[bazel] avoid a little boilerplate in packages (#126309)
* [bazel] avoid a little boilerplate for @types packages

* [bazel/ts] stop building sourcemaps since they're ignored
2022-02-24 12:43:12 -06:00
Tiago Costa
0a75d426b9
chore(NA): splits types from code on @kbn/securitysolution-io-ts-utils (#121677)
* chore(NA): splits types from code on @kbn/securitysolution-io-ts-utils

* chore(NA): fix package.json

* chore(NA): merge and solve conflicts with main

* chore(NA): remove any from types

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-12-21 17:24:58 -07:00
Tiago Costa
df8e6199fe
chore(NA): splits types from code on @kbn/securitysolution-list-constants (#121683) 2021-12-21 04:00:46 +00:00
Tiago Costa
d7a7f6b3fb
chore(NA): splits types from code on @kbn/securitysolution-list-api (#121680)
* chore(NA): splits types from code on @kbn/securitysolution-list-api

* chore(NA): export type only

* chore(NA): merge and solve conflicts wth main
2021-12-21 03:46:00 +00:00
Tiago Costa
e203be1613
chore(NA): splits types from code on @kbn/securitysolution-io-ts-list-types (#121672) 2021-12-21 02:54:49 +00:00
Yara Tercero
69a6cf329c
Fixing exceptions export format (#114920)
### Summary

Fixing exceptions export format and adding integration tests for it.
2021-10-13 23:32:43 -04:00
Tiago Costa
7c4e4f507b
chore(NA): replace babel config files by inline presets by default in the jsts_transpiler rule (#110620)
* chore(NA): replace babel config files by inline presets by default in the js_ts_transpiler rule

* chore(NA): update @kbn/ace build to exclude worker file

* chore(NA): remove config file support

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-09-03 21:17:46 +01:00
Tiago Costa
127402415c
chore(NA): moving @kbn/securitysolution-list-api to babel transpiler (#110265)
* chore(NA): moving @kbn/securitysolution-list-api to babel transpiler

* chore(NA): add web targets
2021-08-27 15:56:18 +01:00
Spencer
c0395c9ef6
[build_ts_refs] improve caches, allow building a subset of projects (#107981)
* [build_ts_refs] improve caches, allow building a subset of projects

* cleanup project def script and update refs in type check script

* rename browser_bazel config to avoid kebab-case

* remove execInProjects() helper

* list references for tsconfig.types.json for api-extractor workload

* disable composite features of tsconfig.types.json for api-extractor

* set declaration: true to avoid weird debug error

* fix jest tests

Co-authored-by: spalger <spalger@users.noreply.github.com>
2021-08-10 22:12:45 -07:00
Frank Hassanabad
aa2f5b535d
[Security Solution] Utilizes constants package and deletes duplicate code (#100513)
## Summary

Utilizes constants package and deletes duplicate code

* Renames the `securitysolution-constants` to be `securitysolution-list-constants` to be specific
* Deletes duplicated code found during cleanup
* Moves more tests into the packages found along the way with the duplicated code
* Moves `parseScheduleDates` from `@kbn/securitysolution-io-ts-types` to `@kbn/securitysolution-io-ts-utils`

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-24 18:38:14 -06:00
Tiago Costa
50616aa0f9
chore(NA): correctly setup dep order on js_library to fix windows build (#100420) 2021-05-21 00:03:54 +01:00
Frank Hassanabad
192c8ba743
Adds two more packages and moves files into the packages (#100375)
## Summary

* Adds package `kbn-securitysolution-list-api`
* Adds package `kbn-securitysolution-list-hooks`
* Moves files into the packages
* Moves a few additional types into the other packages such as the `kbn-securitysolution-io-ts-types` package to remove more things from the shard_export/shared_import between lists and security solution
* Removes more duplicated code

### Checklist

- [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
2021-05-20 12:36:07 -06:00