## Dearest Reviewers 👋
I've been working on this branch with @mistic and @tylersmalley and
we're really confident in these changes. Additionally, this changes code
in nearly every package in the repo so we don't plan to wait for reviews
to get in before merging this. If you'd like to have a concern
addressed, please feel free to leave a review, but assuming that nobody
raises a blocker in the next 24 hours we plan to merge this EOD pacific
tomorrow, 12/22.
We'll be paying close attention to any issues this causes after merging
and work on getting those fixed ASAP. 🚀
---
The operations team is not confident that we'll have the time to achieve
what we originally set out to accomplish by moving to Bazel with the
time and resources we have available. We have also bought ourselves some
headroom with improvements to babel-register, optimizer caching, and
typescript project structure.
In order to make sure we deliver packages as quickly as possible (many
teams really want them), with a usable and familiar developer
experience, this PR removes Bazel for building packages in favor of
using the same JIT transpilation we use for plugins.
Additionally, packages now use `kbn_references` (again, just copying the
dx from plugins to packages).
Because of the complex relationships between packages/plugins and in
order to prepare ourselves for automatic dependency detection tools we
plan to use in the future, this PR also introduces a "TS Project Linter"
which will validate that every tsconfig.json file meets a few
requirements:
1. the chain of base config files extended by each config includes
`tsconfig.base.json` and not `tsconfig.json`
1. the `include` config is used, and not `files`
2. the `exclude` config includes `target/**/*`
3. the `outDir` compiler option is specified as `target/types`
1. none of these compiler options are specified: `declaration`,
`declarationMap`, `emitDeclarationOnly`, `skipLibCheck`, `target`,
`paths`
4. all references to other packages/plugins use their pkg id, ie:
```js
// valid
{
"kbn_references": ["@kbn/core"]
}
// not valid
{
"kbn_references": [{ "path": "../../../src/core/tsconfig.json" }]
}
```
5. only packages/plugins which are imported somewhere in the ts code are
listed in `kbn_references`
This linter is not only validating all of the tsconfig.json files, but
it also will fix these config files to deal with just about any
violation that can be produced. Just run `node scripts/ts_project_linter
--fix` locally to apply these fixes, or let CI take care of
automatically fixing things and pushing the changes to your PR.
> **Example:** [`64e93e5`
(#146212)](64e93e5806)
When I merged main into my PR it included a change which removed the
`@kbn/core-injected-metadata-browser` package. After resolving the
conflicts I missed a few tsconfig files which included references to the
now removed package. The TS Project Linter identified that these
references were removed from the code and pushed a change to the PR to
remove them from the tsconfig.json files.
## No bazel? Does that mean no packages??
Nope! We're still doing packages but we're pretty sure now that we won't
be using Bazel to accomplish the 'distributed caching' and 'change-based
tasks' portions of the packages project.
This PR actually makes packages much easier to work with and will be
followed up with the bundling benefits described by the original
packages RFC. Then we'll work on documentation and advocacy for using
packages for any and all new code.
We're pretty confident that implementing distributed caching and
change-based tasks will be necessary in the future, but because of
recent improvements in the repo we think we can live without them for
**at least** a year.
## Wait, there are still BUILD.bazel files in the repo
Yes, there are still three webpack bundles which are built by Bazel: the
`@kbn/ui-shared-deps-npm` DLL, `@kbn/ui-shared-deps-src` externals, and
the `@kbn/monaco` workers. These three webpack bundles are still created
during bootstrap and remotely cached using bazel. The next phase of this
project is to figure out how to get the package bundling features
described in the RFC with the current optimizer, and we expect these
bundles to go away then. Until then any package that is used in those
three bundles still needs to have a BUILD.bazel file so that they can be
referenced by the remaining webpack builds.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
* [ts] set allowJs to true by default
* fix scripts/check_ts_projects, original implementation is now wildly inefficient
* produce stats in check_ts_projects to make sure it's actually working
* fix imports
* [build_ts_refs] improve caches, allow building a subset of projects
* cleanup project def script and update refs in type check script
* rename browser_bazel config to avoid kebab-case
* remove execInProjects() helper
* list references for tsconfig.types.json for api-extractor workload
* disable composite features of tsconfig.types.json for api-extractor
* set declaration: true to avoid weird debug error
* fix jest tests
Co-authored-by: spalger <spalger@users.noreply.github.com>
* [Console] Removed x-pack console extensions folder and moved json files to src/plugins/console
* [Console] Generated console definitions with ES repo on master branch
* Added some overrides and doc links fixes
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* wip
* First pass at standalone and embedded redux stores and usage
* wip
* First pass at standalone and embedded redux stores and usage
* wip
* clean up
* wip
* refact(NA): remove extra pkg_npm target and add specific target folders on @kbn/i18n
* cleanup
* - fixes type errors in tests
* WIP remove use_manage_timeline
* wip add query + selector
* finishing integrating timeline manage context from redux
* integrating t-grid in security solution
* fix RowRender type
* WIP begin to move components from package to plugin
* integration of t-grid inside of security solution
* wip to make redux work
* little trick to make it render
* - fixes a few type errors
* better integration betwen tgrid and security solutions
* bringing back tsconfig on timeline
* wip integration t-grid in observability
* fix types
* fix type in security solutions
* add type to import + trie dto get the bundle size as small as possible
* fix type in integration test
* fix type in integration test
* - fix tests
* clean up to use technical fields
* - fixes unit tests
* - mocks the `useDateFormat` function of the `useKibana` service to fix unit tests
* fix t-grid settings vs create timeline + fix inspect button
* fix last suites test
* Update unit tests, snapshots and lint
* Fix bad merge
* fix plugin export
* Fix some failing tests
* fix unit tets in timelines plugins
* fix latest test
* fix i18n
* free obs from t-grid
* Fix timeline functional plugin types
* fix store provider
* Update failing defaultHeader test
* Fix i18n usage in security solution
* Fix remaining i18n errors in timelines plugin
* Dedupe common shared types
* move drag and drop utils in package to avoid duplication
* More shared type cleanup
* add feature flag
* review I
* fix merge with master
* fix i18n translation
* More type deduping
* Use @kbn/common-utils, fix remaining types
* fix types
* fix tests
* missing type
* fix cypress tests
Co-authored-by: Kevin Qualters <kevin.qualters@elastic.co>
Co-authored-by: Tiago Costa <tiagoffcc@hotmail.com>
Co-authored-by: Andrew Goldstein <andrew.goldstein@elastic.co>
* es lint fix
* Add formatter on the buckets labels
* Config the new plugin, toggle tooltip
* Aff filtering on slice click
* minor fixes
* fix eslint error
* use legacy palette for now
* Add color picker to legend colors
* Fix ts error
* Add legend actions
* Fix bug on Color Picker and remove local state as it is unecessary
* Fix some bugs on colorPicker
* Add setting for the user to select between the legacy palette or the eui ones
* small enhancements, treat empty labels with (empty)
* Fix color picker bugs with multiple layers
* fixes on internationalization
* Create migration script for pie chart and legacy palette
* Add unit tests (wip) and a small refactoring
* Add unit tests and move some things to utils, useMemo and useCallback where it should
* Add jest config file
* Fix jest test
* fix api integration failure
* Fix to_ast_esaggs for new pie plugin
* Close legendColorPicker popover when user clicks outside
* Fix warning
* Remove getter/setters and refactor
* Remove kibanaUtils from pie plugin as it is not needed
* Add new values to the migration script
* Fix bug on not changing color for expty string
* remove from migration script as they don't need it
* Fix editor settings for old and new implementation
* fix uistate type
* Disable split chart for the new plugin for now
* Remove temp folder
* Move translations to the pie plugin
* Fix CI failures
* Add unit test for the editor config
* Types cleanup
* Fix types vol2
* Minor improvements
* Display data on the inspector
* Cleanup translations
* Add telemetry for new editor pie options
* Fix missing translation
* Use Eui component to detect click outside the color picker popover
* Retrieve color picker from editor and syncColors on dashboard
* Lazy load palette service
* Add the new plugin to ts references, fix tests, refactor
* Fix ci failure
* Move charts library switch to vislib plugin
* Remove cyclic dependencies
* Modify license headers
* Move charts library switch to visualizations plugin
* Fix i18n on the switch moved to visualizations plugin
* Update license
* Fix tests
* Fix bugs created by new charts version
* Fix the i18n switch problem
* Update the migration script
* Identify if colorIsOverwritten or not
* Small multiples, missing the click event
* Fixes the UX for small multiples part1
* Distinct colors per slice implementation
* Fix ts references problem
* Fix some small multiples bugs
* Add unit tests
* Fix ts ref problem
* Fix TS problems caused by es-charts new version
* Update the sample pie visualizations with the new eui palette
* Allows filtering by the small multiples value
* Apply sortPredicate on partition layers
* Fix vilib test
* Enable functional tests for new plugin
* Fix some functional tests
* Minor fix
* Fix functional tests
* Fix dashboard tests
* Fix all dashboard tests
* Apply some improvements
* Explicit params instead of visConfig Json
* Fix i18n failure
* Add top level setting
* Minor fix
* Fix jest tests
* Address PR comments
* Fix i18n error
* fix functional test
* Add an icon tip on the distinct colors per slice switch
* Fix some of the PR comments
* Address more PR comments
* Small fix
* Functional test
* address some PR comments
* Add padding to the pie container
* Add a max width to the container
* Improve dashboard functional test
* Move the labels expression function to the pie plugin
* Fix i18n
* Fix functional test
* Apply PR comments
* Do not forget to also add the migration to them embeddable too :D
* Fix distinct colors for IP range layer
* Remove console errors
* Fix small mulitples colors with multiple layers
* Fix lint problem
* Fix problems created from merging with master
* Address PR comments
* Change the config in order the pie chart to not appear so huge on the editor
* Address PR comments
* Change the max percentage digits to 4
* Change the max size to 1000
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* cleanup removed dirs
* delete removed folders from other places in the repo
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Phase 1 of a multi-phase cautious approach for adding an experimental application cache for Kibana solutions called `metric_entities` and integrates it within Security Solutions.
Phase 1 is putting experimental support into the application without breaking existing features. Lots of TODO's, conversations and a possible RFC from phase 1 to phase 2 approach. Some features are missing, but for phase 1 the general idea and code is all there.
To enable this first phase after checking out the branch add this to your `kibana.dev.yml`
```yml
xpack.metricsEntities.enabled: true
xpack.securitySolution.enableExperimental: ['metricsEntitiesEnabled']
```
Then go into Stack Management -> Advanced Settings (Under Security Solutions) and set the enabled to true like so:
<img width="1229" alt="Screen Shot 2021-04-08 at 2 21 02 PM" src="https://user-images.githubusercontent.com/1151048/114091276-b3cbb700-9875-11eb-9083-5c1d91dd20ed.png">
Next go to the security_solutions page and you will see it being activated and you will have these transforms running if you look under stack management:
<img width="1710" alt="Screen Shot 2021-04-29 at 2 00 27 PM" src="https://user-images.githubusercontent.com/1151048/116611174-4a2e4e00-a8f3-11eb-9e15-55cb504dfb2a.png">
On the hosts page, network, page, etc... You can see them being activated when you have no query/filter and you click on request:
<img width="1405" alt="Screen Shot 2021-04-29 at 2 01 28 PM" src="https://user-images.githubusercontent.com/1151048/116611274-6a5e0d00-a8f3-11eb-9998-9f5b3d1c5c63.png">
You will see in the request the index patterns all starting with `estc_xyz*`
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)
We have lots of TODO's but no concrete docs with this just yet.
- [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials
Behind a feature flag and this isn't there yet.
- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
* add base config for all the TS projects
* all the project use new tsconfig.project.json
* compile test files in the high-level tsconfig.json
* fix TS error in maps plugin
* fix TS error in infra plugin
* exclude mote test and test until folders
* uptime. do not import test code within prod code
* expressions. do not import test code within prod code
* data: export mocks from high level folder
* task_manager: comply with es client typings
* infra: remove unused enzyme_helpers
* check_ts_project requires "include" key
* ts_check should handle parent configs
* all ts configs should extend base one
* exclude test folders from plugins
* update patterns to fix ts_check errors
* Apply suggestions from code review
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
* uptime: MountWithReduxProvider to test helpers
Co-authored-by: Constance <constancecchen@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
* merge all the typings at root level
* merge x-pack/tsconfig into tsconfig.json
* fix tsconfig after changes in master
* remove unnecessary typings
* update paths to the global typings
* update paths to the global elaticsearch typings
* fix import
* fix path to typings/elasticsearch in fleet plugin
* remove file deleted from master
* fix lint errors
* exclude all the plugins from src/plugins
* move all the used fixtures to discover
* remove src/fixtures alias
* remove unused fixtures
* cleanup x-pack/tsconfig.json
* dont compile apm/scripts
* fix tests
* dont include infra in xpack/tsconfig.json
* update list of includes
* Cleanup OSS code from visualizations wizard
* Remove unecessary translations
* Remove unused translation
* Fix functional test
* Disable the functional test for OSS
* Remove from oss correctly :D
* Fix ci
* remove unused "paths" from tsconfig
* remove unused test utils, import get-url from kbn/test
* dont pollute globals with jest when importing getUrl
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
