We're working on converting all plugins into packages, which turns
almost all code in the repository into a package except for things like
the `test` and `x-pack/test` directories. The problem we ran into is
that the alerting api-integration test plugins are being migrated to
packages but they are currently consuming a handful of helpers from the
parent `../lib` directory. This doesn't work, as packages can only
import other packages and the `../lib` directory isn't in a package,
it's just free-floating `x-pack/test` code. To fix this I've moved the
necessary components out of `common/lib` and into `packages/helpers`
which is importable via `@kbn/alerting-api-integration-helpers` and
updated all the uses to access this new package.
The location of plugins was previously somewhat irrelevant, but as we
move into packages it's more important that we can find all plugins in
the repository, and we would like to be able to do that without needing
to maintain a manifest somewhere to accomplish this. In order to make
this possible we plan to find any plugin/package by spotting all
kibana.json files which are not "fixtures". This allows plugin-like code
(but not actual plugin code) to exist for testing purposes, but it must
be within some form of "fixtures" directory, and any plugin that isn't
in a fixtures directory will be automatically pulled into the system
(though test plugins, examples, etc. will still only be loaded when the
plugin's path is passed via `--plugin-path`, the system will know about
them and use that knowledge for other things).
Since this is just a rename Operations will review and merge by EOD Jan
12th unless someone has a blocking concern.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Moves duplicates of `DatePickerWrapper` and related code to package
`@kbn/ml-date-picker`. The duplicated components across the code base
have been consolidated and diverging features combined. Each duplicate
has been checked with a diff against the package before deletion.
- Moves duplicates of `query_utils.ts` to a package
`@kbn/ml-query_utils`.
- Some jest test were migrated from enzyme to react-testing-lib.
- `i18n` strings and data-test-subjects have been updated to be prefixes
in line with package names.
- Replaces custom code related to the `compact` flag with EUI's
breakpoints.
Moves the `src/plugins/chart_expressions/common` directory to a package.
This was done by:
```sh
# move the existing code out of the way
mv src/plugins/chart_expressions/common src/plugins/chart_expressions/_common
# generate the new package in the previous location
node scripts/generate package @kbn/chart-expressions-common --dir src/plugins/chart_expressions/common
# move the old files into the new package
mv src/plugins/chart_expressions/_common/* src/plugins/chart_expressions/common/
rmdir src/plugins/chart_expressions/_common
```
After that I just needed to fix the eslint violations 👍
## Summary
To remove boundaries between the ResponesOps teams about connectors this
PR removes the distinguishment between connectors by moving all
connectors to the same folder and changing the `CODEOWNERS` of the
`stack_folder` to be the ResponseOps team. I did not change any logic in
the connectors. I fixed path imports when appropriate and added some
`waitFor` for tests that for some reason were failing.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This PR implements a linter like the TS Project linter, except for
packages in the repo. It does this by extracting the reusable bits from
the TS Project linter and reusing them for the project linter. The only
rule that exists for packages right now is that the "name" in the
package.json file matches the "id" in Kibana.jsonc. The goal is to use a
rule to migrate kibana.json files on the future.
Additionally, a new rule for validating the indentation of tsconfig.json
files was added.
Validating and fixing violations is what has triggered review by so many
teams, but we plan to treat those review requests as notifications of
the changes and not as blockers for merging.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Update the codeowners file to point to the new team, `Defend Workflows`.
The new label and team has been created.
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Moves multiple copies of `useStorage()` and related code to a package as
a single source. The different copies with hard coded types have been
adapted so `useStorage()` is now based on generics. Also moves
duplicates of `isDefined()` to its own package.
Effort to deduplicate code. Move nested property utilities and url state
to packages.
Boilerplate for the packages was created likes this:
```
node scripts/generate package @kbn/ml-url-state --web --dir ./x-pack/packages/ml/url_state
node scripts/generate package @kbn/ml-nested-property --web --dir ./x-pack/packages/ml/nested_property
```
I consolidated the different `url_state.ts` files. One thing to note:
Each one had its own definition for `pageKey: AppStateKey`. I changed
that and made it just `pageKey: string`, I suspect it's good enough.
Otherwise we'd have a reverse dependency on all consuming code.
Alternative: We could refactor to require overriding a generic to pass
in allowed values.
## Dearest Reviewers 👋
I've been working on this branch with @mistic and @tylersmalley and
we're really confident in these changes. Additionally, this changes code
in nearly every package in the repo so we don't plan to wait for reviews
to get in before merging this. If you'd like to have a concern
addressed, please feel free to leave a review, but assuming that nobody
raises a blocker in the next 24 hours we plan to merge this EOD pacific
tomorrow, 12/22.
We'll be paying close attention to any issues this causes after merging
and work on getting those fixed ASAP. 🚀
---
The operations team is not confident that we'll have the time to achieve
what we originally set out to accomplish by moving to Bazel with the
time and resources we have available. We have also bought ourselves some
headroom with improvements to babel-register, optimizer caching, and
typescript project structure.
In order to make sure we deliver packages as quickly as possible (many
teams really want them), with a usable and familiar developer
experience, this PR removes Bazel for building packages in favor of
using the same JIT transpilation we use for plugins.
Additionally, packages now use `kbn_references` (again, just copying the
dx from plugins to packages).
Because of the complex relationships between packages/plugins and in
order to prepare ourselves for automatic dependency detection tools we
plan to use in the future, this PR also introduces a "TS Project Linter"
which will validate that every tsconfig.json file meets a few
requirements:
1. the chain of base config files extended by each config includes
`tsconfig.base.json` and not `tsconfig.json`
1. the `include` config is used, and not `files`
2. the `exclude` config includes `target/**/*`
3. the `outDir` compiler option is specified as `target/types`
1. none of these compiler options are specified: `declaration`,
`declarationMap`, `emitDeclarationOnly`, `skipLibCheck`, `target`,
`paths`
4. all references to other packages/plugins use their pkg id, ie:
```js
// valid
{
"kbn_references": ["@kbn/core"]
}
// not valid
{
"kbn_references": [{ "path": "../../../src/core/tsconfig.json" }]
}
```
5. only packages/plugins which are imported somewhere in the ts code are
listed in `kbn_references`
This linter is not only validating all of the tsconfig.json files, but
it also will fix these config files to deal with just about any
violation that can be produced. Just run `node scripts/ts_project_linter
--fix` locally to apply these fixes, or let CI take care of
automatically fixing things and pushing the changes to your PR.
> **Example:** [`64e93e5`
(#146212)](64e93e5806)
When I merged main into my PR it included a change which removed the
`@kbn/core-injected-metadata-browser` package. After resolving the
conflicts I missed a few tsconfig files which included references to the
now removed package. The TS Project Linter identified that these
references were removed from the code and pushed a change to the PR to
remove them from the tsconfig.json files.
## No bazel? Does that mean no packages??
Nope! We're still doing packages but we're pretty sure now that we won't
be using Bazel to accomplish the 'distributed caching' and 'change-based
tasks' portions of the packages project.
This PR actually makes packages much easier to work with and will be
followed up with the bundling benefits described by the original
packages RFC. Then we'll work on documentation and advocacy for using
packages for any and all new code.
We're pretty confident that implementing distributed caching and
change-based tasks will be necessary in the future, but because of
recent improvements in the repo we think we can live without them for
**at least** a year.
## Wait, there are still BUILD.bazel files in the repo
Yes, there are still three webpack bundles which are built by Bazel: the
`@kbn/ui-shared-deps-npm` DLL, `@kbn/ui-shared-deps-src` externals, and
the `@kbn/monaco` workers. These three webpack bundles are still created
during bootstrap and remotely cached using bazel. The next phase of this
project is to figure out how to get the package bundling features
described in the RFC with the current optimizer, and we expect these
bundles to go away then. Until then any package that is used in those
three bundles still needs to have a BUILD.bazel file so that they can be
referenced by the remaining webpack builds.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
After moving away from composite projects in the IDE we now have an
issue where projects like security solutions are getting `@types/jest`
and `@types/mocha` loaded up, even though the "types" compiler option in
security solutions focuses on jest. To fix this I've removed the
`@types/mocha` package, implemented/copied a portion of the mocha types
into a new `@kbn/ambient-ftr-types` package which can be used in ftr
packages to define the describe/it/etc. globals.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Replace @elastic/kibana-app-services ->
@elastic/kibana-global-experience for `kbn-monaco` package
While working on #146379 I just saw that the plugin has the wrong
code-owner. Let's fix it
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@maximpn brought up the issues caused by the types required by the
rison-node package, which attempted to communicate that "encoded values
must be primitive values, or recursive arrays/object of primitive
values". This isn't actually expressible in TypeScript, which lead to
many instances of `rison.encode(value as unknown as RisonValue)` which
is useless. Additionally, the rison-node library actually supports any
value and will either produce valid rison or `undefined` for that value.
To address this I'm adding a wrapper function which accepts `any` and
returns a `string`. If rison-node is totally unable to produce any rison
for the value (because the value is `undefined` or some other type like
Symbol or BigInt) the `encode()` function will throw. If you're
accepting arbitrary input you can use the `encodeUnknown()` function,
which will return a string or undefined, if the value you provided has
zero rison representation.
Like JSON.stringify() any non-circular primitive, object, or array can
be encoded with either function. If the values within those objects are
not encodable (functions, RegExps, etc) then they will be skipped. Any
object/array with the `toJSON()` method will be converted to JSON first,
and if the prototype of the object has the `encode_rison()` method it
will be used to convert he value into rison.
The changes in this PR are mostly updating usage of rison-node to use
`@kbn/rison` (which is also enforced by eslint). There are also several
changes which remove unnecessary casting.
Closes https://github.com/elastic/kibana/issues/135678
## Summary
This PR continues the work started in
https://github.com/elastic/kibana/pull/142758 to bring field list
grouping from Lens into Discover.
- [x] Integrate new components and hooks into Discover page
- [x] Refactor fields grouping logic
- [x] Render Popular fields under a new separate section
- [x] Remove "Hide empty fields" switch
- [x] Adjust filtering logic
- [x] Refactor fields existence logic in Discover
- [x] Add "Unmapped fields" section
- [x] Highlight the matching term when searching for a field
- [x] Show field icons when in SQL mode
- [x] Add tooltips to field list section headings
- [x] Add tests, clean up
<img width="340" alt="Screenshot 2022-11-15 at 15 39 27"
src="https://user-images.githubusercontent.com/1415710/201947349-726ffc3a-a17f-411b-be92-81d97879765a.png">
For testing on Discover page:
Please check different use cases and toggling Advanced Settings:
- regular vs ad-hoc data views
- data views with and without a time field
- data views with unmapped and empty fields
- data views with a lot of fields
- data views with some fields being filtered out via data view
configuration
- updating query, filters, and time range
- regular and SQL mode
- searching by a field name in the sidebar
- applying a field filter in the sidebar
- adding, editing, and removing a field
- Field Statistics table when some columns are selected or no columns
are selected
- multifields in the field popover should work as before (icon should
change from "+" to "x" when subfield is selected as a column)
- `discover:searchOnPageLoad` should not show fields if turned off
- `discover:searchFieldsFromSource` should show multifields right in the
fields list if enabled
- `discover:enableSql` should show Selected and Available fields only
when enabled
- `discover:showLegacyFieldTopValues` should show old (green) field
stats in its popover
- `doc_table:legacy`
On Lens page:
- scroll position should reset when data view is switched or when
searching by a field name
- regular and SQL mode
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
Co-authored-by: Michael Marcialis <michael@marcial.is>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
Reshuffles all remaining App Services CODEOWNERS paths to various teams.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This is a refactor:
* Move `FilesContext`, `FilePicker` and `UploadFile` components to
`packages/shared-ux/file` as packages
* Renamed `UploadFile` to `FileUpload` for more consistency
* Also created `packages/shared-ux/file/types` and added
`useBehaviourSubject` to `packages/shared-ux/file/util` (we can consider
moving this elsewhere since that function is not necessarily tied to the
files domain).
* Removed the storybook config from `files` public plugin since there
are no more components there
## How to test
👉🏻 `yarn storybook shared_ux` to see the components in a lab environment
OR
👉🏻 `yarn start --run-examples` then "Developer examples" > "Files
example" to see the components being used in Kibana
Look out for any regressions: for example, in the `FileImage` component
importing `import bh from 'blurhash'` caused a regression because
blurhash does not expose a default export. This was fixed by doing:
`import * as bh from 'blurhash`.
## Notes
* With this change, we needed to move `FilesClient` interface to
packages since it is used by the components. However, we also wanted to
keep `FilesClient` interface as it is currently exported from `files`
plugin because it exposes methods that only the server of `files` plugin
should know about (e.g., the metrics endpoint). I created the
`BaseFilesClient` in the packages directory that is extended in the
`files` plugin as needed. This is a snapshot of the types as they are
provided from the server implementation and will need to be
updated/maintained by hand from here on out.
* With `BaseFilesClient` in `packages`, we lost the type check between
`files` server endpoints and the client methods. To re-establish this
link the `CreateRouteDefinition` type helper got a parameter where the
client method can be passed in to do checks that the server inputs
(query, param and body) as well as outputs (the responses) match what
the client expects using the `X extends Y ? X : unknown` capability of
TS. See this in action in, for example
`src/plugins/files/server/routes/find.ts`. DX will be: if these ever get
out of sync, the server values for `query`, `param` or `body` will map
to `unknown` causing a type issue when trying to use these values. This
can only be fixed by bringing the `FilesClient` types in sync with the
server types.
* Server endpoints that should match expected `FilesClient`
inputs/outputs should use the `CreateRouteDefinition` type helper, but
if the endpoint does not need to map to a client method we can always
skip using `CreateRouteDefinition`.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
* Creates new packages `@kbn/shared-ux-file-image`,
`@kbn/shared-ux-file-image-types`, `@kbn/shared-ux-file-image-mocks`
* Replace all instance of usage to import from package (for now, just
files example plugin)
* Created a `@kbn/shared-ux-file-util` package for the helpers shared
across components
## How to test
See `yarn storbook shared_ux`'s new section "Files"
## Additional notes
First step just focussed on moving `Image` component, we still need to
move `FilePicker` and `UploadFile`
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
In order to get us closer to the developer experience we want for
packages, we are trying to move package builds out of bazel and instead
we want to build files on demand. In the case of .peggy files this means
importing them directly and teaching babel/jest/webpack how to handle
these imports by automatically transpiling and caching the results.
This change does just that, adding a `@kbn/peggy` package which wraps
peggy for types, and also adds support for defining peggy config
adjacent to a peggy grammar file in a `${basename}.config.json` file.
This file will be parsed and used to configure things like
`allowedStartRules` as described in [the peggy
docs](https://peggyjs.org/documentation.html#generating-a-parser-javascript-api).
This PR also implements `@kbn/peggy-loader` which uses `@kbn/peggy` to
transpile peggy files in webpack, and a peggy transform for both Jest
and our custom babel register hook.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/144255
Create the `@kbn/core-root-server-internal` package, and move `Root`,
`Server` and `bootstrap` to it.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Moves the ownership for some of the appservices code to the
visualizations team:
- expressions plugin
- unified search
- data plugin, we decided for now to co-own it with the
@elastic/kibana-data-discovery team
## Summary
In order to reduce the number of irrelevant PR review requests, this PR
updates the CODEOWNERS file and removes the
@elastic/security-detections-response team from code owners of the
`rule_registry` plugin. This affects the following sub-teams:
- @elastic/security-detections-response-alerts. Although this team owns
the alerts generation and indexing of the Detection Engine rule types
(and this code depends on the `RuleDataClient`), they prefer not to be a
code owner, because they keep getting pinged for PRs that don't need
their review.
- @elastic/security-detections-response-rules area team. This team
doesn't depend on the rule registry and doesn't do any work related to
alerts-as-data.
- @elastic/security-solution-platform. This team won't be doing anything
in the rule registry for the foreseeable future.
https://github.com/elastic/kibana/pull/144326 attempted to change the
owners for packages, but because of a misconfiguration in CI the correct
codeowners file wasn't determined until after the PR was merged. This
fixed the issue by modifying the kibana.jsonc files of the packages to
use the correct owner, which then ends up in the codeowners file.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
