## Summary
closes: https://github.com/elastic/kibana/issues/155301
Moves _packages/security-solution/_ to
_x-pack/packages/security-solution/_
Moves _x-pack/packages/kbn-securitysolution-*_ into the new
_x-pack/packages/security-solution/_
It contains 3 packages now:
- data_view/
- ecs_data_quality_dashboard/
- side_nav/
Package names and ids have not changed.
## Other
- eslint configured for all the packages in the directory
- i18n prefix `securitySolutionPackages` configured for all packages in
the directory
- generic storybook configuration, run with: `yarn storybook
security_solution_packages`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Allow user's to set their desired theme on their User Profile
## How to test
Login as a non-cloud user, navigate to User Profile:
<img width="1051" alt="Screenshot 2023-02-28 at 1 40 34 PM"
src="https://user-images.githubusercontent.com/21210601/221948512-a3e9b485-d3fa-4646-ae7d-63a68777cf19.png">
## Release Note
Users can now select their theme preference for Kibana in their User
Profile
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Michael Marcialis <michael.l.marcialis@gmail.com>
Closes#154733
Creates a new plugin for logs onboarding with wizard to organize steps
into discrete views.
#### TODO:
- [x] rename plugin to observability_onboarding
- [x] configure: UI and server plugin
- [x] enable/disable new plugin
- [x] remove the link to it from Observability nav
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Yngrid Coello <yngrid.coello@elastic.co>
Co-authored-by: Yngrid Coello <yngrdyn@gmail.com>
## Summary
Abstract types for using Saved Objects with the content management api.
This should significantly reduce the amount of code to use additional
saved object types.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Moves code from `x-pack/plugins/ml/common/util/errors` that was shared
via `x-pack/plugins/ml/public/shared.ts` to `@kbn/ml-error-utils`.
- `data_visualizer` and `aiops` plugins now use that package instead of
code duplication.
## Summary
The EUI team recently changed its GitHub team name to @elastic/eui-team.
We're updating all references in Kibana's CODEOWNERS as a result.
### Checklist
N/A, CODEOWNERS change only
- Refactors individual helpers to manage the `random_sampler`
aggregation into a single reusable wrapper. The helper's factory can be
supplied with either a sample probability right away or a document count
to generate a dynamic sample probability based on it.
- Applies random sampling to the main date histogram chart.
Closes#153202, closes#153850
## Summary
This PR adds alert start annotation and also uses a custom time range
for the alert details' charts depending on the alert duration. The logic
to calculate the time range was added in a separate package to be used
in other use cases as well.
## 🧪 How to test
Create a metric threshold alert and go to the related alert details
page, verify:
- Alert start annotation
- The time range of the charts should be before the alert was started
(1/8 of the duration was added to each side)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Now that we merged https://github.com/elastic/kibana/pull/153543, this
PR exposes the versioned router for teams to start using. The versioned
router will be available on `IRouter` under a new `versioned` property.
Primary benefit of this approach is that plugin developers will not need
to do anything other than "get" the `versioned` property to get a
versioned router.
Drawback is that this precludes us from passing in additional
configuration, like a version, to scope the versioned router instance.
For that we would need some kind of `createVersionedRouter({ version:
... })`. At this point it is not clear this is necessary, we could
revisit this decision based on actual usage. Plugin developers could
also do something like:
```ts
// common const
const MY_API_VERSION: ApiVersion = '1';
// in routes
import {MY_API_VERSION} from '../from/common';
router.versioned.get({ path: ... })
.addVersion({ version: MY_API_VERSION });
```
In this way they could get many of the same benefits of a version-scoped
version router, with the drawback that they need to pass this in for
every route.
### TODO
- [x] Add an integration test for the versioned router
### Future work
* We still need to consider revisiting some of the router design to
better support internal cases like adding support for registering a
handler for a version range and adding a default version to continue
supporting on-prem where introducing versions will be a breaking change
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Creates a package for sharing general trained model util code.
Initially only contains the constants which had been copied in the
enterprise search plugin.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
resolves https://github.com/elastic/kibana/issues/142874
The alerting framework now generates an alert UUID for every alert it
creates. The UUID will be reused for alerts which continue to be active
on subsequent runs, until the alert recovers. When the same alert (alert
instance id) becomes active again, a new UUID will be generated. These
UUIDs then identify a "span" of events for a single alert.
The rule registry plugin was already adding these UUIDs to it's own
alerts-as-data indices, and that code has now been changed to make use
of the new UUID the alerting framework generates.
- adds property in the rule task state
`alertInstances[alertInstanceId].meta.uuid`; this is where the alert
UUID is persisted across runs
- adds a new `Alert` method getUuid(): string` that can be used by rule
executors to obtain the UUID of the alert they just retrieved from the
factory; the rule registry uses this to get the UUID generated by the
alerting framework
- for the event log, adds the property `kibana.alert.uuid` to
`*-instance` event log events; this is the same field the rule registry
writes into the alerts-as-data indices
- various changes to tests to accommodate new UUID data / methods
- migrates the UUID previous stored with lifecycle alerts in the alert
state, via the rule registry *INTO* the new `meta.uuid` field in the
existing alert state.
## Summary
Fixes https://github.com/elastic/kibana/issues/147467
Users will be able to specify params in following format in project
monitors or UI
```
- type: http
name: Admin Check
url: ...
username: ${admin_user:admin}
password: ${admin_password}
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dominique Clarke <dominique.clarke@elastic.co>
## Summary
Implements the designs from
https://github.com/elastic/kibana/pull/151596
* Move `packages/versioning/*` into `packages/core/http` to follow
existing structure more closely
* Implements the first iteration of the versioned router as a
wrapper/layer around the existing router
* Adds some integration tests
* Future work needed! Once we have a the versioned spec we should
implement it in this wrapper layer
* Validation is a little bit tricky because of when the
`CoreKibanaResponse` object is instantiated, the approach taken here is
to replace body, params, query on the route-level's request object
Closes https://github.com/elastic/kibana/issues/149286
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Actions to add issues to specific projects are failing with the
following:
```Project label assigner action failed with error: Error querying project ID for project number 669: Your token has not been granted the required scopes to execute this query. The 'projectV2' field requires one of the following scopes: ['read:project'], but your token has only been granted the: ['repo', 'workflow', 'write:org'] scopes. Please modify your token's scopes at: https://github.com/settings/tokens. ```
This PR attempts to fix it by removing the offending token and relying on `GITHUB_TOKEN` instead as recommended in [https://github.com/richkuz/projectnext-label-assigner#github-token](https://github.com/richkuz/projectnext-label-assigner#github-token)
Closes https://github.com/elastic/kibana/issues/151702
## Summary
This PR migrates drag and drop logic from Lens plugin to a new package
so we can reuse it on Discover page later. At this point there should be
no visual changes. If you notice something, please comment on the PR.
- [x] Migrate drag&drop code to its own package `@kbn/dom-drag-drop`
- [x] Clean up i18n strings
- [x] Clean up styles
- [x] Adjust tests
- [x] Make telemetry optional
- [x] Configurable `data-test-subj`
Please test by using your mouse and also by using keyword shortcuts.
# Next steps
- Redesign for field list item (smaller button, a separate handle icon,
pill styles)
- Redesign for draggable buttons in the Lens layer panels (smaller
buttons)
-
[Figma](https://www.figma.com/file/SvpfCqaZPb2iAYnPtd0Gnr/KUI-Library?node-id=674%3A198901&t=OnQH2EQ4fdBjsRLp-0)
- https://github.com/elastic/kibana/issues/151703
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
As we discussed in an email, this is our proposal to facilitating the
Observability teams to create Kibana instances attached to the
Observability test environments by using the GitHub command
`/oblt-deploy`
If an Elastician added a GitHub comment then, the automation will create
the Kibana instance based on the PR changes. Then a comment with the
link to the GitHub issue that contains all the configuration details.
The GitHub issue is not accessible to the public since it's under a
private GitHub repository.
## Summary
This plugin will contain the asset inventory and topology API in Kibana,
giving Kibana projects access to inventory and topology data via an HTTP
and/or JS API on the server and client.
[Currently proposed API
docs](https://github.com/elastic/o11y-topology-playground/tree/main/docs/api)
will be moved to this repo as well, contained inside this plugin folder,
as a part of this PR.
## Enabling the plugin
This plugin is entirely in "technical preview" and because of this, must
be specifically enabled via config for it to do anything besides being
run by the core plugin framework. To enable the server API layer, as
well as the index template management, put the following line in your
kibana.yml file:
```yml
xpack.assetManager.alphaEnabled: true
```
## Running the API integration tests
Run the functional test server with the asset manager config in place:
```shell
$ node scripts/functional_tests_server --config x-pack/test/api_integration/apis/asset_manager/config.ts
```
Then run the functional test runner with the same config, to target just
these tests:
```shell
$ node scripts/functional_test_runner --config=x-pack/test/api_integration/apis/asset_manager/config.
ts
```
_Note:_ The config file added in this folder enables the tech preview
plugin ([see file
here](https://github.com/elastic/kibana/pull/152456/files#diff-bc00de6c34c9bc131cfbdf3570c487fe9ee947e9a88a84c59d6b139b79d7708eR20)).
### Running the integration tests for verifying that the plugin is
"disabled" by default
There is a small set of tests that confirm that the endpoints return 404
and there is no index template installed if the config value is not set
in the kibana.yml file. To run this suite, use the following config:
```shell
$ node scripts/functional_tests_server --config x-pack/test/api_integration/apis/asset_manager/config_when_disabled.ts
$ node scripts/functional_test_runner --config=x-pack/test/api_integration/apis/asset_manager/config_when_disabled.
ts
```
## Testing this PR with sample data
There are some sample data mechanisms in place inside this PR to allow
us to build out the endpoints.
### View sample docs
```http
GET /api/asset-manager/assets/sample
```
This will return a list of the assets that are included if you elect to
write assets. This is a good endpoint to use to find EAN (Elastic Asset
Name) values that you may want to exclude from writing for a given time
period, to simulate assets appearing/disappearing over time.
### Write sample docs
```http
POST /api/asset-manager/assets/sample
{
"baseDateTime": "2023-02-28T12:00:00.000Z",
"excludeEans": ["k8s.cluster:cluster-002"]
}
```
This posts all of the sample asset documents to Elasticsearch using the
`baseDateTime` value as the timestamp. Any valid string or number that
is accepted by `new Date()` should work for `baseDateTime`.
The `excludeEans` value is an array of EAN ("Elastic Asset Name") values
that you don't want to write on this particular run. This way you can
have assets appear (exclude them in the past, don't exclude them during
a later run) or disappear (vice versa) and see how that shows up in
other endpoints.
**Note:** *Remember that when you curl a Kibana server API with a POST
request, you must include a `kbn-xsrf` header with any string value you
want.*
### Get asset docs from ES
```http
GET /api/asset-manager/assets?type=k8s.cluster&from=now-10m
```
This is the primary "real" endpoint available right now. It should
retrieve a list of assets based on the type/from/to/ean filter values
you specify. Once you load the sample data, this endpoint should return
results.
## Debug logging
There are some extra debug logs for ES queries that are running in the
code in this PR. To print those logs to the Kibana server console, run
Kibana using `DEBUG_LOGGER=true`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The owner of the security-solution/storybook package was incorrectly set
to `appex-sharedux` (sorry, copy/pasta 🍝 mistake)
Changed to `security-threat-hunting-explore`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/144887
## Summary
This PR adds an ESLint Plugin which checks specific `Eui` elements for
the existence of a `data-test-subj` prop. This rule will make having one
for these elements required.
This rule is currently only enabled for Observability apps (APM, Infra,
Observability, Synthetics, Uptime).
The plugin is also able to generate a suggestion based on the context in
which the element is used. In the IDE this suggestion can be applied by
using the autofix capability (see video below).
When opening a PR, the CI will automatically apply the suggestion to
qualifying Eui elements in the branch.
https://user-images.githubusercontent.com/535564/225449622-bbfccb40-fdd2-4f69-9d5a-7d5a97bf62e6.mov
## Why do this?
There is an increased push to move towards data driven feature
development. In order to facilitate this, we need to have an increased
focus on instrumenting user event generating elements in the Kibana
codebase. This linting rule is an attempt to nudge Kibana engineers to
not forget to add this property when writing frontend code. It also
saves a bit of work for engineers by suggesting a value for the
`data-test-subj` based on the location of the file in the codebase and
any potential default values that might be present in the JSX node tree.
Finally, because the suggestion is always of the same form, it can
increase the consistency in the values given to these elements.
## Shape of the suggestion
The suggestion for the value of data-test-subj is of the form:
`[app][componentName][intent][euiElementName]`.
For example, when working in a component in the location:
`x-pack/plugins/observability/public/pages/overview/containers/overview_page/header_actions.tsx`,
and having the code:
```
function HeaderActions() {
return (
<EuiButton>{i18n.translate('id', { defaultMessage: 'Submit Form' })}</EuiButton>
)
}
```
the suggestion becomes:
`data-test-subj=o11yHeaderActionsSubmitFormButton`.
For elements that don't take a `defaultMessage` prop / translation, the
suggestion takes the form: `[app][componentName][euiElementName]`
## Which elements are checked by the ESLint rule?
In its current iteration the rule checks these Eui elements:
* `EuiButton`
* `EuiButtonEmpty`
* `EuiLink`
* `EuiFieldText`
* `EuiFieldSearch`
* `EuiFieldNumber`
* `EuiSelect`
* `EuiRadioGroup`
* 'EuiTextArea`
## What types of prop setting does this rule support?
* `<EuiButton data-test-subj="foo">` (direct prop)
* `<EuiButton {...foo}>` (via spreaded object; rule checks for
`data-test-subj` key in object)
## What types of function declarations does this rule support?
* `function Foo(){}` (Named function)
* `const Foo = () => {}` (Arrow function assigned to variable)
* `const Foo = memo(() => {})` (Arrow function assigned to variable
wrapped in function)
* `const Foo = hoc(uponHoc(uponHoc(() => {})))` (Arrow function assigned
to variable wrapped in infinite levels of functions)
## Things to note
* If an element already has a value for `data-test-subj` the rule will
not kick in as any existing instrumentation might depend on the value.
* the auto suggestion is just a suggestion: the engineer can always
adjust the value for a `data-test-subj` before or after committing. Once
a value is present (autofixed or manually set) the rule will not kick
in.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dario Gieselaar <d.gieselaar@gmail.com>
Co-authored-by: Katerina Patticha <kate@kpatticha.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
## Summary
Notify appex-qa for changes to FTR
Also, cleanup...keeps qa related items
in one area in the file.
Drop vars directory declaration...no longer needed.
## Summary
Close https://github.com/elastic/kibana/issues/152002
In https://github.com/elastic/kibana/pull/151163 we introduced a simple
demo todo app run in a storybook with a custom client-side content
management client (no server-side cm registry usage).
This is a follow-up PR that re-uses the same demo todo app, but also
runs it in an example plugin with proper server-side content management
registry usage, so now we have a basic end-to-end demonstration of
content management capabilities. The demo app is covered by functional
tests, so now we also have basic end-to-end test coverage.
As this is the first kind of real-world end-to-end usage of the CM APIs,
I'd like to use this and
[previous](https://github.com/elastic/kibana/pull/151163) prs as a base
for the discussion and polishing current APIs. I'll leave a review with
comments where I think some API polishing is needed.
**Notable changes apart from the example plugin itself:**
1. Move `demo/` todo app and its stories introduced in
https://github.com/elastic/kibana/pull/151163 from
`src/plugins/content_management` to
`examples/content_management_examples`. This was mostly needed to not
export `demo/` code on the public plugin export to avoid increasing
bundle size.
2. Add needed exports to the plugin contract
3. Reshuffle `common/` to not import `@kbn/schema` client side
48aa41403b
4. Fix client-side RPC client to work with the latest server-side
changes (shouldn't break from now on because of the end-to-end test
coverage)
Resolves https://github.com/elastic/kibana/issues/150358
## Summary
In a previous [PR](https://github.com/elastic/kibana/pull/145581) we
started installing a common component template for framework alerts as
data when the `xpack.alerting.enableFrameworkAlerts` config flag is set
to true. In that PR we used a different naming pattern than what is used
by the rule registry for its component templates.
In this PR we are doing the following:
* Renaming the installed `alerts-common-component-template` to
`.alerts-framework-mappings`.
* Creating and installing `.alerts-legacy-alert-mappings` component
template when `enableFrameworkAlerts: true` on alerting plugin setup
* The combination of the two component templates creates the same set of
mappings as the rule registry technical component template
* Creating and installing `.alerts-ecs-mappings` component template when
`enableFrameworkAlerts: true` on alerting plugin setup (when
`enableFrameworkAlerts: false`, the rule registry continues to install
this component template
* Using the `@kbn/ecs` package provided by core to generate the ECS
field map. The rule registry will continue to install the existing ECS
field map which is actually a subset of ECS fields
* Adding `useLegacy` and `useEcs` flags that allow rule types to specify
whether to include the legacy alerts component template and the ECS
component template when registering with framework alerts-as-data.
* Moved some common functions to alerting framework from the rule
registry
## Things to note
* When generating the ECS field map, we are now including the
`ignore_above` setting from the `@kbn/ecs` package. This changes the ECS
component template to include those settings. I tested updating an index
with just `"type":"keyword"` mappings to add the `ignore_above` field to
the mapping and had no issues so this seems like an additive change to
the mapping that will hopefully prevent problems in the future.
* The rule registry ECS component template also includes the technical
fields which is redundant because the technical component template is
automatically installed for all index templates so the framework ECS
component template only contains ECS fields.
| Previous mapping | Updated mapping |
| ----------- | ----------- |
| `{ "organization": { "type": "keyword" } }` | `{ "organization": {
"type": "keyword", "ignore_above": 1024 } }` |
## To Verify
### Verify that the generated component templates are as expected:
Get the following
**While running `main`:**
1. Get the ECS component template `GET
_component_template/.alerts-ecs-mappings`
2. Get the technical component template `GET
_component_template/.alerts-technical-mappings`
3. Create a detection rule that creates an alert and then get the index
mapping for the concrete security alert index `GET
.internal.alerts-security.alerts-default-000001/_mapping`
**While running this branch with `xpack.alerting.enableFrameworkAlerts:
false`:**
4. Get the ECS component template `GET
_component_template/.alerts-ecs-mappings`
5. Get the technical component template `GET
_component_template/.alerts-technical-mappings`
6. Create a detection rule that creates an alert and then get the index
mapping for the concrete security alert index `GET
.internal.alerts-security.alerts-default-000001/_mapping`
**While running this branch with `xpack.alerting.enableFrameworkAlerts:
true`:**
7. Get the ECS component template `GET
_component_template/.alerts-ecs-mappings`
8. Get the technical component template `GET
_component_template/.alerts-technical-mappings`
9. Create a detection rule that creates an alert and then get the index
mapping for the concrete security alert index `GET
.internal.alerts-security.alerts-default-000001/_mapping`
10. Verify that component templates exist for
`.alerts-framework-mappings` and `.alerts-legacy-alert-mappings`
**Compare the ECS component templates**
Compare 1 and 4 (ECS component template from `main` and installed by
rule registry in this branch). The difference should be:
* no difference in ECS fields
* because the rule registry ECS component template also includes
technical fields, you will see the 2 new technical fields in this branch
Compare 4 and 7 (ECS component template from rule registry & alerting
framework in this branch).
* some new ECS fields for alerting installed template
* each `keyword` mapped field for alerting installed template should
have `ignore_above` setting
* no `kibana.*` fields in the alerting installed template
**Compare the technical component templates**
Compare 2 and 5 (technical component template from `main` and installed
by rule registry in this branch). The difference should be:
* 2 new `kibana.alert` fields (`flapping_history` and `last_detected`)
Compare 5 and 8 (technical component template from rule registry &
alerting framework in this branch).
* there should be no difference!
**Compare the index mappings**
Compare 3 and 6 (index mapping from `main` and installed by rule
registry in this branch). The difference should be:
* 2 new `kibana.alert` fields (`flapping_history` and `last_detected`)
Compare 6 and 9 (index mapping from rule registry & alerting framework
in this branch).
* some new ECS fields
* each `keyword` mapped ECS field should have `ignore_above` setting
### Verify that the generated component templates work with existing
rule registry index templates & indices:
1. Run `main` or a previous version and create a rule that uses both ECS
component templates & technical component templates (detection rules use
both). Let it run a few times.
2. Using the same ES data, switch to this branch with
`xpack.alerting.enableFrameworkAlerts: false` and verify Kibana starts
with no rule registry errors and the rule continues to run as expected.
3. Using the same ES data, switch to this branch with
`xpack.alerting.enableFrameworkAlerts: true` and verify Kibana starts
with no alerting or rule registry errors and the rule continues to run
as expected. Verify that the mapping on the existing
`.internal.alerts-security.alerts-default-000001` has been updated to
include the latest ECS mappings and the two new technical fields.
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Mike Côté <mikecote@users.noreply.github.com>
## Summary
Removes the `server/lib/detection_engine/signals/` folder and moves
files from it into the appropriate places in
`server/lib/detection_engine/rule_types/`. The intent is to keep logic
related to each rule type co-located, instead of randomly split across
`/signals` and `/rule_types`.
No functional changes in this PR.
## Summary
The @elastic/observability-design team was added as a joint code owner
for the page components in the shared Observability plugin, so that
overall design and UX changes to the main pages would notify that team.
As that team has gotten a bit smaller and as more technical code has
landed in those sections, this no longer seems like a good idea to have
this review bottleneck.
