Related to https://github.com/elastic/kibana/issues/169759.
## Summary
This PR creates the initial plugin for Dataset Quality.
The plugin has the basic boilerplate that allows us to continue working
on the UI and Server sides in parallel.
Further configs are to be added along with upcoming commits like:
1. Storybook
2. FTR configs
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
Entity Analytics/Risk Engine
- Introduced new folders called `entity_analytics/risk_engine` under
`security_solution_api_integration`
- Moved the utility files associated with risk_engine to the new
directory `security_solution_api_integration`. Files that were not
actively used in the previous folder were moved, while any duplicate
files remained in their original positions.
- Updated the CodeOwner file for the newly moved tests
- Old / new groups details and execution time
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
| Action | File | New Path if moved |
|--------|------|----------|
| Moved|
group10/risk_engine/init_and_status_apis|/entity_analytics/default_license/risk_engine/init_and_status_apis.ts
|
| Moved|
group10/risk_engine/risk_score_calculation|/entity_analytics/default_license/risk_engine/risk_score_calculation.ts
|
| Moved|
group10/risk_engine/risk_score_preview|/entity_analytics/default_license/risk_engine/risk_score_preview.ts
|
| Moved|
group10/risk_engine/risk_scoring_task_execution|/entity_analytics/default_license/risk_engine/risk_scoring_task_execution.ts
|
| Moved|
group10/risk_engine/telemetry_usage|/entity_analytics/default_license/risk_engine/telemetry_usage.ts
|
| Moved| group10/risk_engine/utils|entity_analytics/utils/risk_engine.ts
|
| Moved| utils/get_stats|entity_analytics/utils/get_risk_engine_stats.ts
|
Tests skipped on main:
https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/risk_engine/init_and_status_apis.ts#L363
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Nikita Khristinin <nkhristinin@gmail.com>
## Summary
This PR is simply moving the `risk_engine` folder into the
`entity_analytics` folder, to be able better consolidate future
architecture modules like `asset_criticality` and potential others
Part of https://github.com/elastic/kibana-team/issues/646
This PR adds an example plugin in `examples/error_boundary` that shows
usage of KibanaErrorBoundary.
The example plugin is used in a functional test to ensure errors are
caught in the appropriate way, and error messages include a working
Refresh button.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Updates new teams as codeowners for Observability team changes.
Also took the opportunity to:
- Delete some paths that no longer exist
- Split infra code ownership between teams (from #168992)
## Summary
- Following the initial work in this
https://github.com/elastic/kibana/pull/166755
- Addresses part of https://github.com/elastic/kibana/issues/151902 for
Alerts
- Introduced a new folder called `alerts` under
`security_solution_api_integration`
- Moved the utility files associated with alerts to the new directory
`security_solution_api_integration`. Files that were not actively used
in the previous folder were moved, while any duplicate files remained in
their original positions.
- Updated the CodeOwner file for the newly moved tests
- Old / new groups details and execution time
[document](https://docs.google.com/document/d/1CRFfDWMzw3ob03euWIvT4-IoiLXjoiPWI8mTBqP4Zks/edit)
| Action | File | New Path if moved |
|--------|------|----------|
| Delete| group1/find_rule_exception_references.ts| Already moved in
previous PR |
| Delete|security_and_spaces/group6 | - |
| Move|detection_engine_api_integration/security_and_spaces/group6
|detections_response/default_license/alerts/alerts_compatibility.ts|
| Move|detection_engine_api_integration/security_and_spaces/group1|
detections_response/default_license/alerts/aliases.ts |
| Move |detection_engine_api_integration/security_and_spaces/group1|
detections_response/default_license/alerts/create_index.ts|
|
Move|detection_engine_api_integration/security_and_spaces/group10/create_signals_migrations|
detections_response/default_license/alerts/migrations/create_alerts_migrations.ts
|
|
Move|detection_engine_api_integration/security_and_spaces/group10/delete_signals_migrations|
detections_response/default_license/alerts/migrations/delete_alerts_migrations.ts|
|
Move|detection_engine_api_integration/security_and_spaces/group10/finalize_signals_migrations|
detections_response/default_license/alerts/migrations/finalize_alerts_migrations.ts
|
|
Move|detection_engine_api_integration/security_and_spaces/group10/get_signals_migration_status|
detections_response/default_license/alerts/migrations/get_alerts_migration_status.ts|
|
Move|detection_engine_api_integration/security_and_spaces/group10/open_close_signals|
detections_response/default_license/alerts/open_close_alerts.ts|
|
Move|detection_engine_api_integration/security_and_spaces/group10/alert_tags|
detections_response/default_license/alerts/alert_tags.ts|
Note: these tests are skipped on the main branch
https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/open_close_signals.ts#L215https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/open_close_signals.ts#L252https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group10/finalize_signals_migrations.ts#L192https://github.com/elastic/kibana/blob/main/x-pack/test/detection_engine_api_integration/security_and_spaces/group1/create_index.ts#L42
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR makes the @elastic/security-engineering-productivity team the
owner of the `x-pack/test/security_solution_cypress/package.json` file
to prevent accidentally merging temporary changes to this file like in
https://github.com/elastic/kibana/pull/169182.
Also, it simplifies the ownership rules a bit. The following lines
assign the team to all files under the two specified folders, but not
files inside their subfolders, so you don't have to specify individual
files:
```
/x-pack/test/security_solution_cypress/* @elastic/security-engineering-productivity
/x-pack/test/security_solution_cypress/cypress/* @elastic/security-engineering-productivity
```
## Summary
close https://github.com/elastic/kibana/issues/168696
- Remove old copy from `kibana_react`
- Remove -types and -mocks packages, move them into a single package
(not needed since we don't use bazel now)
### Risks
No Risks, since old copy was not used
**Parent meta ticket:
https://github.com/elastic/security-team/issues/7491**
Resolves: https://github.com/elastic/security-team/issues/7582
Resolves: https://github.com/elastic/security-team/issues/7580
Resolves: https://github.com/elastic/security-team/issues/7581
## Summary
This PR migrates the rules schema to OpenAPI, Zod, and code generation.
The following APIs now have complete OpenAPI specifications and are
enabled for code generation:
| Method | Endpoint | OpenAPI spec | Fully migrated |
| ------ |
---------------------------------------------------------------- |
------------ | -------------- |
| POST | /api/detection_engine/rules | ✅ | ✅ |
| GET | /api/detection_engine/rules | ✅ | ✅ |
| PUT | /api/detection_engine/rules | ✅ | ✅ |
| PATCH | /api/detection_engine/rules | ✅ | ✅ |
| DELETE | /api/detection_engine/rules | ✅ | ✅ |
| POST | /api/detection_engine/rules/\_bulk_create | ✅ | ✅ |
| PUT | /api/detection_engine/rules/\_bulk_update | ✅ | ✅ |
| PATCH | /api/detection_engine/rules/\_bulk_update | ✅ | ✅ |
| DELETE | /api/detection_engine/rules/\_bulk_delete | ✅ | ✅ |
| POST | /api/detection_engine/rules/\_bulk_delete | ✅ | ✅ |
### Rule schemas are now forward-compatible
We now allow extra fields in schemas for forward compatibility, but we
remove them from the payload during parsing. So from now on, extra
fields are simply ignored and won't lead to validation errors.
## Summary
As the owners of ES|QL it makes sense to also own the monaco package
relevant files
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
### Summary
Fix https://github.com/elastic/kibana/issues/166688
Implements dynamic contract resolving for plugins, allowing to retrieve
contracts after their respective lifecycle is completed, and therefore
working around cyclic dependencies.
In term of workflow execution, we're basically going from
<img width="842" alt="Screenshot 2023-09-27 at 08 09 27"
src="251637d1-ec97-4071-a445-2f59512ce187">
to:
<img width="1092" alt="Screenshot 2023-09-27 at 08 09 32"
src="de466cda-7e43-4fd3-81ec-4339d05d279d">
### API
This functionality is exposed by the now publicly exposed `plugins`
service contracts:
```ts
setup(core) {
core.plugins.onSetup<{pluginA: SetupContractA, pluginB: SetupContractA}>('pluginA', 'pluginB')
.then(({ pluginA, pluginB }) => {
if(pluginA.found && pluginB.found) {
// do something with pluginA.contract and pluginB.contract
}
});
}
```
```ts
start(core) {
core.plugins.onStart<{pluginA: StartContractA, pluginB: StartContractA}>('pluginA', 'pluginB')
.then(({ pluginA, pluginB }) => {
if(pluginA.found && pluginB.found) {
// do something with pluginA.contract and pluginB.contract
}
});
}
```
**remark:** the `setup` contract exposed both `onSetup` and `onStart`,
while the `start` contract only exposed `onStart`. The intent is to
avoid fully disrupting the concept of lifecycle stages.
### Guardrails
To prevent developer from abusing this new API, or at least to add some
visibility on its adoption, plugins can only perforn dynamic contract
resolving against dependencies explicitly defined in their manifest:
- any required dependencies (*existing concept*)
- any optional dependencies (*existing concept*)
- any runtime dependencies (**new concept**)
Runtime dependencies must be specified using the new
`runtimePluginDependencies` field of a plugin's manifest.
```json
{
"type": "plugin",
"id": "@kbn/some-id",
"owner": "@elastic/kibana-core",
"plugin": {
"id": "some-id",
"...": "...",
"runtimePluginDependencies" : ["someOtherPluginId"]
}
}
```
Using the contract resolving API will throw at call time when trying to
resolve the contract for an undeclared dependency.
E.g this would throw at invocation time (not returning a rejected
promise - throw).
```ts
setup(core) {
core.plugins.onSetup<{undeclaredDependency: SomeContract}>('undeclaredDependency');
}
```
The reasoning behind throwing is that these errors should only occur
during the development process, and an hard fail is way more visible
than a promise rejection that should be more easily shallowed.
### Code reviews
This PR defines @elastic/kibana-core as codeowner of all `kibana.jsonc`
files in the `src/plugins` and `x-pack/plugins` directories, so that a
code review will be triggered whenever anyone changes something in any
manifest. The intent is to be able to monitor new usages of the feature,
via the addition of entries in the `runtimePluginDependencies` option of
the manifest.
### Remarks
Exposing this API, and therefore making possible cyclic dependencies
between plugins, opens the door to other questions.
For instance, cross-plugin type imports are not technically possible at
the moment, given that plugins are referencing each others via TS refs,
and refs forbid cyclic dependencies. Which means that to leverage this
to address cyclic dependency issues, the public types of **at least one
of the two** plugins will have to be extracted to a shared place (likely
a package).
Resolving, or trying to improve the developer experience around this
issue, is absolutely out of scope of the current PR (and the issue it
addresses).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
#### Code update:
`/detection_engine/routes/signals/set_alert_tags_route.ts`
- When enabling the alert tags cypress test, noticed it was failing as
the tag updates did not show in the UI until a few seconds later upon a
second refresh. I was able to recreate this locally on serverless, not
on ESS. I updated the alerts tag route to include `refresh: true` and
that seemed to fix this issue.
- `/detection_engine/routes/signals/open_close_signals_route.ts`
- When testing on serverless, alert status was stale after update.
Confirmed this with tests that were failing for ESS. Upon updating route
to use `refresh: true`, tests began passing and could see expected
behavior. This may make the call a bit heavier so we will want to see if
there are any performance impacts.
## Summary
Unskips Spaces a11y tests that are no longer failing. Also updates
CODEOWNERS to properly assign ownership of our a11y tests.
Resolves https://github.com/elastic/kibana/issues/144155
## Summary
This PR is copying over part of the API integration tests for the
`cloud_security_posture` plugin. It's a part of the work on
https://github.com/elastic/security-team/issues/7439.
Some tests are not yet copied over as they need more research, as they
rely on either creating new users in Kibana or on fleet utils. This
functionality is not yet available in serverless out of the box.
Why copy?
From [Kibana Serverless e2e Test
Guide](https://docs.google.com/document/d/1tiax7xoDYwFXYZjRTgVKkVMjN-SQzBWk4yn1JY6Z5UY/edit):
> Stateful tests should be copied over and modified as required for
these features to ensure thorough coverage.
which tests are copied and adopted for serverless:
`tests/api_integration/apis/cloud_security_posture/status/status_not_deployed_not_installed`
`tests/api_integration/apis/cloud_security_posture/status/status_indexed`
`tests/api_integration/apis/cloud_security_posture/status/status_indexing`
`tests/api_integration/apis/cloud_security_posture/benchmark`
`tests/api_integration/apis/cloud_security_posture/get_csp_rule_template`
`tests/cloud_security_posture_api/telemetry`
which tests are not yet adopted for serverless:
`tests/api_integration/apis/cloud_security_posture/status/status_unprivileged`
(user/roles model is different in serverless)
`tests/api_integration/apis/cloud_security_posture/status/status_waiting_for_results`
(fleet test utils are not available)
`tests/api_integration/apis/cloud_security_posture/status/status_index_timeout`
(fleet test utils are not available)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
# Summary
- Addresses https://github.com/elastic/kibana/issues/161537
## Description
- This PR follows the second option defined in this
[document](https://docs.google.com/document/d/1mqkpjDdFQRFvx_RPvNmjstVj8SXYMr2mrETMv3esda8/edit#heading=h.rpv1zyeb04ay)
the [Mocha tagging ](https://github.com/mochajs/mocha/wiki/Tagging)
- It introduces a new folder
`x-pack/test/security_solution_api_integration` which will serve as a
centralized location to meet all the requirements related to renaming
tests appropriately and grouping similar tests together. It will
facilitate the management of tests that must be run in Serverless and
ESS environments.
- Within this folder, there is a "config" subdirectory that stores base
configurations specific to both the
[Serverless](https://github.com/elastic/kibana/pull/166755/files#diff-afe1f42d5ac2006de8dc09069448b9e8734a6a950586376cd6e8eeb9110ab5f1R1)
and
[ESS](https://github.com/elastic/kibana/pull/166755/files#diff-4a60bd8c91da08a3f7ec14bf3bfef8449af155611374c32579b0318da03e292cR1)
environments, These configurations build upon the base configuration
provided by test_serverless and api_integrations, incorporating
additional settings such as environment variables and tagging options.
- It demonstrates scenarios involving `@ess`, `@serverless`, and
`@brokenInServerless`.
- The file`
x-pack/test/security_solution_api_integration/test_suites/detections_response/rule_creation/create_rules.ts`
is functional in both **Serverless** and **ESS**. However, some tests
related to roles are currently skipped for Serverless until they are
resolved, and these tests are tagged with `@brokenInServerless`
## CI
- It includes a new entry in the ftr_configs.yml to execute the newly
added tests in the pipeline.
- It involves the addition of mochaOptions in both
serverless/config.base.ts and ess/config.base.ts. In the case of
serverless, it includes **@serverless** while excluding
@brokenInServerless. Similarly, for **ess**, it includes @ess and
excludes **@brokenInEss**.
from `x-pack/test/security_solution_api_integration/config/serverless`
## Update in x-pack/test/detection_engine_api_integration
- The `create_rules.ts` and `create_rule_exceptions` files have been
relocated from
`x-pack/test/detection_engine_api_integration/security_and_spaces/group1`
to their respective domains within the
`x-pack/test/security_solution_api_integration` folder.
- The util files now are copied over from the old folder
`x-pack/test/detection_engine_api_integration` to the new folder and
will be removed once all tests are moved to the new folder to don't
break the existing tests
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
We've seen it the previous 2 times that the `create-deploy-tag.yml` is
failing when it's about to send the success slack message.
Now setting that step not to break the workflow - because in fact the
tag is created and the process is running.
The reasons why the slack messaging is failing to be investigated.
- Resolves https://github.com/elastic/kibana/issues/166906
## Summary
This PR adds field tokens to column header for the grid.
There are also additional necessary changes:
- field utils (name, description, icon) were moved from
`@kbn/unified-field-list` and `@kbn/discover-utils` to its own new
package `@kbn/field-utils`
- Unified Data Table and Unified Doc Viewer got a new prop `columnTypes`
which allows to render correct field icons for ES|QL searches (before
types were derived from the data view fields which could be misleading
in text-based searches as users can customize field names via query)
<img width="600" alt="Screenshot 2023-09-25 at 19 30 21"
src="388de9bb-94f7-4d3e-878a-ca0da99fcec2">
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
## Summary
This PR copies, with limited modifications, functional test suites from
`test/functional` and `x-pack/test/functional`
## Lens Smoke tests
- [x] Smokescreen tests
(x-pack/test/functional/apps/lens/group1/smokescreen.ts)
- [x] Basic vega tests
(test/functional/apps/visualize/group6/_vega_chart.ts)
- [x] Sanity checks for TSDB functionality in Lens
x-pack/test/functional/apps/lens/group4/tsdb.ts
## Convert to Lens
These tests outside of serverless, assume the following vis types are
accessible via the **Visualize editor**, however all the following types
are designated **readonly** in serverless and thus the fn tests fail.
In place of these tests I created the identical visualizations, added
them to a dashboard and exported the SOs. These dashboard and dependent
visualizations are loaded in each respective test, and the assertions
are run on the `Convert to Lens` option from the dashboard. All
assertions are mostly unchanged or achieve the same effect.
- ✅ Agg based -
`x-pack/test/functional/apps/lens/open_in_lens/agg_based`
- ✅ TSVB - `x-pack/test/functional/apps/lens/open_in_lens/tsvb`
- ❌ Dashboard -
`x-pack/test/functional/apps/lens/open_in_lens/dashboard/config.ts`
- Not applicable to serverless env and/or duplicate of other tests.
Closes#162346
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
Create utilities allowing to simulate a model version cohabitation
period, by creating two SO repositories of different versions so that
type/test owners can use it to assert the behavior of documents created
by one version and read by the other, or vice-versa.
### Usage example
```ts
describe('myIntegrationTest', () => {
const testbed = createModelVersionTestBed();
let testkit: ModelVersionTestKit;
beforeAll(async () => {
await testbed.startES();
});
afterAll(async () => {
await testbed.stopES();
});
beforeEach(async () => {
testkit = await testbed.prepareTestKit({
savedObjectDefinitions: [{
definition: mySoTypeDefinition,
modelVersionBefore: 1,
modelVersionAfter: 2,
}]
})
});
afterEach(async () => {
if(testkit) {
await testkit.tearsDown();
}
});
it('can be used to test model version cohabitation', async () => {
// last registered version is `1`
const repositoryV1 = testkit.repositoryBefore;
// last registered version is `2`
const repositoryV2 = testkit.repositoryAfter;
// do something with the two repositories, e.g
await repositoryV1.create(someAttrs, { id });
const v2docReadFromV1 = await repositoryV2.get('my-type', id);
expect(v2docReadFromV1.attributes).toEqual(something);
})
})
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## What this PR changes
Follow up of elastic/kibana/pull/164107/
For serverless ES/Kibana, it gates exception list API for endpoint
exceptions and restricts endpoint exceptions tab on Endpoint Security
rule details based on project PLIs. If no endpoint PLIs, endpoint
exceptions should not be accessible.
- [x] Add upselling to `app/security/exceptions/details/endpoint_list`
page
- [ ] Tests (WIP) - in a follow up PR
### How to review
Best to follow along commits for a code review. Below are details to
manually test the changes.
- Setup for _Servlerless_
- Run `yarn es serverless --kill --clean --license trial -E
xpack.security.authc.api_key.enabled=true` on a terminal window to start
ES.
- Copy `config/serverless.security.yml` to
`config/serverless.security.dev.yml`
- Run `yarn serverless-security --no-base-path` on another terminal
window to start kibana in serverless mode
- Log in using `serverless_security` user.
### Tests (Serverless)
This needs to be tested with a custom user/role and not
`elastic_serverless` which has `superuser` role.
1. ### PLI configs
`{ product_line: 'security', product_tier: 'essentials' }` or `{
product_line: 'security', product_tier: 'complete' }`
and
`{ product_line: 'endpoint', product_tier: 'essentials' }` or `{
product_line: 'endpoint', product_tier: 'complete' }`
- #### UX
1. Navigate to Rules via `http://localhost:5601/app/security/rules/`.
Click on `Add Elastic rules`.
2. Select and add `Endpoint Security` rule.
3. Click `Endpoint Security` and navigate to the rules details page, and
you should see `Endpoint exceptions` tab. The tabs visible are `Alerts`,
`Endpoint exceptions`, `Rule exceptions`, `Execution results`.
4. Navigate to Rules>Shared Exception Lists > Endpoint Security
Exception List via `app/security/exceptions/details/endpoint_list` and
you should be able to see the page with any added endpoint exceptions.
- #### API requests (with user `serverless_security`)
1. should get a status `200` on`POST api/exception_lists/items`
2. should get a status `200` on `POST
api/exception_lists/_export?id=endpoint_list&list_id=endpoint_list&namespace_type=agnostic&include_expired_exceptions=true`
3. should get a status `200` on `PUT api/exception_lists/items`
5. should get a status `200` on `DELETE api/exception_lists/items`
6. should get a status `200` on `GET
api/exception_lists/items/_find?list_id=endpoint_list&namespace_type=agnostic`
2. ### PLI configs
`{ product_line: 'security', product_tier: 'essentials' }` or `{
product_line: 'security', product_tier: 'complete' }`
- #### UX
1. Navigate to Rules via `http://localhost:5601/app/security/rules/`.
Click on `Add Elastic rules`.
2. Select and add `Endpoint Security` rule.
3. Click `Endpoint Security` and navigate to the rules details page, and
you should not see `Endpoint exceptions` tab. The only tabs visible are
`Alerts`, `Rule exceptions`, `Execution results`.
4. Navigate to Rules>Shared Exception Lists > Endpoint Security
Exception List via `app/security/exceptions/details/endpoint_list` and
you should see an upsell message.
- #### API requests
1. should get a status `403` on`POST api/exception_lists/items`
2. should get a status `403` on `POST
api/exception_lists/_export?id=endpoint_list&list_id=endpoint_list&namespace_type=agnostic&include_expired_exceptions=true`
3. should get a status `403` on `PUT api/exception_lists/items`
6. should get a status `403` on `DELETE api/exception_lists/items`
7. should get a status `403` on `GET
api/exception_lists/items/_find?list_id=endpoint_list&namespace_type=agnostic`
---
**Flaky FTRs**
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3248https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3255
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR follows https://github.com/elastic/kibana/pull/167466, after
deploying the GPCTL in production. After separating the logs from
nonprod and prod GPCTL, we have to go to 2 different dashboard based on
the environment we would like to check (non-prod/prod)
## Summary
Re-enable a subset of cypress serverless tests:
- Restructured
`x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_creation/custom_query_rule.cy.ts`
to break it down into and enable the following tests:
- `rule_creation/common_components.cy.ts`
- `rule_creation/custom_query_rule.cy.ts`
- `rule_details/common_components.cy.ts`
- `rule_details/custom_query_rule.cy.ts`
- `rule_edit/custom_query_rule.cy.ts`
## Summary
Close https://github.com/elastic/kibana/issues/159691
[Requirements](https://docs.google.com/document/d/1uXgyDIGuIqkYXmavdMTpBgQEiOP07ObJYb7GNq5GiSE/edit#heading=h.okl11rz12ytg)
[A/B test
description](https://docs.google.com/document/d/1yzfZF8mtlRNH4X6HjosD6Exh24zAhN__LR_zh3DOzKw/edit?usp=sharing)
[Figma](https://www.figma.com/file/WGhmfgyy9FBOltLtycfGPE/Getting-started?type=design&node-id=92-44804&mode=design&t=mwbIexn5Fs754HQz-0)
Testing - see _testing_ section below for more details
(https://dosant-pr-167374-d-2023-06-14-global-drift-with-experiment.kbndev.co/.
elastic/changeme)
This PR enables cloud chat (Drift) globally. This is done by adding a
custom chat button in the Kibana header which manually toggles Drift
widget. We attempt to manually position the widget to the top of the
screen so it pops up close to the chat button that triggered it.
Previously Drift chat was available only on specific pages like
Solutions onboarding pages, integrations, setup guides as a regular chat
widget with the floating chat bubble in the bottom right corner. We
couldn't enable it on all pages, because on a lot of them the floating
chat bottom would have overlapped the application UI.
We also were asked to add [an a/b
test](https://docs.google.com/document/d/1yzfZF8mtlRNH4X6HjosD6Exh24zAhN__LR_zh3DOzKw/edit?usp=sharing):
- A: The chat button appears in the header for all pages (new)
- B: The chat button appears as floating action button in the
bottom-right corner on pages where Drift was previously available
(solutions onboarding pages, integrations, setup guides)
### Screenshots / Videos
#### Global Chat in the header
#### The tour on the first appearance
#### (Part of A/B test) Drift in the header on new pages and as floating
action button on old pages
0386ccbd-ab6c-4eb2-a57b-f9324fcf73eb
### Implementation notes
- **We still enable Drift only for trial users + gap window**
- We exposed additional APIs from drift iframe to manually control its
visibility and react to more events
https://github.com/elastic/cloud/pull/118761. This changes are required
for the code in this PR to work. ~The updated frame code wasn't deployed
yet.~ the changes were deployed
- We use [`playbookFired` event
](https://devdocs.drift.com/docs/drift-events#playbook-fired) to know if
Drift chat should be visible for the current user. We show the button in
the header only when it fires.
- To react to the event and to display the button, we have to kick of
Drift iframe initialization first
- This means Drift codes loads before we show the button and before user
interacts with it (Only when Drift is enabled, meaning, only for trial
users + gap window)
- Subsequent launches or opens of the same playbook will not re-trigger
the `playbookFired` event, I used local storage flag to workaround this
and show the live chat button, but it also has it's own edge case. As an
alternative we can always show the chat button and don't rely on the
playbook event, more details here:
https://docs.google.com/document/d/1j313mVOIz19Rkoj8TDFWaLc7Pgk_ByBBKJFNIC-jbyc/edit?usp=sharing.
For now was decided to rely on the `playbookFired` event.
- A/B: **to support for both new and old implementation, I had to
refactor the old one from drop-in chat to global chat that is controlled
by list of hardcoded URLs.** This is not ideal, but this allows two
implementations to co-exist with much tech-debt and we plan to get rid
of this after the a/b test.
- When we navigate between pages with different implementations, Drift
re-initializes itself (just like in old implementation), this
performance debt should go away when we get rid of the a/b test.
- When end-to-end testing the a/b experiment, I found a bug in the a/b
test setup in Kibana https://github.com/elastic/kibana/issues/167240
this needs to be addressed separately for a/b test to work properly.
- When the user receives a message from Drift - the custom "Live Chat"
button doesn't indicate that there is a new message. This needs a follow
up, @Dosant to create an issue
### Testing
Version with the experiment where on old pages Drift appears as a
floating chat bubble -
https://dosant-pr-167374-d-2023-06-14-global-drift-with-experiment.kbndev.co/
elastic/changeme
> [!NOTE]
> If the live chat button doesn't appear, it is likely because the
playbook was recently activated by someone else. you can workaround this
for testing by creating a different user or by setting
`cloudChatPlaybookFiredOnce : true` to localstorage. This issue and
mitigation is described in details in the "implementation details"
section
#### To test locally:
```
xpack.cloud.id: 'some-id'
xpack.cloud.trial_end_date: '2023-09-21T00:00:00.000Z'
xpack.cloud_integrations.chat.trialBuffer: 45
xpack.cloud.chat.enabled: true
xpack.cloud.chatIdentitySecret: <pls react out>
xpack.cloud.chat.chatURL: https://elasticcloud-production-chat-us-east-1.s3.amazonaws.com/drift-iframe.html
xpack.cloud_integrations.experiments.flag_overrides:
"cloud-chat.chat-variant": "bubble" or "header"
```
## Summary
This PR adds a Settings application component for rendering the Advanced
Settings page in serverless.
### How to test:
1. Start Es with `yarn es serverless` and Kibana with `yarn
serverless-{es/oblt/security}`
2. Go to Management -> Advanced Settings
3. Verify that the settings can be changed and saved.
### Advanced Settings page:
<img width="1495" alt="Screenshot 2023-09-28 at 20 56 25"
src="374b3bbd-7bf6-4de7-8129-8b293dd1698e">
### Added an Advanced Settings card to the Management landing page:
<img width="1575" alt="Screenshot 2023-09-28 at 12 24 23"
src="c08b8b36-ff40-4772-87d6-597629d78342">
<!--- ### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
---------
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
## Summary
This PR wraps up the work the @elastic/kibana-presentation team has done
to finish the MVP of [Phase
1](https://github.com/elastic/kibana/issues/154354) of the `Link`
embeddable, which enables users to add panels to their dashboard that
contain links to other dashboards + external links - with respect to
dashboard links, we give the author control over which pieces of context
should be kept across dashboards so that things like filter pills,
queries, and time ranges are not lost. This marks a huge improvement in
dashboard navigation overall, which was previously only available via a
variety of different workarounds including (but not limited to):
- Creating (essentially) a `noop` dashboard-to-dashboard drilldown
- Using markdown panels with hard Dashboard links, which are prone to
break across updates
- Avoiding navigation all together, which resulted in large,
slow-to-load dashboards.
As an added benefit, because these panels contain **references** to each
dashboard rather than hard links, (1) unlike markdown links, they should
not break after updates and (2) if a links panel is exported and
imported into another space or instance, all of the dashboards it links
to will also be imported.
1a86b713-47e7-4db9-8a04-29d41b13681a
> **Note**
> 🔉 The above video has audio! Turn on your sound for the best
experience.
### Note about this PR
- A majority of this work was done on a feature branch, with thorough
reviews from @andreadelrio on behalf of @elastic/kibana-design along the
way. Therefore, while feedback on the design is encouraged, any large
concerns brought up in this PR should be filed as separate issues and
addressed in follow-up PRs.
- This PR contains work for giving embeddables control over their own
panel size / default positioning on the dashboard. This was especially
important for the links panel, since we assume that (a) most links
panels would be located somewhere near the top of the dashboard and (b)
the horizontal links panel should have a different default "shape"
(longer than it is tall) than the vertical panel (taller than it is
long).
- This PR also contains work for caching dashboard saved objects, which
makes navigation much more seamless.
### Flaky Test Runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3251
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
~[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials~ This will
be addressed in a follow up:
https://github.com/elastic/kibana/issues/166750
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios - ~Units tests
are added, functional tests are forthcoming~ Edit: All tests are in.
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Nick Peihl <nick.peihl@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Andrea Del Rio <delrio.andre@gmail.com>
Co-authored-by: Devon Thomson <devon.thomson@elastic.co>
Co-authored-by: Nick Peihl <nickpeihl@gmail.com>
Co-authored-by: Gerard Soldevila <gerard.soldevila@elastic.co>
