Closes#161758
## Summary
In this PR, I am saving the groupings information for the new threshold
in AAD in a similar format as the security team does, you can check the
format in the following screenshots. (Please check this
[RFC](https://docs.google.com/document/d/1DlykydM8Hk7-VAPOcuoUXp0L_qSi2jCZabJkPdO44tQ/edit#heading=h.2b1v1tr0ep8m)
for more information)
### Alert as data document
### Groupings action variable
### Alert table
It is also possible to search based on these new variables:
f07b39c2-52e8-4f50-b713-577da7ab1c42
## Summary
Resolves: https://github.com/elastic/kibana/issues/166301
Adds support for solution/category filtering to maintenance windows by
adding a new property: `category_ids`. Selecting one or more solutions
when creating/updating a maintenance window will cause the maintenance
window to only suppress rule types belonging to said solutions. In order
to achieve filtering by solution/category, we are adding a new field to
the rule types schema called `category`. This field should map to the
feature category that the rule type belongs to (`observability`,
`securitySolution` or `management`).
Our initial plan was to use feature IDs or rule type IDs to accomplish
this filtering, we decided against using rule type IDs because if a new
rule type gets added, we need to change the API to support this new rule
type. We decided against feature IDs because it's a very anti-serverless
way of accomplishing this feature, as we don't want to expose feature
IDs to APIs. We decided on app categories because it works well with
serverless and should be much easier to maintain if new rule types are
added in the future.
This means the `rule_types` API has to be changed to include this new
field, although it shouldn't be a breaking change since we're just
adding a new field. No migrations are needed since rule types are in
memory and maintenance windows are backwards compatible.
### Error state:
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dima Arnautov <arnautov.dima@gmail.com>
## Summary
Close https://github.com/elastic/kibana/issues/159691
[Requirements](https://docs.google.com/document/d/1uXgyDIGuIqkYXmavdMTpBgQEiOP07ObJYb7GNq5GiSE/edit#heading=h.okl11rz12ytg)
[A/B test
description](https://docs.google.com/document/d/1yzfZF8mtlRNH4X6HjosD6Exh24zAhN__LR_zh3DOzKw/edit?usp=sharing)
[Figma](https://www.figma.com/file/WGhmfgyy9FBOltLtycfGPE/Getting-started?type=design&node-id=92-44804&mode=design&t=mwbIexn5Fs754HQz-0)
Testing - see _testing_ section below for more details
(https://dosant-pr-167374-d-2023-06-14-global-drift-with-experiment.kbndev.co/.
elastic/changeme)
This PR enables cloud chat (Drift) globally. This is done by adding a
custom chat button in the Kibana header which manually toggles Drift
widget. We attempt to manually position the widget to the top of the
screen so it pops up close to the chat button that triggered it.
Previously Drift chat was available only on specific pages like
Solutions onboarding pages, integrations, setup guides as a regular chat
widget with the floating chat bubble in the bottom right corner. We
couldn't enable it on all pages, because on a lot of them the floating
chat bottom would have overlapped the application UI.
We also were asked to add [an a/b
test](https://docs.google.com/document/d/1yzfZF8mtlRNH4X6HjosD6Exh24zAhN__LR_zh3DOzKw/edit?usp=sharing):
- A: The chat button appears in the header for all pages (new)
- B: The chat button appears as floating action button in the
bottom-right corner on pages where Drift was previously available
(solutions onboarding pages, integrations, setup guides)
### Screenshots / Videos
#### Global Chat in the header
#### The tour on the first appearance
#### (Part of A/B test) Drift in the header on new pages and as floating
action button on old pages
0386ccbd-ab6c-4eb2-a57b-f9324fcf73eb
### Implementation notes
- **We still enable Drift only for trial users + gap window**
- We exposed additional APIs from drift iframe to manually control its
visibility and react to more events
https://github.com/elastic/cloud/pull/118761. This changes are required
for the code in this PR to work. ~The updated frame code wasn't deployed
yet.~ the changes were deployed
- We use [`playbookFired` event
](https://devdocs.drift.com/docs/drift-events#playbook-fired) to know if
Drift chat should be visible for the current user. We show the button in
the header only when it fires.
- To react to the event and to display the button, we have to kick of
Drift iframe initialization first
- This means Drift codes loads before we show the button and before user
interacts with it (Only when Drift is enabled, meaning, only for trial
users + gap window)
- Subsequent launches or opens of the same playbook will not re-trigger
the `playbookFired` event, I used local storage flag to workaround this
and show the live chat button, but it also has it's own edge case. As an
alternative we can always show the chat button and don't rely on the
playbook event, more details here:
https://docs.google.com/document/d/1j313mVOIz19Rkoj8TDFWaLc7Pgk_ByBBKJFNIC-jbyc/edit?usp=sharing.
For now was decided to rely on the `playbookFired` event.
- A/B: **to support for both new and old implementation, I had to
refactor the old one from drop-in chat to global chat that is controlled
by list of hardcoded URLs.** This is not ideal, but this allows two
implementations to co-exist with much tech-debt and we plan to get rid
of this after the a/b test.
- When we navigate between pages with different implementations, Drift
re-initializes itself (just like in old implementation), this
performance debt should go away when we get rid of the a/b test.
- When end-to-end testing the a/b experiment, I found a bug in the a/b
test setup in Kibana https://github.com/elastic/kibana/issues/167240
this needs to be addressed separately for a/b test to work properly.
- When the user receives a message from Drift - the custom "Live Chat"
button doesn't indicate that there is a new message. This needs a follow
up, @Dosant to create an issue
### Testing
Version with the experiment where on old pages Drift appears as a
floating chat bubble -
https://dosant-pr-167374-d-2023-06-14-global-drift-with-experiment.kbndev.co/
elastic/changeme
> [!NOTE]
> If the live chat button doesn't appear, it is likely because the
playbook was recently activated by someone else. you can workaround this
for testing by creating a different user or by setting
`cloudChatPlaybookFiredOnce : true` to localstorage. This issue and
mitigation is described in details in the "implementation details"
section
#### To test locally:
```
xpack.cloud.id: 'some-id'
xpack.cloud.trial_end_date: '2023-09-21T00:00:00.000Z'
xpack.cloud_integrations.chat.trialBuffer: 45
xpack.cloud.chat.enabled: true
xpack.cloud.chatIdentitySecret: <pls react out>
xpack.cloud.chat.chatURL: https://elasticcloud-production-chat-us-east-1.s3.amazonaws.com/drift-iframe.html
xpack.cloud_integrations.experiments.flag_overrides:
"cloud-chat.chat-variant": "bubble" or "header"
```
A follow up for https://github.com/elastic/kibana/pull/165866
## Summary
This PR replaces icons for the sidebar toggle button.
<img width="200" alt="Screenshot 2023-09-29 at 13 00 39"
src="6ed10562-9a50-48ce-b6b2-030ab7b11e11">
<img width="200" alt="Screenshot 2023-09-29 at 13 00 47"
src="e286e152-a6bb-4b21-a97a-44419757dafb">
## Summary
This PR adds a Settings application component for rendering the Advanced
Settings page in serverless.
### How to test:
1. Start Es with `yarn es serverless` and Kibana with `yarn
serverless-{es/oblt/security}`
2. Go to Management -> Advanced Settings
3. Verify that the settings can be changed and saved.
### Advanced Settings page:
<img width="1495" alt="Screenshot 2023-09-28 at 20 56 25"
src="374b3bbd-7bf6-4de7-8129-8b293dd1698e">
### Added an Advanced Settings card to the Management landing page:
<img width="1575" alt="Screenshot 2023-09-28 at 12 24 23"
src="c08b8b36-ff40-4772-87d6-597629d78342">
<!--- ### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
---------
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Clint Andrew Hall <clint.hall@elastic.co>
## Summary
This PR wraps up the work the @elastic/kibana-presentation team has done
to finish the MVP of [Phase
1](https://github.com/elastic/kibana/issues/154354) of the `Link`
embeddable, which enables users to add panels to their dashboard that
contain links to other dashboards + external links - with respect to
dashboard links, we give the author control over which pieces of context
should be kept across dashboards so that things like filter pills,
queries, and time ranges are not lost. This marks a huge improvement in
dashboard navigation overall, which was previously only available via a
variety of different workarounds including (but not limited to):
- Creating (essentially) a `noop` dashboard-to-dashboard drilldown
- Using markdown panels with hard Dashboard links, which are prone to
break across updates
- Avoiding navigation all together, which resulted in large,
slow-to-load dashboards.
As an added benefit, because these panels contain **references** to each
dashboard rather than hard links, (1) unlike markdown links, they should
not break after updates and (2) if a links panel is exported and
imported into another space or instance, all of the dashboards it links
to will also be imported.
1a86b713-47e7-4db9-8a04-29d41b13681a
> **Note**
> 🔉 The above video has audio! Turn on your sound for the best
experience.
### Note about this PR
- A majority of this work was done on a feature branch, with thorough
reviews from @andreadelrio on behalf of @elastic/kibana-design along the
way. Therefore, while feedback on the design is encouraged, any large
concerns brought up in this PR should be filed as separate issues and
addressed in follow-up PRs.
- This PR contains work for giving embeddables control over their own
panel size / default positioning on the dashboard. This was especially
important for the links panel, since we assume that (a) most links
panels would be located somewhere near the top of the dashboard and (b)
the horizontal links panel should have a different default "shape"
(longer than it is tall) than the vertical panel (taller than it is
long).
- This PR also contains work for caching dashboard saved objects, which
makes navigation much more seamless.
### Flaky Test Runner
-
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3251
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
~[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials~ This will
be addressed in a follow up:
https://github.com/elastic/kibana/issues/166750
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios - ~Units tests
are added, functional tests are forthcoming~ Edit: All tests are in.
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Nick Peihl <nick.peihl@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Andrea Del Rio <delrio.andre@gmail.com>
Co-authored-by: Devon Thomson <devon.thomson@elastic.co>
Co-authored-by: Nick Peihl <nickpeihl@gmail.com>
Co-authored-by: Gerard Soldevila <gerard.soldevila@elastic.co>
## Summary
Add the `use_text_extraction_service` rich configurable field to some
native connectors.
Native connectors will not have access to the feature, but the field is
required in case the user converts their native connector to a
self-managed connector.
## Summary
Let's automate E2E against Serverless
Changelog:
- updated certs to include additional dns names we are using for testing
locally, `host.docker.internal`, `es01`
- updated certs generation README to include changes related to
`openssl@3`
- added new certs for Fleet server
- added fleet-server service token
- added support for `ca_trusted_fingerprint` in fleet preconfig
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tomasz Ciecierski <ciecierskitomek@gmail.com>
Co-authored-by: Tomasz Ciecierski <tomasz.ciecierski@elastic.co>
Co-authored-by: Kevin Logan <kevin.logan@elastic.co>
## Summary
Updates the exceptions flyout UI `match_any` operator to accept numerous
duplicate values that differ in case. Prior to this change, a user could
not add a field value of `foo` and `FOO` - the UI would display that the
value is a duplicate. We now will allow this as exceptions are case
sensitive and this is a necessary use case for the current exceptions
behavior.
Cypress tests and FTR tests are added.
Closes#167075
## Summary
Adds a public asset client available in the `setup` lifecycle hook for
plugins that depend on this one. `getHosts` is the only method available
on this client for now.
TODO, before merge:
- [x] Add docs for the server client
- [x] Add docs for the public client
- [x] Remove REST docs from plugin docs, not needed
- [x] Add unit tests for public client
### Testing this PR
One way of testing this new client is to apply the attached
test-assets.patch file locally, adjust the date range in the getHosts
query that is added in the infra plugin, and then start Kibana and
navigate to the infra app. You should see print out in the browser
console.
[test-assets.patch](12718693/test-assets.patch)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR is next step completing discover timeline integration. All
previous/nest steps have been defined here:
https://github.com/elastic/security-team/issues/6677
## Summary
This PR implements the integration between timeline State v/s Discover
State. The purpose of this PR is to add functionality related to the
persistence of saved search which will always be linked to the timeline
user is working in.
Below diagram shows briefly how saved search is working with timeline.
```mermaid
graph TD;
DS(Discover State) -. user updates .-> SS(Saved Search);
SS(Saved Search) -. updates savedSearchId .-> TS(Timeline State) ;
TS(Timeline State) -. restores Saved Search to App state .->DS(Discover State);
```
Primarily, this PR implements below technical components:
1. `DiscoverInTimleineContext` : provides the ability across security
solution to manipulate discover state.
2. `useDiscoverInTimelineActions`: acts as a helper to provide
short-hand actions to manipulate discover state. For eg.
`resetDiscoverAppState` or `restoreAppStateFromSavedSearch`.
Here is the small demo video:
006465ba-19ce-4209-ac46-21dbb746508d
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Michael Olorunnisola <michael.olorunnisola@elastic.co>
## Summary
This PR follows #166460 by adding Category panels to the Form.
<img width="1807" alt="Screenshot 2023-09-27 at 3 36 16 PM"
src="2abe8cf5-5822-473f-affd-148fb7949316">
## Notes
This PR is divided into several commits, the first few being
prerequisite codemods. I recommend reviewing each commit separately, as
the codemods might obscure the actual component work.
- [e78586f - Make SettingType pre-defined to clean up
references](e78586fe44)
- This makes the `SettingType` optional, to clean up areas where the
generic need not be specific.
- [80a9988 - [codemod] Make onFieldChange and onInputChange more
distinct](80a9988516)
- The `onChange` handlers weren't very clear as you work your way up the
component tree. This makes the implementation and usage easier to
understand, (and easier to [replace with state
management](https://github.com/elastic/kibana/issues/166579)).
- [5d0beff - [fix] Fix logged errors in form
tests](5d0beff00c)
- This fixes some logged errors in the Form from `Monaco` and from some
missing `act` and `waitFor` calls.
## Summary
Close https://github.com/elastic/kibana/issues/167152
Log a warning instead of throwing an error in
`saved_object_content_storage` when response validation failed.
We decided to do this as a precaution and as a follow up to an issue
found in saved search https://github.com/elastic/kibana/pull/166886
where storage started failing because of too strict validation.
As of this PR the saved_object_content_storage covers and this change
cover:
- `search`
- `index_pattern`
- `dashboard`
- `lens`
- `maps`
For other types we agreed with @dej611 that instead of applying the same
change for other types (visualization, graph, annotation) the team would
look into migrating their types to also use
`saved_object_content_storage`
https://github.com/elastic/kibana/issues/167421
## Summary
This PR introduces the new color mapping feature into Lens.
The color mapping feature is introduced as a standalone sharable
component available from `@kbn/coloring`. The
[README.md](ddd216457d/packages/kbn-coloring/src/shared_components/color_mapping/README.md)
file describes the components and the logic behind it.
The Color Mapping component is also connected to Lens and is available
in the following charts:
- XY (you can specify the mappings from a breakdown dimension
- Partition (you can specify the mappings from the main slice/group by
dimension)
- Tag cloud (you can specify the mappings from the tags dimension)
This MVP feature will be released under the Tech Preview flag.
This PR needs to prove the user experience and the ease of use. UI
styles, design improvements and embellishments will be released in
subsequent PRs.
The current MVP-provided palettes are just a placeholder. I'm
coordinating with @gvnmagni for a final set of palettes.
close https://github.com/elastic/kibana/issues/155037
close https://github.com/elastic/kibana/issues/6480
fix https://github.com/elastic/kibana/issues/28618
fix https://github.com/elastic/kibana/issues/96044
fix https://github.com/elastic/kibana/issues/101942
fix https://github.com/elastic/kibana/issues/112839
fix https://github.com/elastic/kibana/issues/116634
## Release note
This feature introduces the ability to change and map colors to break
down dimensions in Lens. The feature provides an improved way to specify
colors and their association with categories by giving the user a
predefined set of color choices
or customized one that drives the user toward a correct color selection.
It provides ways to pick new colors and generate gradients.
This feature is in Tech Preview and is enabled by default on every new
visualization but can be turned off at will.
## Summary
I had to change `waitForRender` since `page.waitForFunction` tries to
run a script on page and it is not working due to CSP settings on Cloud.
Instead of injecting a script, we use a classical API to find
elements/attributes in the DOM.
Since `PUT /internal/core/_settings` is merged in 8.11.0, journeys run
on Cloud with on-fly labels update is supported starting deployments
8.11.0+. I added error message for 404 code just in case someone runs it
on earlier version.
`many_fields_discover` journey was update since on Cloud the data view
used by scenario is not selected by default.
How it works:
Create a deployment with QAF and re-configure it for journey run:
```
export EC_DEPLOYMENT_NAME=my-run-8.11
qaf elastic-cloud deployments create --stack-version 8.11.0-SNAPSHOT --environment staging --region gcp-us-central1
qaf elastic-cloud deployments configure-for-performance-journeys
```
Run any journey, e.g. many_fields_discover
```
TEST_CLOUD=1 TEST_ES_URL=https://username:pswd@es_url:443 TEST_KIBANA_URL=https://username:pswd@kibana-ur_url node scripts/functional_test_runner --config x-pack/performance/journeys/many_fields_discover.ts
```
You should see a log about labels being updated:
```
Updating telemetry & APM labels: {"testJobId":"local-a3272047-6724-44d1-9a61-5c79781b06a1","testBuildId":"local-d8edbace-f441-4ba9-ac83-5909be3acf2a","journeyName":"many_fields_discover","ftrConfig":"x-pack/performance/journeys/many_fields_discover.ts"}
```
And then able to find APM logs for the journey in
[Ops](https://kibana-ops-e2e-perf.kb.us-central1.gcp.cloud.es.io:9243/app/apm/services?comparisonEnabled=true&environment=ENVIRONMENT_ALL&kuery=labels.testJobId%20%3A%20%22local-d79a878c-cc7a-423b-b884-c9b6b1a8d781%22&latencyAggregationType=avg&offset=1d&rangeFrom=now-24h%2Fh&rangeTo=now&serviceGroup=&transactionType=request)
cluster
## Summary
### `@kbn/es` package
- Introduces `--resources` option to the `es serverless` command
- Allows for
[`serverless_resources`](https://github.com/elastic/kibana/tree/main/packages/kbn-es/src/serverless_resources)
to be overwritten
### Security Solution Plugin
- Added customized `users`, `users_roles` and `roles.yml` files that
includes:
- Updated `roles.yml` file (sync'ed with project controller version)
- Updated `users` / `users_roles` file that includes one one per
security project role (name the same as the role)
- New CLI script - `node
x-pack/plugins/security_solution/scripts/endpoint/start_es_serverless_with_security_users.js`
- that will start ES in serverless mode and inject the customized
`--resources` into the command
- Any existing option that can be passed to `es` or `es serverless` can
still continue to be passed to this new script. I will pass them along.
closes#167092
__
## Summary
Save to library action should not be present in ES|QL panels. For now we
only allow by value embeddables and we don't want them to be
edited/created in Lens editor
<img width="915" alt="image"
src="d6e5e8a7-459f-4b1d-8461-2863b45db882">
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
When a user is writing am ES|QL query with the enrich command and
doesn't have a policy then we want to navigate them from the editor to
the index management page to create one.
In the future the same action can open a flyout in order for the users
to not change context but we don't have it atm.
---------
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
## Summary
We're breaking https://github.com/elastic/kibana/pull/166813 up into
smaller PRs in the interest of getting PRs through sooner for type
fixes. These are the changes for AppEx SharedUX.
## Summary
This PR adds data drift detection workflow from Trained models to Data
comparison view. It also renames Data comparison to Data Drift.
**From the new map view in Trained model list:**
- Clicking on the index icon in the map view will give an option/action
to Analyze data drift
a68163ab-8a83-4378-8cf3-ea49f4480a06
- If model has detected related indices, it will also give an option to
Analyze data drift in the Transform actions
**From the data comparison/drift page:**
- Default screen with list of available data views and saved search will
be shown
<img width="1470" alt="Screen Shot 2023-09-07 at 00 22 01"
src="db13b8b7-9d90-4220-b03e-9f9d12ab53e9">
- But can also customize index patterns for the data sets to analyze.
Upon 'analyzing', a new data view will be created if needed (either
permanently or temporarily).
<img width="1271" alt="Screen Shot 2023-08-29 at 16 56 57"
src="e000e920-162b-4369-8762-70b6244e50e7">
<img width="1470" alt="Screen Shot 2023-09-07 at 00 22 49"
src="6577a530-c3b0-4ab9-95e4-d1d8fd1c9f0a">
- If there exists a data view with exact combination of index patterns
and time field, it will use that data view
- If there exists a data view with the same index patterns but different
time field, it will create a new data view with name
`{referencePattern},{comparisonPattern}-{timeField}`
- If no data view exists that matches, it will create a new data view
with name `{referencePattern},{comparisonPattern}`
## For reviewers:
- **appex-sharedux**: [Small change in the exported type interface for
BaseSavedObjectFinder](https://github.com/elastic/kibana/pull/162853/files#diff-5e2e62df8aba5ac9445962bfa00eee933a386110d0a24dfe6ac0f300a796ccc3)
to correctly list `children` as an accepted prop. This prop which is
used for the `toolsRight`.
- **security-solution**: Renaming of `Data comparison` to `Data Drift`
## Tests:
[Flaky test suite runner with Data Drift
test](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3216#018accc2-d33b-4cd6-a178-589e6698b675)
... successful after 50 runs✅
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR fixes the errors of the type checkers on some of the Management
team's plugins.
The errors were identified by running the command `node
scripts/type_check --project <path-to-tsconfig.json>` with these files
as suggested by the Operations team:
- ./packages/kbn-generate-console-definitions/tsconfig.json
- ./src/plugins/console/tsconfig.json
- ./packages/kbn-management/settings/components/field_row/tsconfig.json
## Summary
Address #166459
This PR scaffolds telemetry into the notification service, to enable
reporting when deduped toasts are dismissed. When a deduped toast is
dismissed; the message recurrence count and the toast message would be
emitted.
Telemetry Event Definition;
```typescript
{
eventType: "global_toast_list_toast_dismissed",
schema: {
"toast_deduplication_count": {
type: "long",
_meta: {
description: "toast message text"
}
},
"toast_message": {
type: "keyword",
_meta: {
description: "recurrence count for particular toast message"
}
},
"toast_message_type": {
type: "keyword",
_meta: {
description: "toast message type, accepted values are warning, danger, primary"
}
}
}
}
```
Testing;
- Ensure you have an existing dashboard, if you don't installing any
sample data would bootstrap a dashboard automatically
- Search for `visualize library` from the global search bar, and
navigate to visualize library.
- Attempt to create a visualization, select the `TSVB` visualization
type. At this step there typically wouldn't be an error
- We can simulate an error, by open Dev tools and blocking the URL
`/internal/metrics/vis/data` like so <img width="604" alt="Screenshot
2023-09-21 at 11 49 41"
src="ad320569-33b2-4335-8052-981d1761ea67">
on doing this, we then attempt refreshing the query, we would then be
presented with an error toast, similar to the screenshot below;
<img width="482" alt="Screenshot 2023-09-21 at 11 52 51"
src="fef07e72-625b-4457-abc3-9214d64f9e48">
click the refresh query button as much as you like to increase it's
recurrence count.
- From here on we verify that; clicking the dismiss button sends a
telemetry event, also not interacting with the toast and it's display
lifetime elapsing with the toast clearing itself, the telemetry gets
sent still. In either case there should be a request calling the
resource `kibana_browser` with a payload containing the telemetry event
definition above.
### Checklist
<!-- Delete any items that are not applicable to this PR. -->
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
<!-- - [ ] Any UI touched in this PR is usable by keyboard only (learn
more about [keyboard
accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers) ->
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Addresses https://github.com/elastic/kibana/issues/160411
## Summary
This PR adds a package that contains a form component for the Advanced
Settings UI in serverless.
This implementation was extracted from the the `Form` component in the
`advancedSettings` plugin, excluding some functionalities:
- The form doesn't support search queries.
- The form doesn't divide the settings into categories.
### Testing
The form can be tested in the Storybook Preview from the CI build. Some
things to be tested:
- Making changes to any of the fields displays the bottom bar.
- Clicking the Cancel button clears the changes.
- Clicking the Save button triggers a `saveChanges` action with the
correct changes.
- The bottom bar correctly shows the number of unsaved settings.
- Toggling the `isSavingEnabled` control to `false` disables all fields.
- Toggling the `requirePageReload` control to `true` causes saving of
changes to any of the fields to trigger a `showReloadPagePrompt` action.
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
<!--
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Clint Andrew Hall <clint@clintandrewhall.com>
## Summary
This PR is the core part of #166813. The original work seems to grow
large, and we'd like to enable a preventive check beforehand to prevent
more errors from entering the codebase.
The idea is to have a selective type check that would only check changed
files' projects.
- [x] when there's no extra label, run the selective type check only on
the diffing files' projects (success:
https://buildkite.com/elastic/kibana-pull-request/builds/161837)
- [x] when the label `ci:hard-typecheck` is present, run the regular
(but now, working) full typecheck (expected to fail: )
cc: @watson
---------
Co-authored-by: Brad White <brad.white@elastic.co>
Co-authored-by: Thomas Watson <w@tson.dk>
Co-authored-by: Thomas Watson <watson@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Fixes a long list of julian's UI bugs. Tested on both stateful and
serverless. See videos on visual fixes.
1a450bf6-7477-40a4-a020-a5172b56ef4c
92b40ecd-d888-4fd6-af91-045e81a1843f
Things to note:
- I had to adjust the asset path here as locally on main the images were
broken (the header for example).
## Summary
Closes https://github.com/elastic/kibana/issues/166874
Hides indices starting with . (and considered as system from the
autocomplete)
<img width="785" alt="image"
src="9c4cce79-c844-41b6-a30e-06dad49f7c52">
Followed the exact pattern that the dataview management page is using.
## Summary
Prepares the serverless FTR tests to be runnable with a custom ES image.
(`--esServerlessImage` cli arg)
Creates a pipeline for testing and promoting ES Serverless docker
releases.
The job can be triggered here:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote
The three main env variables it takes:
- BUILDKITE_BRANCH: the kibana branch to test with (maybe not as
important)
- BUILDKITE_COMMIT: the kibana commit to test with
- ES_SERVERLESS_IMAGE: the elasticsearch serverless image, or tag to use
from this repo:
`docker.elastic.co/elasticsearch-ci/elasticsearch-serverless`
## TODOS:
- [x] set `latest_verified` with full img path as default
- [x] ~~find other CLIs that might need the `esServerlessImage` argument
(if the docker runner has multiple usages)~~ | I confused the `yarn es
docker` with this, because I thought we only run ES serverless in a
docker container, but `elasticsearch` can also be run in docker.
- [x] set `latest-compatible` or similar flag in a manifest in gcs for
Elastic's use-case
- [ ] ensure we can only verify "forward" (ie.: to avoid a
parameterization on old versions to set our pointers back) [on a second
thought, this might be kept as a feature to roll back (if we should ever
need that)]
There are two confusing things I couldn't sort out just yet:
#### Ambiguity in --esServerlessImage
We can either have 2 CLI args: one for an image tag, one for an image
repo/image url, or we can have one (like I have it now) and interpret
that in the code, it can be either the image url, or the tag. It's more
flexible, but it's two things in one. Is it ok this way, or is it too
confusing?
e.g.:
```
node scripts/functional_tests --esFrom serverless --esServerlessImage docker.elastic.co/elasticsearch-ci/elasticsearch-serverless:git-8fc8f941bd4d --bail --config x-pack/test_serverless/functional/test_suites/security/config.ts
# or
node scripts/functional_tests --esFrom serverless --esServerlessImage latest --bail --config x-pack/test_serverless/functional/test_suites/security/config.ts
```
#### Ambiguity in the default image path
The published ES Serverless images will sit on this image path:
`docker.elastic.co/elasticsearch-ci/elasticsearch-serverless`, however,
our one exception is the `latest-verified` which we will be tagging
under a different path, where we have write rights:
`docker.elastic.co/kibana-ci/elasticsearch-serverless:latest-verified`.
Is it okay, that by default, we're searching in the `elasticsearch-ci`
images for any tags as parameters (after all, all the new images will be
published there), however our grand default will ultimately be
`docker.elastic.co/kibana-ci/elasticsearch-serverless:latest-verified`.
## Links
Buildkite:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote
eg.:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote/builds/24
Closes: https://github.com/elastic/kibana/issues/162931
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
This PR removes the Profiling dependency from APM, introduced on `8.10`.
- Exposes a new service in profiling-data-access plugin
- Create a new APM API that calls the new service and checks if
Profiling is initialized
- Move Locators from the Profiling plugin to the Observability-shared
plugin
- Move logic to check Profiling status (has_setup/has_data...) from
Profiling server to profiling-data-access plugin
- Create API tests, testing the status services based on different
scenarios:
- When profiling hasn't been initialized and there's no data
- When profiling is initialized but has no data
- When collector integration is not installed
- When symbolized integration is not installed
- When APM server integration is not found
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
