related to: https://github.com/elastic/kibana/pull/179257
Backport failed, so I'm manually applying the changes on 8.13
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR replaces kbn/ecs package with the official ecs typescript defs
available on npm https://www.npmjs.com/package/@elastic/ecs
The idea here is to be able to update and maintain ecs type defs
independently, and in a transparent way.
ECS version used in kibana currently is 8.10.0, and after this PR is
merged I will generate current version typings & file another PR to
update the kibana dependency,
The `alpha` suffix will be removed once this new flow of ecs management
stabilizes.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
closes https://github.com/elastic/kibana/issues/170247
## 📝 Summary
This PR introduces a new state machine for controlling the new Dataset
Quality Summary Panel.
As part of this work, we had to introduce a new endpoint to fetch and
calculate the Estimated Data in last 24h.
## 💡For Reviewers
### State Machine
The new state machine introduces 3 parallel states to fetch the values
displayed in the summary panel.
In case of failures in any of them, a retry mechanism is introduced to
try the fetch 1 more time after 5 seconds interval.
If the fetch fails again, we display an error toast notification.
### New Endpoint
A new endpoint `GET
/internal/dataset_quality/data_streams/estimated_data` has been
introduced to calculate the Estimated Data in last 24h.
The endpoint first retrieves the doc count and total size in bytes for
`logs-*` and uses them to calculate the average size per doc which is
then multiplied by the number of total doc in the last 24h to get an
estimate of data in last 24h.
## ✅ Testing
1) Navigate to /app/observability-logs-explorer/dataset-quality
2) The summary panel is displayed at the top of the table
3) Filterations shouldn't affect the data displayed as the panel is
completely isolated
## 🎥 Demos
- Normal Scenario
c88c3e73-973e-4dd2-babe-63e2c6ae2dda
- Retry On Failures
b952963a-5d67-472a-bd69-9cd9e49b0ed1
- Failing Again After Max Retries
31cb2e4c-cb90-4490-8bcc-ccb11994f9fa
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR overrides console functions only in production, in order to
sanitize input parameters for any potential calls made to the global
console from Kibana's dependencies.
This initial implementation overrides the `debug`, `error`, `info`,
`log`, `trace`, and `warn` functions, and only sanitizes string inputs.
Future updates may expand this to handle other types, or strings nested
in objects.
The unmodified console methods are now exposed internally in Kibana as
`unsafeConsole`. Where needed for formatting (log appenders, core
logger), calls to the global console have been replaced by
`unsafeConsole`. This PR also adds a new es linting rule to disallow
calls to `unsafeConsole` unless `eslint-disable-next-line
@kbn/eslint/no_unsafe_console` is used.
### Testing
Not sure how we could test this. The overrides are only enabled when
running in a true production environment (e.g. docker) by checking
`process.env.NODE_ENV`.
I was able to manually test by adding additional console output denoting
when the console functions were being overriden or not.
Closes https://github.com/elastic/kibana-team/issues/664Closes#176340
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Identify ES|QL data views as such by setting type attribute. Also
creates `data-view-utils` package so dependency annoyances can be
avoided by consumers.
Part of https://github.com/elastic/kibana/issues/168131
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
This PR removes unused kbn url state package. It was only used in the
flyout and is now replaced with a shared utility, used by several other
modules.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR adds support for ES|QL queries in Data visualizer.
<img width="1695" alt="Screenshot 2024-01-26 at 17 07 59"
src="8a54b859-60d6-4c47-b3dd-e5f3ed43b6b0">
<img width="1695" alt="Screenshot 2024-01-26 at 17 12 39"
src="32fd08e4-7f3b-43e6-81a7-7ec4e777bac0">
a3f540e9-461d-4ebc-bd69-de4ffa2bc554
### Changes:
- Add a new card from the Data visualizer main page
- Add a link from the ML navigation
<img width="1717" alt="Screenshot 2024-01-08 at 18 03 50"
src="832f7890-4ce6-44c1-ab87-cde01f4bf1c0">
- Added a new button to Use ES|QL
<img width="1714" alt="Screenshot 2024-01-09 at 11 23 09"
src="a38a9360-6691-4f3b-a824-8481ab543250">
- Support for **keyword**, **text**, **numeric**, **boolean**, **date**,
and **ip** fields
<img width="1714" alt="Screenshot 2024-01-09 at 11 24 38"
src="b122ee5c-1500-4e2b-9434-e64b0b6ea3be">
<img width="1441" alt="Screenshot 2024-01-09 at 11 25 25"
src="eb35ee78-8a34-467e-84da-2026b01fcda1">
<img width="969" alt="Screenshot 2024-01-09 at 11 44 02"
src="d0f9947d-2b2c-4c14-89ba-9fc5d0a2bf64">
<img width="981" alt="Screenshot 2024-01-10 at 12 01 42"
src="aa5a8d44-7447-41fc-a544-d1b626bf8bce">
- Default to user's fieldFormats for fields that are dynamic generated
by ES|QL, else use Data view's format
- Default to Data view's setting (e.g. type `bytes` in this case for
field `bytes_normal_counter`)
<img width="1037" alt="Screenshot 2024-01-10 at 12 10 38"
src="9fb7e31c-f397-4209-a463-e1a43fe27ffd">
- Default to user's fieldFormats formatting for dynamically generated
fields (e.g. type `number` in this case for field `avg_price`)
<img width="1283" alt="Screenshot 2024-01-10 at 12 01 03"
src="acc25358-50bb-4237-9476-86067ef0badf">
- Add a new UI control to allow users to limit analysis to 5,000 -
10,000 - 100,000 - 1,000,000, rows. This speeds up fetching of the stats
for big data sets and avoid potential circuit breaking exceptions.
- Break overall stats request into smaller parallel requests (which
prevent time out or payload too big due by too many fields), at 10
requests at a time
- Break field stats for individual fields into more efficient batches
(which prevent time out or payload too big due by too many fields), at
10 requests at at ime
- Improve error handling by propagating up the error AND the ES|QL
request in both the UI and the developer's console (for better
debugging)
- Improve error handling in field stats rows: If one field, or a group
of fields, say 'keyword' fields fail to fetch for some reasons, it will
show error for that field but not affect all other fields.
<img width="1690" alt="Screenshot 2024-01-26 at 16 04 28"
src="6e240e12-76b4-42d6-b3be-c05342d76df9">
- Add deep linking in the top search bar
<img width="1185" alt="Screenshot 2024-01-26 at 16 56 49"
src="4f24df68-edc5-41c5-b2ed-d6150ba1e20b">
- More robust support for keyword fields with geo data
<img width="1438" alt="Screenshot 2024-01-26 at 16 55 01"
src="3b97925b-ca28-4952-8082-8d3242e3cb3f">
### Todos:
- [x] Add earliest/latest for date time fields -> Current blocker:
escape special characters in esql variable names
- [x] Fix formatting of numbers for dynamic query, where we don't know
the formatting based on the data view
- [x] Fix date time 'Update' not updating until Refresh is clicked
- [x] Better optimization to not fetch distribution & expanded row
content for pages that are not visible
### Good to have:
- [ ] Investigate bringing back the +/- filter buttons (either by
modifying the ES|QL query directly or by adding separate DSL filters?)
------------
### Checklist
Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
- Rename `log_explorer` to `logs_explorer` and move it to an
`observability_solution` subfolder
- Rename `observability_log_explorer` to `observability_logs_explorer`
and move it to an `observability_solution` subfolder
- Use `renameFromRoot` to rename old configs
Related to https://github.com/elastic/kibana/issues/171991 and
https://github.com/elastic/kibana/pull/170759
This PR is mostly focused on renaming the plugins, follow ups will be
created to change mentions of `log explorer`, `logExplorer` etc.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani01@gmail.com>
## Summary
issue https://github.com/elastic/security-team/issues/7382
### Data Stream Adapter
This PR introduces the `@kbn/data-stream-adapter` package, which is a
utility library to facilitate Data Stream creation and maintenance in
Kibana, it was inspired by the data stream implementation in the Alerts
plugin.
The library has two exports:
- `DataStreamSpacesAdapter`: to manage space data streams. It uses the
`name-of-the-data-stream-<spaceId>` naming pattern.
- `DataStreamAdapter`: to manage single (not space-aware) data streams.
Usage examples in the package
[README](450be0369d/packages/kbn-data-stream-adapter/README.md)
### Data Quality Dashboard
The `DataStreamSpacesAdapter` has been integrated into the data quality
dashboard to store all the quality checks users perform. The information
stored is the metadata (also used for telemetry) and the actual data
rendered in the tables.
FieldMap definition
[here](450be0369d/x-pack/plugins/ecs_data_quality_dashboard/server/lib/data_stream/results_field_map.ts)
### Demo
311a0bf5-004b-46d7-8140-52a233361c91
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Philippe Oberti <philippe.oberti@elastic.co>
Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
Co-authored-by: Efe Gürkan YALAMAN <efeguerkan.yalaman@elastic.co>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Sander Philipse <94373878+sphilipse@users.noreply.github.com>
Co-authored-by: JD Kurma <JDKurma@gmail.com>
Co-authored-by: Jan Monschke <jan.monschke@elastic.co>
Co-authored-by: Patryk Kopyciński <contact@patrykkopycinski.com>
Co-authored-by: Khristinin Nikita <nikita.khristinin@elastic.co>
Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
Co-authored-by: Julia Rechkunova <julia.rechkunova@elastic.co>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
Co-authored-by: Eyo O. Eyo <7893459+eokoneyo@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Marta Bondyra <4283304+mbondyra@users.noreply.github.com>
Co-authored-by: Søren Louv-Jansen <soren.louv@elastic.co>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Co-authored-by: Candace Park <56409205+parkiino@users.noreply.github.com>
## Summary
Close https://github.com/elastic/kibana/issues/166720
I marked this a breaking change since it is preventing users from doing
something they have been able to do before. They can no longer save
changes to managed Lens visualizations. Instead, they have to save
changes to a new visualization.
To test, import this `ndjson` file which includes both a managed and an
unmanaged visualization:
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
## Summary
This PR adds the [High Cardinality
Indexer](https://github.com/elastic/high-cardinality-cluster) to Kibana
as a new package called `kbn-data-forge`. It also replaces
`kbn-infra-forge` usage in the test and is the preferred way to generate
data for Observability use cases, specifically for SLO testing.
### Todo
- [x] Replace `kbn-infra-forge` usage
- [x] Create convenience functions for testing (`generate` and
`cleanup`)
- [x] Make the logger (`LoggingTool`) configurable as an injected
dependency
- [x] Make the Elasticsearch client (`Client`) configurable as an
injected dependency
- [x] Fix the ECS Generate commands
- [x] Add CLI options via Commander
### CLI Help Screen
```
Usage: data_forge.js [options]
A data generation tool that will create realistic data with different scenarios.
Options:
--config <filepath> The YAML config file
--lookback <datemath> When to start the indexing (default: "now-15m")
--events-per-cycle <number> The number of events per cycle (default: 1)
--payload-size <number> The size of the ES bulk payload (default: 10000)
--concurrency <number> The number of concurrent connections to Elasticsearch (default: 5)
--index-interval <milliseconds> The interval of the data in milliseconds (default: 60000)
--dataset <dataset> The name of the dataset to use. Valid options: "fake_logs", "fake_hosts", "fake_stack" (default: "fake_logs")
--scenario <scenerio> The scenario to label the events with (default: "good")
--elasticsearch-host <address> The address to the Elasticsearch cluster (default: "http://localhost:9200")
--elasticsearch-username <username> The username to for the Elasticsearch cluster (default: "elastic")
--elasticsearch-password <password> The password for the Elasticsearch cluster (default: "changeme")
--elasticsearch-api-key <key> The API key to connect to the Elasticsearch cluster
--kibana-url <address> The address to the Kibana server (default: "http://localhost:5601")
--kibana-username <username> The username for the Kibana server (default: "elastic")
--kibana-password <password> The password for the Kibana server (default: "changeme")
--install-kibana-assets This will install index patterns, visualizations, and dashboards for the dataset
--event-template <template> The name of the event template (default: "good")
--reduce-weekend-traffic-by <ratio> This will reduce the traffic on the weekends by the specified amount. Example: 0.5 will reduce the traffic by half (default: 0)
--ephemeral-project-ids <number> The number of ephemeral projects to create. This is only enabled for the "fake_stack" dataset. It will create project IDs that will last 5 to 12 hours. (default: 0)
-h, --help output usage information
```
### Testing an Example
Run the following command against a clean Kibana development enviroment:
```
node x-pack/scripts/data_forge.js --events-per-cycle 200 --lookback now-1h --install-kibana-assets --ephemeral-project-ids 10 --dataset fake_stack
```
This should install a handful of DataViews (Admin Console, Message
Processor, Nginx Logs, Mongodb Logs) along with a few dashboards and
visualizations.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes#160719 and #164413
This PR contains some work about Lens datatable, here's a short list:
* moved out the sorting logic of the datatable into an independent
package: `@kbn/sort-predicates`
* leverage the EUI Datagrid `schemaDetectors` for the table rows sorting
* apply datatable columns sorting also to the CSV exporter
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Dealing with circular dependencies between plugins has become a sharp
pain point for anyone developing plugins in Kibana.
### Providing dependencies to a plugin
First, a plugin defines its dependencies in its `kibana.jsonc` file as
one of three types:
- `required` - the dependency must be present and enabled -- will be
guaranteed in the lifecycle
- `optional` - the dependency can be missing or disabled -- will be
`undefined` in the lifecycle
- `requiredBundle` - the dependency is required as static code only --
will not be present in the lifecycle
Missing or circular dependencies are detected by the Kibana platform
when it starts.
### Providing dependencies in code
Our plugins are written in and type-checked by Typescript. As such, each
plugin needs to maintain Typescript types defining what the platform is
providing. This is done manually, and there is no enforcement mechanism
between that and the plugin Typescript types. If these dependency
definitions are inconsistent or stale, it can lead to host of issues:
- optional plugins typed as required that are disabled crash the plugin
at runtime;
- plugins that are no longer used still included in dependency checks;
- plugins marked as required or optional that are actually required
bundles.
- etc.
### Dependencies with side-effects
One of the interesting things that has come out of this has been
identifying plugins that provide dependent logic through side-effects,
rather than lifecycles.
As an example, `licensing` provides a lifecycle contracts, but also a
[route handler
context](https://github.com/elastic/kibana/blob/main/x-pack/plugins/licensing/server/licensing_route_handler_context.ts)
as middleware for a dependent plugin. Unfortunately, while this
dependency can be stated as `required` in a dependent plugin's
`kibana.jsonc`, the fact that this is a side-effect makes it incredible
difficult to understand the dependency without searching the code.
<img width="735" alt="Screenshot 2023-12-13 at 10 08 00 AM"
src="b4201c86-4811-4506-b2d0-be5bf8c372b0">
So the side-effect is more or less hidden from developers. This is
likely why we see other plugins using the lifecycle
[logic](https://github.com/elastic/kibana/blob/main/src/plugins/maps_ems/public/kibana_services.ts#L33-L37),
or copy-pasting licensing check code
[[1](https://github.com/elastic/kibana/blob/main/x-pack/plugins/actions/server/lib/license_state.ts),
[2](https://github.com/elastic/kibana/blob/main/x-pack/plugins/alerting/server/lib/license_state.ts)],
or relying on the route context side-effect.
## Proposed (initial) solution
This script is an initial attempt to both identify these problems and
surface a plugin's dependencies in a useful way. In addition, the script
will warn if the classes aren't typed well, not typed at all, or even
don't extend the `core` `Plugin` class.
<img width="1426" alt="Screenshot 2023-12-13 at 12 37 25 AM"
src="e044afb7-26f5-4d96-92db-d2eb0a3dfc6e">
<img width="1413" alt="Screenshot 2023-12-13 at 12 38 07 AM"
src="69217a34-9840-4d32-98de-eeeb863d4a50">
<img width="1071" alt="Screenshot 2023-12-13 at 12 38 35 AM"
src="57736027-2d10-44bf-8230-29fdb8b77cb2">
For side-effects, identifying them is key, and then refactoring the
plugins to provide appropriate logic in the `start` or `setup`
contracts.
## Next steps
- [x] refine the logic
- [ ] write tests
- [ ] add `--fix` option
I'm also considering (in another PR), outputting a consistent type
definition file-- perhaps `kibana.d.ts`-- to the plugin from which the
implementing classes could `Omit<>` or `Pick<>` the relevant contracts.
## Summary
Rewrites datatable toolbar tests to testing-library.
It also starts a new package – shared space to park EUI component test
helpers that we eventually donate to EUI for ongoing maintenance. So far
it's just one helper but hopefully we'll be adding helpers soon!
## Summary
This PR removes the legacy kibana react code-editor, alongside replacing
all import declarations of this legacy component to the one offered by
shared-ux, i.e import declaration source of `'@kbn/kibana-react/public'`
is switched to `@kbn/code-editor`.
Also in this PR an helper for writing jest tests has been included
through the package `@kbn/code-editor-mock`, this would facilitate
mocking the editor, especially given that the code editor leverages
couple of APIs that are aren't included by default in jsdom, among them,
`matchMedia`, `ResizeObserver`. The provided mock is sufficient for most
use cases and can be setup in any package within kibana as a
[`node_module`
mock](https://jestjs.io/docs/manual-mocks#mocking-node-modules) without
having to repeatedly manually mock the editor within individual test
files. An example for how this might be done can be found here
ec5ba25368
### Checklist
<!-- Delete any items that are not applicable to this PR.
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
-->
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
<!--
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/167632
This PR provides a simpler api for the Lens embeddable consumers who
want to provide inline editing capabilities. I added an example to help
with the integration. Run kibana with
```
yarn start --run-examples
```
http://localhost:5601/app/lens_embeddable_inline_editing_example
<img width="1381" alt="image"
src="58e7ef2d-2f92-4bab-9cb4-d04a90d87e15">
<img width="2498" alt="image"
src="0a050e8d-f22f-4c48-88e4-20c42683a279">
It also allows the consumers to render the inline editing component in a
custom element in case you don't want to open a push flyout.
I included a readme on how to use the api.
### Note
This is the first PR which uses the new Lens config builder so some of
the changes are not related to the api improvements but they are fixing
some bugs on the builder.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/170236.
### Changes
This PR focuses on adding an action per dataset + namespace to navigate
to Observability log explorer. At the same time I took the opportunity
to update `Open in discover` link to include the controls present in
`Observability Log Explorer` state (atm just `namespace`)
1. Extracted `getRouterLinkProps` to a new package for reusability.
2. New `Actions` column was added to table.
3. `LogExplorerLink ` component was introduced, to reuse the navigation
logic between the table and the flyout.
4. `getDiscoverFiltersFromState` was added to combine state filters and
controls into discover state when navigating to discover
#### Demo
a3f38615-d8ae-432b-ba7b-05a6901f870c
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR adds a script that determines GitHub code ownership for
functional test files in the Kibana repository.
### Why do we need this?
We want to be able to determine test ownership to allow teams to get a
better overview of their tests (number of tests, number of skipped
tests, number of failures in the last x days, etc).
### What does this PR bring?
This PR is a first step on closing the test ownership gaps. It adds
functionality to determine the GitHub code owner for a given file (in
the `@kbn/code-owners` package) and adds a script that makes use of this
to check if all functional test files have a code owner, reporting the
gaps.
### Future plans
The idea is to include the test ownership information in our ingested
test results, such that we can create dashboards, reports, etc based on
it.
At some point (once all ownership gaps are closed), we might consider
running this check on CI to prevent new test files without owners.
### How to run?
```
node scripts/check_ftr_code_owners.js
```
The script lists the functional test files that are not covered by code
owners and also gives a summary like this:
```
ERROR Found 2592 test files without code owner (checked 7550 test files in 12.73 s)
```
- Closes https://github.com/elastic/kibana/issues/172570
## Summary
This PR creates a new package `@kbn/visualization-utils` and moves
`getTimeZone` helper into it. Also the PR removes duplicates of other
similar helpers.
And the histogram in the field popover has now the same time zone
configuration as the the main hits histogram:
<img width="500" alt="Screenshot 2023-12-06 at 18 46 25"
src="2d350c91-1a41-419a-9d80-f203f1c90327">
## For testing
Change `dateFormat:tz` in Advanced Settings and check if histograms are
rendered accordingly.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#161002Closes#170073
## Summary
This PR implements a createModelVersion API in the Encrypted Saved
Objects plugin to support upward migrations for model version encrypted
saved objects.
Much like how the `createMigration` API provided a way to wrap migration
functions to support migration of encrypted saved objects prior to the
model version paradigm, the new `createModelVersion` API provides a way
to wrap a model version definition for the same purpose.
`createModelVersion` manipulates the changes defined for a model version
('unsafe_transform', 'data_backfill', 'data_removal'), merging them into
a single transform function in which the saved object document is
decrypted, transformed, and then encrypted again. The document is
decrypted with the `encrypted saved object type registration` provided
by the required `inputType` parameter. Similarly, the document is by
encrypted with the `encrypted saved object type registration` provided
by the required `outputType` parameter.
An example plugin (`examples/eso_model_version_example`) provides a
demonstration of how the createModelVersion API should be used. The UI
of the example plugin gives an idea of what the encrypted saved objects
look like before and after the model version changes are applied.
## Testing
### Manual Testing
- Modify the example plugin implementation in
`examples/eso_model_version_example` to include different changes or
additional model versions.
### Unit Tests
-
`x-pack/plugins/encrypted_saved_objects/server/create_model_version.test.ts`
### Functional Tests
-
`x-pack/test/encrypted_saved_objects_api_integration/tests/encrypted_saved_objects_api.ts`
-
`x-pack/test/encrypted_saved_objects_api_integration/tests/encrypted_saved_objects_decryption.ts`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Part of https://github.com/elastic/kibana/issues/171570
PR breaks EsError out of data plugin and into @kbn/search-errors
package. This will allow for future efforts to move [lens error
handing](https://github.com/elastic/kibana/blob/main/x-pack/plugins/lens/public/editor_frame_service/error_helper.tsx)
into this package and more closely align error handling across kibana.
An unplanned part of this PR was moving `BfetchRequestError` into its
own package `@kbn/bfetch`. This is required because `@kbn/search-errors`
package can not import code from a plugin. `BfetchRequestError` could
not be part of `@kbn/search-errors` because that created a circular
dependency graph.
Another unexpected problem found while working on this PR is that the
package had to be included in `ui-shared-deps`. The reason for this is
that `renderSearchError` uses `instanceof EsError` check to determine if
an error is an EsError. This check failed when the package was not
included in `ui-shared-deps` because each plugin got its own copy of
`kbn/search-error` plugin. This meant that `data` plugin had its own
copy of `EsError` class when the error is generated, then `discover`
plugin had its own copy of `EsError` class when the instanceof check
occurs. `instanceof` check unexpectedly returned false in this case.
Moving plugin to `ui-shared-deps` results in a single instance of the
plugin. Thanks @mistic for the help with moving package into
`ui-shared-deps`
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiago.costa@elastic.co>
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
## [Security Solution] [Elastic AI Assistant] Retrieval Augmented Generation (RAG) for Alerts
This PR implements _Retrieval Augmented Generation_ (RAG) for Alerts in the Security Solution. This feature enables users to ask the assistant questions about the latest and riskiest open alerts in their environment using natural language, for example:
- _How many alerts are currently open?_
- _Which alerts should I look at first?_
- _Did we have any alerts with suspicious activity on Windows machines?_
### More context
Previously, the assistant relied solely on the knowledge of the configured LLM and _singular_ alerts or events passed _by the client_ to the LLM as prompt context. This new feature:
- Enables _multiple_ alerts to be passed by the _server_ as context to the LLM, via [LangChain tools](https://github.com/elastic/kibana/pull/167097)
- Applies the user's [anonymization](https://github.com/elastic/kibana/pull/159857) settings to those alerts
- Only fields allowed by the user will be sent as context to the LLM
- Users may enable or disable anonymization for specific fields (via settings)
- Click the conversation's `Show anonymized` toggle to see the anonymized values sent to / received from the LLM:
### Settings
This feature is enabled and configured via the `Knowledge Base` > `Alerts` settings in the screenshot below:
- The `Alerts` toggle enables or disables the feature
- The slider has a range of `10` - `100` alerts (default: `20`)
When the setting above is enabled, up to `n` alerts (as determined by the slider) that meet the following criteria will be returned:
- the `kibana.alert.workflow_status` must be `open`
- the alert must have been generated in the last `24 hours`
- the alert must NOT be a `kibana.alert.building_block_type` alert
- the `n` alerts are ordered by `kibana.alert.risk_score`, to prioritize the riskiest alerts
### Feature flag
To use this feature:
1) Add the `assistantRagOnAlerts` feature flag to the `xpack.securitySolution.enableExperimental` setting in `config/kibana.yml` (or `config/kibana.dev.yml` in local development environments), per the example below:
```
xpack.securitySolution.enableExperimental: ['assistantRagOnAlerts']
```
2) Enable the `Alerts` toggle in the Assistant's `Knowledge Base` settings, per the screenshot below:
## How it works
- When the `Alerts` settings toggle is enabled, http `POST` requests to the `/internal/elastic_assistant/actions/connector/{id}/_execute` route include the following new (optional) parameters:
- `alertsIndexPattern`, the alerts index for the current Kibana Space, e.g. `.alerts-security.alerts-default`
- `allow`, the user's `Allowed` fields in the `Anonymization` settings, e.g. `["@timestamp", "cloud.availability_zone", "file.name", "user.name", ...]`
- `allowReplacement`, the user's `Anonymized` fields in the `Anonymization` settings, e.g. `["cloud.availability_zone", "host.name", "user.name", ...]`
- `replacements`, a `Record<string, string>` of replacements (generated on the server) that starts empty for a new conversation, and accumulates anonymized values until the conversation is cleared, e.g.
```json
"replacements": {
"e4f935c0-5a80-47b2-ac7f-816610790364": "Host-itk8qh4tjm",
"cf61f946-d643-4b15-899f-6ffe3fd36097": "rpwmjvuuia",
"7f80b092-fb1a-48a2-a634-3abc61b32157": "6astve9g6s",
"f979c0d5-db1b-4506-b425-500821d00813": "Host-odqbow6tmc",
// ...
},
```
- `size`, the numeric value set by the slider in the user's `Knowledge Base > Alerts` setting, e.g. `20`
- The `postActionsConnectorExecuteRoute` function in `x-pack/plugins/elastic_assistant/server/routes/post_actions_connector_execute.ts` was updated to accept the new optional parameters, and to return an updated `replacements` with every response. (Every new request that is processed on the server may add additional anonymized values to the `replacements` returned in the response.)
- The `callAgentExecutor` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/execute_custom_llm_chain/index.ts` previously used a hard-coded array of LangChain tools that had just one entry, for the `ESQLKnowledgeBaseTool` tool. That hard-coded array was replaced in this PR with a call to the (new) `getApplicableTools` function:
```typescript
const tools: Tool[] = getApplicableTools({
allow,
allowReplacement,
alertsIndexPattern,
assistantLangChain,
chain,
esClient,
modelExists,
onNewReplacements,
replacements,
request,
size,
});
```
- The `getApplicableTools` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/index.ts` examines the parameters in the `KibanaRequest` and only returns a filtered set of LangChain tools. If the request doesn't contain all the parameters required by a tool, it will NOT be returned by `getApplicableTools`. For example, if the required anonymization parameters are not included in the request, the `open-alerts` tool will not be returned.
- The new `alert-counts` LangChain tool returned by the `getAlertCountsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_tool.ts` provides the LLM the results of an aggregation on the last `24` hours of alerts (in the current Kibana Space), grouped by `kibana.alert.severity`. See the `getAlertsCountQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/alert_counts/get_alert_counts_query.ts` for details
- The new `open-alerts` LangChain tool returned by the `getOpenAlertsTool` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_tool.ts` provides the LLM up to `size` non-building-block alerts generated in the last `24` hours (in the current Kibana Space) with an `open` workflow status, ordered by `kibana.alert.risk_score` to prioritize the riskiest alerts. See the `getOpenAlertsQuery` function in `x-pack/plugins/elastic_assistant/server/lib/langchain/tools/open_alerts/get_open_alerts_query.ts` for details.
- On the client, a conversation continues to accumulate additional `replacements` (and send them in subsequent requests) until the conversation is cleared
- Anonymization functions that were only invoked by the browser were moved from the (browser) `kbn-elastic-assistant` package in `x-pack/packages/kbn-elastic-assistant/` to a new common package: `x-pack/packages/kbn-elastic-assistant-common`
- The new `kbn-elastic-assistant-common` package is also consumed by the `elastic_assistant` (server) plugin: `x-pack/plugins/elastic_assistant`
## Summary
This PR adds a bunch of plugins to help manage AI Assistant Management
settings.
It offers a 'selection' plugin inside Stack Management where a user can
select which AI Assistant she wants to manage.
The Security team can hook into this one, so settings for both AI
Assistants can be accessed from inside one place inside Stack
Management.
This PR also adds the plugin to manage settings for the AI Assistant for
Observability, including Knowledge Base management. This plugin is
available both in Stack Management (stateful) and Project Settings
(serverless).
## What it looks like
51392ec5-05c9-4947-9bf2-810d8d0b7525
## Detailed
1. **Adds a Stack Management plugin**
(`/src/plugins/ai_assistant_management/selection`). Its primary function
is to render a selection screen to help users navigate to the settings
plugin for the AI Assistant for a specific solution. This plugin is
displayed in Stack Management, which is only available in stateful
versions of Kibana.
2. **Adds a AI Assistant for Observability Settings plugin**
(`/src/plugins/ai_assistant_management/observability`). This plugin
allows management of specific Observability AI Assistant settings. It is
available in stateful versions of Kibana (via the aforementioned Stack
Management plugin) or in serverless versions via Project Management.
3. **Knowledge Base management for Observability AI Assistant**: The AI
Assistant for Observability Settings plugin has a Knowledge Base tab,
which allows users to add / read / update / delete and bulk import
entries into the Knowledge Base of the Observability AI Assistant.
4. **Moving of KB endpoints in Observability AI Assistant plugin**: KB
endpoints and functions were located in the same folder. As this PR adds
new endpoints for the KB for CRUD operations, it also moves the existing
ones from the function folder into a dedicated one so there's a clearer
distinction between kb and functions.
5. **Adding of GenAI Connector inside Chat Flyout**: If the user has
admin rights, it is possible to set up a GenAI connector from within the
Observability AI Assistant Chat Flyout. This provides a faster and more
seamless onboarding experience. If the user does not, she will be
redirected to the Settings page.
## Bug fixes
* Fixes chat item styling issues (padding, background color).
## How to test
* Check if the Stack Management plugin works on stateful
* Check if the AI Assistant Settings plugin works on stateful +
serverless
* Check if CRUD operations on KB work
* Check if searching on KB entries work
* Check if its possible to navigate to KB tab directly
(`app/management/kibana/aiAssistantManagementObservability?tab=knowledge_base`)
## Todo
- [x] Add sorting to getEntries
- [x] Add params for tab routing
- [x] Add unit tests
- [ ] Add API tests
- [ ] Add fallback for already indexed entries when searching
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR:
* extracts `enterprise_search` index documents component to common
package.
* Uses EUI pagination, converting Elasticsearch`from` & `size` to EUI
pagination standard for documents list
### Screen Recording
f585d9cc-f92c-44f4-aead-23c75c107a0f
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Partially resolves: https://github.com/elastic/kibana/issues/164255,
this is 2/3 of the scoped query changes.
Maintenance window scoped query frontend changes. Adds the ability to
add and edit scoped query for maintenance windows. Due to limitations
with the alerts search bar and each solution fetches AAD fields, we only
allow users to associate scoped query with 1 category (manangement,
o11y, or security solution). The intended usage in this case is for the
user to create multiple maintenance windows if they wish to apply scoped
queries to multiple solutions.
### To test:
go to
`x-pack/plugins/alerting/public/pages/maintenance_windows/constants.ts`
and set `IS_SCOPED_QUERY_ENABLED` to `true`
### Scoped query off, multiple category allowed:
### Scoped query on, multiple category disallowed:
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Closes#171613
## Summary
This PR adds the viewInApp URL to the custom threshold rule type. This
URL will send the user to the log explorer with the selected data view
and the rule's query filter. If there is only one document aggregation,
then the filter related to this aggregation will be added as shown
below:
|Rule|Discover with pre-fill data|
|---|---|
||
For the ad-hoc data view, you should be able to see the selected index
pattern in discover similar to this:
<img
src="046493ae-ba59-46b7-a40f-68d1836d43f1"
width=400 />
### 🧪 How to test
- Check the viewInApp URL both in action variables and the alert table
for the following scenarios:
- A rule with a persisted data view
- A rule with an ad-hoc data view
- A rule with count aggregation and filter
- A rule with an optional query filter
- A rule with non-count aggregation
In all the above scenarios, the starting time in the Discover should be
before the alert's start time.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Consolidates UI elements and backend code to create/delete data views
and destination indices related to transforms and data frame analytics.
We ended up with two different approaches for creating data views in the
wizards for transforms and data frame analytics, the original reason was
we were not aware of the `allowNoIndex: true` setting and worked around
that in different ways.
This PR aligns UI workflows and moves related code to a new package
`@kbn/ml-data-view-utils` for data views and
`@kbn/ml-creation-wizard-utils` for the destination index form. The
latter might be used for other shared components across wizard..
In Data Frame Analytics, the checkbox to create a data view was removed
from the last "Create" step, instead the option to create a data view
was moved to the "Details" step.
In Transforms, the UI component to create the destination index was
brought over from DFA where there is a switch option to automatically
use the job ID as the name for the destination index by default.
**Addresses:** https://github.com/elastic/security-team/issues/7981
## Summary
This PR adds an OpenAPI spec bundler to simplify integration with the
Docs Engineering team. The bundler produces a single bundled file by
dereferencing and inlining some of external references and bundling them
and paths into a single file.
## Details
Currently we maintain a number of schema files inside
`x-pack/plugins/security_solution/common/api/**.schema.yaml` and it
might be hard for external teams to keep track of all the changes in our
schemas. By creating a singular schema file, we provide a clear
integration point for others.
The bundler addresses the following issues
- hide endpoints that we don't want to expose (Endpoints related to
features hidden under a feature flag and all internal endpoints should
be excluded from the file)
- hide not finished data structures related to features hidden under a
feature flag or data structures that are not designed to be public (For
example `RuleActionAlertsFilter` or `RuleActionParams` are exposed
directly from the Alerting framework and might be considered
implementation details, we don't want to document interfaces that are
not designed to be public so hiding them is a good option)
- modify spec based on presence of `x-modify` property (Instead of
exposing `x-modify: partial` we need to make the exported data structure
partial and instead of exposing `x-modify: required` we need to make the
exported data structure required)
- remove any internal attributes used for code generation like
`x-codegen-enabled` and `x-modify`
- inline some of the reused data structures (We have a lot of low-level
reusable data structures `in common_attributes.schema.yaml` which might
make the final documentation hardly usable from the UX perspective, so
we can inline them)
and lives in a new `@kbn/openapi-bundler` package under
`packages/kbn-openapi-bundler` folder.
### Related changes
- Implicit version type `version: 2023-10-31` has been changed to
explicit string type `version: '2023-10-31'` for all specs under
`security_solution/common/api` folder. Implicit type causes `js-yaml`
parsing it as a `Data` JS object leading to serializing it like
`2023-10-31T00:00:00.000Z`.
- `ListRequestQuery` schema in
`security_solution/common/api/endpoint/actions/list.schema.yaml ` has
been renamed to `EndpointActionListRequestQuery` to avoid conflicts with
`ListRequestQuery` in
`security_solution/common/api/endpoint/metadata/list_metadata.schema.yaml`.
While it's not an issue to have completely different schemas sharing the
same name in different files it may be an indication of pitfalls in the
API design. I'd say it's an open question if such cases need to be
always resolved automatically or reviewed manually. At this moment the
bundler can't resolve such conflicts.
## How to test?
There is a a new JS script added to Security Solution plugin located at
`x-pack/plugins/security_solution/scripts/openapi/bundle.js` with a
corresponding entry in `package.json` named `openapi:bundle`.
To test the PR change directory to Security Solution plugin's root
folder and run the bundler like below
```sh
cd x-pack/plugins/security_solution
yarn openapi:bundle
```
It should produce a bundled OpenAPI spec at
`x-pack/plugins/security_solution/target/openapi/security_solution.bundled.schema.yaml`.
## Open issues
- [x] Circular references (implemented in
dfdf0a51ea)
- [x] Mix of OpenAPI `3.0` and `3.1` specs (Maybe convert automatically
to `3.1`?). Folder like OpenAPI bundling format implemented
[here](0ae7ad5abb)
allows to mix OpenAPI `3.0` and `3.1` specs.
## Improvements
- [ ] Flexible configuration
- [ ] CLI support?
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fix https://github.com/elastic/kibana/issues/170428
The bug this is intended to resolve requires some in-depth steps to
reproduce. Follow the instructions in the issue above. Then, merge in
this branch and compare.
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes partially two remaining tasks from
https://github.com/elastic/kibana/issues/168753
Fixes partially dataview issue from
https://github.com/elastic/kibana/issues/170398
It stretches to maximum approximate 60 characters if any of the labels
in the list is of this length. If the content doesn't need the container
to stretch, it doesn't do it.
<details>
<summary> Field picker in Lens</summary>
minimum width:
<img width="445" alt="Screenshot 2023-11-21 at 15 56 03"
src="2f0f8482-bd00-4ec2-bbde-cbc4f3198eed">
auto-expanded width:
<img width="575" alt="Screenshot 2023-11-21 at 15 58 22"
src="df7bab4d-0a08-4d49-8a91-9386eba15d93">
</details>
<details>
<summary>Layer data view picker in Lens</summary>
<img width="376" alt="Screenshot 2023-11-21 at 16 01 17"
src="b8a98d83-dabc-49bd-a3cc-fc3856de6d3e">
<img width="455" alt="Screenshot 2023-11-21 at 15 58 09"
src="f2c5bde8-7a4a-485f-bf97-fc2179171e50">
<img width="615" alt="Screenshot 2023-11-21 at 15 56 27"
src="0574fc6c-69a3-44e9-9d48-8d427c1c5dba">
</details>
<details> <summary>Data view picker in Unified Search</summary>
<img width="341" alt="Screenshot 2023-11-21 at 16 00 29"
src="1c838ded-0dc5-4632-94e4-1d94586f667c">
<img width="441" alt="Screenshot 2023-11-21 at 15 58 04"
src="87e4f1c0-7922-4b94-a114-f23ece544395">
<img width="561" alt="Screenshot 2023-11-21 at 15 56 20"
src="3ea0f222-5241-4c5b-b00b-4311972754cc">
</details>
<details>
<summary> Data view picker in dashboard Create control flyout</summary>
<img width="677" alt="Screenshot 2023-11-21 at 16 14 00"
src="0455b6ed-555d-4cff-9e34-0de377be6e04">
<img width="682" alt="Screenshot 2023-11-21 at 15 54 56"
src="2a67685c-379d-4c0b-bf56-dbf7c35b3bd4">
</details>
<details>
<summary> Unified search data view select component (tested in
maps)</summary>
<img width="570" alt="Screenshot 2023-11-22 at 14 38 25"
src="bb52ab22-626d-4556-b40c-c9bcc925f426">
</details>
<details>
<summary>Unified search field and value picker</summary>
Adds `panelMinWidth`, removes the custom flex width change behavior
<img width="1142" alt="Screenshot 2023-11-22 at 14 40 26"
src="2450957f-38b7-4a73-b531-7acb29cb56bc">
f4f33624-9287-403e-8472-81f705440f97
</details>
<details>
<summary> Discover breakdown field</summary>
Removes the focus stretching and instead uses the panelMinWidth prop
<img width="419" alt="Screenshot 2023-11-21 at 16 46 50"
src="e35125ad-8823-4bff-954b-8119a352829c">
<img width="619" alt="Screenshot 2023-11-21 at 16 48 20"
src="89e63daf-a59e-43e1-a6ec-91d1b15b0fcd">
</details>
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
In this PR, I'm relocating all Kibana Security types (along with a few
schemas necessary for some of these types, unfortunately) that are part
of public contracts to separate packages. This change will enable any
plugin to utilize Security APIs via "static" or
["runtime"](https://github.com/elastic/kibana/pull/167113) dependencies,
regardless of whether Kibana Security already relies on these plugins or
not.
__NOTE TO REVIEWERS:__ I tried to minimize changes as much as I could
via moving only necessary types. I also didn't move deprecated parts of
the Setup/Start contracts to these new packages.
__Triggered by:__ https://github.com/elastic/kibana/pull/168910
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The package data isn't needed anymore, so we can remove that package. On
top, it seems like the package was causing some issues.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Attempting to merge #170852 again now that the release artefact step has
been fixed as part of https://github.com/elastic/kibana/pull/171457
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Adds the ability to quickly create a categorisation anomaly detection
job from the pattern analysis flyout.
Adds a new `created_by` ID `categorization-wizard-from-pattern-analysis`
which can be picked up by telemetry.
Creates a new package for sharing our AIOPs ui actions IDs. I think we
should move the pattern analysis ID to this package too, but that can be
done in a separate PR.
51349f93-f072-4983-85f0-98741902fb5a
6e618581-8916-4e63-930f-945c96c25e6c
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Closes#170728
This work comes from the need to use agent and cloud provider icons in
the new Log Detail flyout.
Since those icons were already used across the `infra` and `apm`
plugins, this was a good opportunity to extract the shared logic into
packages.
The results of this refactoring are two new packages:
- **@kbn/elastic-agent-utils**: exports small utilities and type
definition used to parse the icon to render and exploits also across the
APM plugin.
- **@kbn/custom-icons**: exports custom icons built on top of EuiIcon,
encapsulating logic related to mapping from data to the relative icon.
Apart from creating the new plugins, this also applies their usage to
the `infra` and `apm` plugins, while the Log Explorer flyout will
benefit from these working on
https://github.com/elastic/kibana/issues/170721.
## 🧪 How to test
### Infra
- Navigate to `Infrastructure -> Hosts`
- Verify the hosts table correctly renders the cloud provider icon for
each table entry.
### APM
- Navigate to `APM -> Services`.
- Verify each table entry correctly displays the related agent icon.
- Navigate to `APM -> Services`.
- Click on a service where t a cloud provider icon is expected to appear
next to the service name.
- Verify the icon is correctly displayed.
- Navigate to `APM -> Services -> Service Map`.
- Create a new group.
- Verify the agent icon is correctly displayed for each entry in the
preview list.
- Navigate to `APM -> Traces`.
- Verify each table entry correctly displays the related agent icon.
- Navigate to `APM -> Settings -> Agent Explorer`.
- Verify each table entry correctly displays the related agent icon.
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR refactors the export type classes into their own packages to be
then instantiated in the reporting plugin. This will reduce bloat in the
central reporting plugin.
**Main packages**
- `kbn/reporting-export-types-{png,pdf,csv}` are server packages with
export type declarations
- `kbn/reporting-export-types-{png,pdf,csv}-common` are shared common
packages with type declarations and constants
**Other changes**
- Remove `reporting.getScreenshots()`
- Remove duplicated `schema_utils.ts`
- Consolidate `JOB_STATUS` declaration as an enum
<img width="1063" alt="image"
src="bced8321-93c5-4ebd-b31e-1fd946166241">
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Timothy Sullivan <tsullivan@elastic.co>
Related to [#166340](https://github.com/elastic/kibana/issues/166340)
## Summary
Add mock identity provider and utils to test serverless user roles.
## Screenshot
### 1. Login selector
<img width="767" alt="Screenshot 2023-11-08 at 15 18 18"
src="82b4a29f-65b4-45d2-bed3-6d9f74043c48">
### 2. Single sign on screen
<img width="437" alt="Screenshot 2023-11-09 at 12 30 46"
src="3d5b6f26-5409-4169-a627-bcf6d09836d9">
### 3. User profile page
<img width="1041" alt="Screenshot 2023-11-08 at 17 36 22"
src="50bd4a5a-f9a8-4643-9384-9a352701b011">
## Testing
SAML is only supported by ES when running in SSL mode.
1. To test the mock identity provider run a serverless project in SSL
mode using:
```bash
yarn es serverless --ssl
yarn start --serverless=es --ssl
```
2. Then access Kibana and login in using "Continue as Test User".
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Related to https://github.com/elastic/kibana/issues/169759.
## Summary
This PR creates the initial plugin for Dataset Quality.
The plugin has the basic boilerplate that allows us to continue working
on the UI and Server sides in parallel.
Further configs are to be added along with upcoming commits like:
1. Storybook
2. FTR configs
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Part of https://github.com/elastic/kibana-team/issues/646
This PR adds an example plugin in `examples/error_boundary` that shows
usage of KibanaErrorBoundary.
The example plugin is used in a functional test to ensure errors are
caught in the appropriate way, and error messages include a working
Refresh button.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
