## Summary
Resolves https://github.com/elastic/kibana/issues/196491.
Adds a `LIMIT` to our ES|QL Discover tests. Without the `LIMIT` we are
getting the following deprecation messages:
> Elasticsearch deprecation: 299
Elasticsearch-9.0.0-c74c06daee0583562c82597b19178268b9f415e5 "No limit
defined, adding default limit of [1000]"
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Closes https://github.com/elastic/kibana/issues/197281
PR replaces `DashboardContainer`, which implements legacy Container and
Embeddable interfaces, with plain old javascript object implementation
returned from `getDashboardApi`.
The following are out of scope for this PR and will be accomplished at a
later time:
1) re-factoring dashboard folder structure
2) removing all uses of Embeddable and EmbeddableInput types
3) removing legacy types like DashboardContainerInput
---------
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Hannah Mudge <Heenawter@users.noreply.github.com>
Co-authored-by: Devon Thomson <devon.thomson@elastic.co>
- Closes https://github.com/elastic/kibana/issues/167582
## Summary
This PR removes the code related to the legacy doc table and 2 Advanced
Settings: `doc_table:legacy` and `truncate:maxHeight`.
The legacy table in Discover was replaced by the new data grid in v8.3.
The `doc_table:legacy` Advanced Setting was added to let users switch
back to the legacy table if necessary. The removal of the setting and
the legacy table entirely would allow us to reduce bundle size,
maintenance burden, and code complexity.
Also the legacy table does not support many new features which were
added to the grid only (e.g. comparing selected documents, context-aware
UI based on current solution project, column resizing, bulk row
selection, copy actions, new doc viewer flyout, and more).
Since v8.15 `doc_table:legacy` is marked as deprecated on Advanced
Settings page via https://github.com/elastic/kibana/issues/179899
Since v8.16 `truncate:maxHeight` is marked as deprecated too via
https://github.com/elastic/kibana/pull/183736
The removal of these 2 settings and the associated code is planned for
v9.
### Checklist
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
## 📓 Summary
Closes#183498
This work sets the Breakdown selector for the Histogram to the
`log.level` field once a logs data source is resolved.
It also applies to ES|QL queries, the change is applied/removed on each
source change.
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
## Summary
This PR contains the refactor of the Lens embeddable with the new React
architecture.
fix https://github.com/elastic/kibana/issues/174957
fixes https://github.com/elastic/kibana/issues/180672
**Current status**:
✅ Ready to review
### Notes for testing and reviewers
Other than reworking the Lens embeddable with the new architecture this
PR contains the following major changes.
#### Edit flow
The `Edit` flow has changed to in-line first using the new `Edit` API
provided by the new system
* The impact of this change can be noticed in the code on the `Canvas`
case where the Custom Lens component is instructed to avoid the inline
editing. In all the other cases in-line editing is enabled by default
now.
* Another side effect of this has been the replacement of the special
`INLINE_EDIT` action id into the regular `EDIT` action. Some tests have
been affected by this replacing the `clickEdit` function with the
`openEditorFromFlyout` one.
* The Inline editing codebase **as been reworked entirely** so make sure
to stress test this side of things.
#### Attribute service
Another important aspect changed in this PR is the `attributeService`:
this was tied to the previous Embeddable system and it is now completely
skipped. The Lens wrapper around that has been reworked to be thinner
and directly call the CM services.
* Please make sure to test thoroughly save/load SO flows
#### Transformation API (by-value <=> by-reference flow)
The new system adopts the new Transformation API (who prevents the panel
to fully reload on change).
* Please make sure to test thoroughly Visualize library <=> by value
flows
* In particular moving from one type and another should change how the
Panel Settings interpret "default" values to reset
#### Message system
Also this part of the code was partially rewritten to be more manageable
ont he embeddable surface, maintaining the core functionalities.
* Please make sure to test thoroughly error messages, warnings and info
messages
* Some scenarios to test includes
* multi-layer errors (i.e. use a broken KQL query for an
annotation/multi-layers). Check that the panel recovers correctly from
it when resolved
* Missing references
* Missing dataViews
* Wrong formatted SO
* Configuration mistakes - check that a broken config is not saveable
### Other areas to check
* Change filters in dashboard/viz and check that are correctly handled
* Check drilldowns
* Check that `Unsaved changes` are correctly detected
* Check that the panel updates correctly on `View` mode change
## Main type changes
This PR contains also some important `type` changes, here's listed:
* the `query` property now explicitly supports ES|QL query type.
* in `main` it used to work without type support
* `LensEmbeddableInput`/`LensEmbeddableOutput` types have changed, but
the type names remained the same.
## Follow ups already planned:
Some enhancements have been already collected and will be addressed in a
follow up [here](https://github.com/elastic/kibana/issues/195355)
### Tasks
<details>
<summary>Detailed list of tasks for the refactor</summary>
* New embeddable factory
* [x] Define visualization context
* [x] Define observables to track
* [x] Basic panel settings
* [x] Basic edit api
* [x] inspector api
* [x] Library services
* [x] Unified search api
* [x] Basic integrations api
* [x] State management api for inline editing
* Publish correct observables
* [x] `dataViews`
* [x] `query`
* [x] `filters`
* [x] `dataLoading`
* [x] `savedObjectId`
* Actions
* [x] View underlying data api
* Custom renderer
* [x] Basic implementation
* [x] Support callbacks
* [x] Support custom styling/paddings
* Expose
* [x] Handle searchSession
* Edit
* [x] Open panel in Lens editor
* Inline editing
* [x] rework references logic
* #180726
* integrate the logic to extract filters dataViews from filters as for
the first bug in #188545
* DSL flyout
* [x] open flyout
* [x] save
* ES|QL
* [x] open flyout on creation
* [x] open flyout on editing
* [x] save
* [x] revisit mounting logic to avoid detach if possible (not possible
yet)
* [x] explore the integration with the new `onEdit` api method used for
the inline editing~~
* [x] created panel management module and sorted it out
* [x] open in Editor
* [x] fix the save on return to dashboard
* ~~migrate by ref to by value on inline editing~~ will do it in a
follow up PR
* Add from library issues
* [x] Fix missing title and tags
* Data loading
* [x] Compute all required data params for rendering
* Render the panel
* [x] hook up user messaging system
* [x] Merge search context
* [x] Expression variables
* [x] panel settings
* [x] per panel time range
* [x] per panel filter
* test with both DSL and ES|QL mode
* Reload
* [x] on unified search updates
* [x] on config changes
* [x] on drilldown changes?
* [x] on view mode change
* Attributes service
* [x] load from library
* [x] save to library
</details>
### Pending issues:
<details>
<summary>Detailed list of issues</summary>
* [x] Unified histogram does not render in Discover
* [x] Saving to library from context menu in dashboard doesn't save the
title
* [x] When adding a vis from the library the new panel has no title
* [x] Vis disappears when opening inline editor and cancel
* Create a viz, save and return to dashboard, then edit it and cancel.
* Saving an edit inline doesn't apply the changes (i.e. changing the
chart type)
* [x] Changing the chart type on the layer panel leads to a crash
* [x] Changing the chart type won't update the visualization (via both
config panel or suggestions)
* [x] Edit a dimension will stretch the panel to overflow the fly-out
* [x] duplicating a dimension in the inline editor by drag and drop
works buggy visually
* When duplicating a panel, the new panel gets the same title rather
than “title (copy)”
* [x] by-value panels
* [x] by-reference panels
* [x] brushing throughout the timerange doesn’t work
* [x] filtering when clicking on value doesn’t work
* [x] filtering from legend doesn’t work
* [x] for lens table, the sort ascending/descending actions don’t have
an effect
* [x] filtering doesn’t display on table either
* Discover related issues
* thanks to @davismcphee investigation the source of the issue seems to
be related to the way the `abortController` is managed in the new
embeddable implementation as Discover is relying on that.
* [x] needs to investigate for a fix that restores the previous
behaviour of the `abortController` management
* [x] the hits total count is not in sync with the chart/table now
* [x] Change chart type via suggestion panel when inline editing in
Discover doesn't update the chart
* [x] Dirty panel issue (see @nickofthyme 's
[comment](https://github.com/elastic/kibana/pull/186642#discussion_r1792659477)
)
* [x] `Unsaved changes` issue (see @mbondyra
[comment](https://github.com/elastic/kibana/pull/186642#discussion_r1795384587))
* [x] Multiple errors not rendered correctly in panel when blocking
(i.e. missing field - `lens-message-list-trigger` related)
* [x] recover from a blocker error required 2 renders
* Missing SO error should not be handled for the custom render component
(legacy behaviour) but should be correctly handled for dashboard (will
be handled in a follow up PR given that is broken on `main` too)
* [x] Too many requests on Unified Histogram when in Discover (3 vs 2)
* [x] Too many request on slow queries for Unified Histogram (2 vs 1)
* [x] Annotations preview issues (chart rendering with height `0px`)
* [x] `uuid` not propagated correctly
* [x] another flavour of this was `id` not propagated correctly into the
`data-test-embeddable-id` attribute
* [x] Dispatch correctly the `render` events
* [x] refresh interval does not propagate thru the Lens custom component
in Discover (thanks to @jughosta to sort this out )
</details>
---------
Co-authored-by: Marta Bondyra <4283304+mbondyra@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Marco Vettorello <vettorello.marco@gmail.com>
Co-authored-by: Marta Bondyra <marta.bondyra@elastic.co>
Co-authored-by: Bhavya RM <bhavya@elastic.co>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
- Closes https://github.com/elastic/kibana/issues/199608
## Summary
This PR changes the logic around when suggestions from lens API are
used. Previously for non-transformational query and non-time-based data
it would try to render one of lens suggestions supplying chart data via
`table` prop. Now, it would not render any chart.
Before:
- Data view mode => Static histogram configuration
- ES|QL mode and non-transformational query => _**Gets lens
suggestions.**_ If histogram chart is not possible, **_takes the first
lens suggestion for rendering the chart_**
- ES|QL mode and transformational query => Gets lens suggestions. Takes
the first lens suggestion for rendering the chart.
After:
- Data view mode => Static histogram configuration (same)
- ES|QL mode and non-transformational query => ~~_**Gets lens
suggestions.**_~~ If histogram chart is not possible, **_renders
nothing_** (updated)
- ES|QL mode and transformational query => Gets lens suggestions. Takes
the first lens suggestion for rendering the chart. (same)
### Testing
As per originally reported case:
1. `node scripts/es_archiver
--kibana-url=http://elastic:changeme@localhost:5601
--es-url=http://elastic:changeme@localhost:9200 load
test/functional/fixtures/es_archiver/many_fields`
2. Navigate to Discover, switch to ES|QL mode and enter `FROM
indices-stats | LIMIT 10`
3. No chart is expected.
Also there should be no regression for
https://github.com/elastic/kibana/pull/195863
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Davis McPhee <davis.mcphee@elastic.co>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Catching an invalid state of properties propagated to the UnifiedHistogram which is using the Lens embeddable in Discover, that causes a rendering error when e.g. ad hoc data views are being edited. Therefore the skipped testview can be unskipped.
## Summary
Closes#192233
Just in time for Thanksgiving - a full buffet of FIPS testing fixes
Usage of non-compliant algorithms manifest as runtime errors, so it is
imperative that we attempt to run all tests possible with Kibana in FIPS
mode. However, several overrides are needed to run Kibana in FIPS mode,
resulting in setup that make it impossible to run.
## In this PR
- Enable Unit tests for FIPS pipeline
- Enable Integration Tests for FIPS pipeline
- Enable Full FTR suite for FIPS pipeline (smoke test had originally run
a subset)
- Skip tests that break with overrides
- Fix/change tests to work in FIPS mode to maximize coverage
- Examine necessity of MD5 when installing from source (TBD based Ops PR
feed back, see self review below)
- Remove md5 from es_file_client options
## Latest Successful FIPS Test Run
https://buildkite.com/elastic/kibana-fips/builds/268
---------
Co-authored-by: Brad White <Ikuni17@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
Co-authored-by: Larry Gregory <larry.gregory@elastic.co>
## Summary
close https://github.com/elastic/kibana/issues/194165
close https://github.com/elastic/kibana-team/issues/1245
### User-facing
<img width="1680" alt="image"
src="https://github.com/user-attachments/assets/6df4ee9f-1b4d-404c-a764-592998a1d430">
This PRs adds a new tab in the editor history component. You can star
your query from the history and then you will see it in the Starred
list. The started queries are scoped to a user and a space.
### Server
To allow starring ESQL query, this PR extends [favorites
service](https://github.com/elastic/kibana/pull/189285) with ability to
store metadata in addition to an id. To make metadata strict and in
future to support proper metadata migrations if needed, metadata needs
to be defined as schema:
```
plugins.contentManagement.favorites.registerFavoriteType('esql_query', {
typeMetadataSchema: schema.object({ query: schema.string(), timeRange:...., etc... }),
})
```
Notable changes:
- Add support for registering a favorite type and a schema for favorite
type metadata. Previosly the `dashboard` type was the only supported
type and was hardcoded
- Add `favoriteMetadata` property to a saved object mapping and make it
`enabled:false` we don't want to index it, but just want to store
metadata in addition to an id.
[code](https://github.com/elastic/kibana/pull/198362/files#diff-d1a39e36f1de11a1110520d7607e6aee7d506c76626993842cb58db012b760a2R74-R87)
- Add a 100 favorite items limit (per type per space per user). Just do
it for sanity to prevent too large objects due to metadata stored in
addtion to ids.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
## Summary
This PR adds a fix for a regression bug introduced with the new
embeddable refactor in 8.16 .
I've added an extra 8.16 FTR test to ensure it works.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
This PR removes the `.empty` file from the `test/functional/screenshots`
directory as it's no longer needed.
### Details
The file has been introduced as part of #14122 when the directory was
cleaned. Since then, new base line screenshots have been added, so the
directory is no longer empty.
1. Speeds up functional test suite by setting the time picker via ui
settings
2. removes legacy tests from editor frame (replaced by adding some
conditions to drag_drop functional tests) as they were not the best
tests anyway
- Related to https://github.com/elastic/kibana/pull/197538
## Summary
This PR removes the support of showing stats in the field popover in
ES|QL mode as this UX will be revisited in the future to provide better
results.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Removes the code that goes through the condition if the vis comes from
new or legacy library. Pie charts or xy charts don't use vislib
implementation anymore, so the condition will always give the same
result.
## Summary
Closes https://github.com/elastic/kibana/issues/184257
This PR removes the modal warning users that unsaved changes would be
lost when navigating away from dashboard to other apps within Kibana.
The modal is not necessary since unsaved changes are saved in session
storage. The benefit is that this removes the unnecessary click for
users.
This does not impact the modal for switching to view mode from edit mode
with unsaved changes as shown below.
https://github.com/user-attachments/assets/c5bdb0ec-b040-40b0-a511-fd16ad084b90
### Checklist
- [x] Update tests
- Closes https://github.com/elastic/kibana/issues/198496
## Summary
This PR fixes an issue when the histogram request returns only a partial
result (0 or greater than 0) by adding a warning icon next to the total
hits counter and not blocking the whole page with "No results" message
(when partial result with 0 hits from histogram).
<img width="1436" alt="Screenshot 2024-10-31 at 15 45 17"
src="https://github.com/user-attachments/assets/9a769fe6-bdcf-4d20-ae6e-698a5b08d76f">
### Testing
Execute the following and open `example*` data view in Discover.
```
PUT example1
PUT example1/_mapping
{
"properties": {
"message": {
"type": "text"
},
"date": {
"type": "date"
}
}
}
PUT example1/_doc/11
{
"message": "11",
"date": "2024-11-11T12:10:30Z"
}
PUT example1/_doc/12
{
"message": "22",
"date": "2024-11-12T12:10:30Z"
}
PUT example2
PUT example2/_mapping
{
"properties": {
"message": {
"type": "keyword"
},
"date": {
"type": "date"
}
}
}
PUT example2/_doc/21
{
"message": "21",
"date": "2024-12-01T12:10:30Z"
}
PUT example2/_doc/22
{
"message": "22",
"date": "2024-12-02T12:10:30Z"
}
```
Then add `message` as a breakdown field.
Notice that the histogram gets some partial results:
<img width="1563" alt="Screenshot 2024-10-31 at 16 11 14"
src="https://github.com/user-attachments/assets/8a53f661-38a2-48f8-b082-823de77ac4f2">
Now, add a filter for `_id: 11` and notice that the histogram request
has no results (it partially failed on some shards) but Discover still
renders the table:
<img width="1564" alt="Screenshot 2024-10-31 at 16 11 31"
src="https://github.com/user-attachments/assets/e154ab5d-c5d4-4703-abd4-7bf3cd7a15fb">
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Davis McPhee <davismcphee@hotmail.com>
