## Summary
This PR removes all imports of Route from react-router-dom and
'@kbn/kibana-react-plugin/public' and instead imports Route from
@kbn/shared-ux-router.
### Context
Based on
https://github.com/elastic/kibana/issues/132629#issue-1243243678 This PR
executes steps 2 - 4:
> 2. To make the transition easier, we want to re-export other
react-router-dom exports alongside the modified' Route'.
> 3. Solutions should start using that Route component in place of the
one from react-router-dom. I.e. replace all occurrences of import { ...
} from 'react-router-dom' with import { ... } from
'@kbn/shared-ux-router'.
> 4. All manual calls to useExecutionContext are not needed anymore and
should be removed.
### Future PR
Looks like this might be getting worked on in:
https://github.com/elastic/kibana/pull/145863 (thanks!)
> Introduce an ESlint rule that ensures that react-router-dom is not
used directly in Kibana and that imports go through the new
@kbn/shared-ux-router package.
This is tangentially accomplished through
https://github.com/elastic/kibana/pull/150340 but only addresses using
Route through @kbn/kibana-react-plugin/public'
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Tiago Costa <tiagoffcc@hotmail.com>
## Summary
Closes#150577
if there are too many agent policies in a system, we were creating too
big a runtime query for elastic and the query would be rejected.
This PR adds a limit, if the user has more than 750 agent policies then
agents will not be marked as inactive anymore. If the user reaches the
limit then a warning badge is displayed (the text underlined has also
been added):
<img width="968" alt="Screenshot 2023-02-13 at 20 14 31"
src="https://user-images.githubusercontent.com/3315046/218565456-f5758e4b-74f6-4e7c-9b49-22f0fd6f9102.png">
Integration test added.
Fixes https://github.com/elastic/kibana/issues/149344
This PR migrates all plugins to packages automatically. It does this
using `node scripts/lint_packages` to automatically migrate
`kibana.json` files to `kibana.jsonc` files. By doing this automatically
we can simplify many build and testing procedures to only support
packages, and not both "packages" and "synthetic packages" (basically
pointers to plugins).
The majority of changes are in operations related code, so we'll be
having operations review this before marking it ready for review. The
vast majority of the code owners are simply pinged because we deleted
all `kibana.json` files and replaced them with `kibana.jsonc` files, so
we plan on leaving the PR ready-for-review for about 24 hours before
merging (after feature freeze), assuming we don't have any blockers
(especially from @elastic/kibana-core since there are a few core
specific changes, though the majority were handled in #149370).
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This change ensures that the `unsafe-eval` source expression isn't included in
the Kibana Content Security Policy (CSP) by default.
Users can set `csp.disableUnsafeEval: false` to reintroduce `unsafe-eval`.
However, since this config option is deprecated as of this commit, it's
recommended to instead set `csp.script_src: ['unsafe-eval']`.
Closes#150156
## Summary
Closes https://github.com/elastic/kibana/issues/150208
The problem is that it adds the filter badge twice. I think that it
doesn't remove the previous one so a new is added. I think that waiting
the chart to be rendered after the clearing of the filters will
stabilize it.
Runner 100 times
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Closes https://github.com/elastic/kibana/issues/140175
Closes https://github.com/elastic/kibana/issues/143580
## Summary
Oh, boy! Get ready for a doozy of a PR, folks! Let's talk about the
three major things that were accomplished here:
### 1) Pagination
Originally, this PR was meant to add traditional pagination to the
options list control. However, after implementing a version of this, it
became apparent that, not only was UI becoming uncomfortably messy, it
also had some UX concerns because we were deviating from the usual
pagination pattern by showing the cardinality rather than the number of
pages:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214687041-f8950d3a-2b29-41d5-b656-c79d9575d744.gif"/></p>
So, instead of traditional pagination, we decided to take a different
approach (which was made possible by
https://github.com/elastic/kibana/pull/148420) - **load more options
when the user scrolls to the bottom!** Here it is in action:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214688854-06c7e8a9-7b8c-4dc0-9846-00ccf5e5f771.gif"/></p>
It is important that the first query remains **fast** - that is why we
still only request the top 10 options when the control first loads. So,
having a "load more" is the best approach that allows users to see more
suggestions while also ensuring that the performance of options lists
(especially with respect to chaining) is not impacted.
Note that it is **not possible** to grab every single value of a field -
the limit is `10,000`. However, since it is impractical that a user
would want to scroll through `10,000` suggestions (and potentially very
slow to fetch), we have instead made the limit of this "show more"
functionality `1,000`. To make this clear, if the field has more than
`1,000` values and the user scrolls all the way to the bottom, they will
get the following message:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214920302-1e3574dc-f2b6-4845-be69-f9ba04177e7f.png"/></p>
### 2) Cardinality
Previously, the cardinality of the options list control was **only**
shown as part of the control placeholder text - this meant that, once
the user entered their search term, they could no longer see the
cardinality of the returned options. This PR changes this functionality
by placing the cardinality in a badge **beside** the search bar - this
value now changes as the user types, so they can very clearly see how
many options match their search:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214689739-9670719c-5878-4e8b-806c-0b5a6f6f907f.gif"/></p>
> **Note**
> After some initial feedback, we have removed both the cardinality and
invalid selections badges in favour of displaying the cardinality below
the search bar, like so:
>
> <p align="center"><img
src="https://user-images.githubusercontent.com/8698078/216473930-e99366a3-86df-4777-a3d8-cf2d41e550fb.gif"/></p>
>
> So, please be aware that the screenshots above are outdated.
### 3) Changes to Queries
This is where things get.... messy! Essentially, our previous queries
were all built with the expectation that the Elasticsearch setting
`search.allow_expensive_queries` was **off** - this meant that they
worked regardless of the value of this setting. However, when trying to
get the cardinality to update based on a search term, it became apparent
that this was not possible if we kept the same assumptions -
specifically, if `search.allow_expensive_queries` is off, there is
absolutely no way for the cardinality of **keyword only fields** to
respond to a search term.
After a whole lot of discussion, we decided that the updating
cardinality was a feature important enough to justify having **two
separate versions** of the queries:
1. **Queries for when `search.allow_expensive_queries` is off**:
These are essentially the same as our old queries - however, since we
can safely assume that this setting is **usually** on (it defaults on,
and there is no UI to easily change it), we opted to simplify them a
bit.
First of all, we used to create a special object for tracking the
parent/child relationship of fields that are mapped as keyword+text -
this was so that, if a user created a control on these fields, we could
support case-insensitive search. We no longer do this - if
`search.allow_expensive_queries` is off and you create a control on a
text+keyword field, the search will be case sensitive. This helps clean
up our code quite a bit.
Second, we are no longer returning **any** cardinality. Since the
cardinality is now displayed as a badge beside the search bar, users
would expect that this value would change as they type - however, since
it's impossible to make this happen for keyword-only fields and to keep
behaviour consistent, we have opted to simply remove this badge when
`search.allow_expensive_queries` is off **regardless** of the field
type. So, there is no longer a need to include the `cardinality` query
when grabbing the suggestions.
Finally, we do not support "load more" when
`search.allow_expensive_queries` is off. While this would theoretically
be possible, because we are no longer grabbing the cardinality, we would
have to always fetch `1,000` results when the user loads more, even if
the true cardinality is much smaller. Again, we are pretty confident
that **more often than not**, the `search.allow_expensive_queries` is
on; therefore, we are choosing to favour developer experience in this
instance because the impact should be quite small.
2. **Queries for when `search.allow_expensive_queries` is on**:
When this setting is on, we now have access to the prefix query, which
greatly simplifies how our queries are handled - now, rather than having
separate queries for keyword-only, keyword+text, and nested fields,
these have all been combined into a single query! And even better -
⭐ now **all** string-based fields support case-insensitive search!
⭐ Yup, that's right - even keyword-only fields 💃
There has been [discussion on the Elasticsearch side
](https://github.com/elastic/elasticsearch/issues/90898) about whether
or not this setting is even **practical**, and so it is possible that,
in the near future, this distinction will no longer be necessary. With
this in mind, I have made these two versions of our queries **completely
separate** from each other - while this introduces some code
duplication, it makes the cleanup that may follow much, much easier.
Well, that was sure fun, hey?
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214921985-49058ff0-42f2-4b01-8ae3-0a4d259d1075.gif"/></p>
## How to Test
I've created a quick little Python program to ingest some good testing
data for this PR:
```python
import random
import time
import pandas as pd
from faker import Faker
from elasticsearch import Elasticsearch
SIZE = 10000
ELASTIC_PASSWORD = "changeme"
INDEX_NAME = 'test_large_index'
Faker.seed(time.time())
faker = Faker()
hundredRandomSentences = [faker.sentence(random.randint(5, 35)) for _ in range(100)]
thousandRandomIps = [faker.ipv4() if random.randint(0, 99) < 50 else faker.ipv6() for _ in range(1000)]
client = Elasticsearch(
"http://localhost:9200",
basic_auth=("elastic", ELASTIC_PASSWORD),
)
if(client.indices.exists(index=INDEX_NAME)):
client.indices.delete(index=INDEX_NAME)
client.indices.create(index=INDEX_NAME, mappings={"properties":{"keyword_field":{"type":"keyword"},"id":{"type":"long"},"ip_field":{"type":"ip"},"boolean_field":{"type":"boolean"},"keyword_text_field":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"nested_field":{"type":"nested","properties":{"first":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"last":{"type":"text","fields":{"keyword":{"type":"keyword"}}}}},"long_keyword_text_field":{"type":"text","fields":{"keyword":{"type":"keyword"}}}}})
print('Generating data', end='')
for i in range(SIZE):
name1 = faker.name();
[first_name1, last_name1] = name1.split(' ', 1)
name2 = faker.name();
[first_name2, last_name2] = name2.split(' ', 1)
response = client.create(index=INDEX_NAME, id=i, document={
'keyword_field': faker.country(),
'id': i,
'boolean_field': faker.boolean(),
'ip_field': thousandRandomIps[random.randint(0, 999)],
'keyword_text_field': faker.name(),
'nested_field': [
{ 'first': first_name1, 'last': last_name1},
{ 'first': first_name2, 'last': last_name2}
],
'long_keyword_text_field': hundredRandomSentences[random.randint(0, 99)]
})
print('.', end='')
print(' Done!')
```
However, if you don't have Python up and running, here's a CSV with a
smaller version of this data:
[testNewQueriesData.csv](10538537/testNewQueriesData.csv)
> **Warning**
> When uploading, make sure to update the mappings of the CSV data to
the mappings included as part of the Python script above (which you can
find as part of the `client.indices.create` call). You'll notice,
however, that **none of the CSV documents have a nested field**.
Unfortunately, there doesn't seem to be a way to able to ingest nested
data through uploading a CSV, so the above data does not include one -
in order to test the nested data type, you'd have to add some of your
own documents
>
> Here's a sample nested field document, for your convenience:
> ```json
> {
> "keyword_field": "Russian Federation",
> "id": 0,
> "boolean_field": true,
> "ip_field": "121.149.70.251",
> "keyword_text_field": "Michael Foster",
> "nested_field": [
> {
> "first": "Rachel",
> "last": "Wright"
> },
> {
> "first": "Gary",
> "last": "Reyes"
> }
> ],
> "long_keyword_text_field": "Color hotel indicate appear since well
sure right yet individual easy often test enough left a usually
attention."
> }
> ```
>
### Testing Notes
Because there are now two versions of the queries, thorough testing
should be done for both when `search.allow_expensive_queries` is `true`
and when it is `false` for every single field type that is currently
supported. Use the following call to the cluster settings API to toggle
this value back and forth:
```php
PUT _cluster/settings
{
"transient": {
"search.allow_expensive_queries": <value> // true or false
}
}
```
You should pay super special attention to the behaviour that happens
when toggling this value from `true` to `false` - for example, consider
the following:
1. Ensure `search.allow_expensive_queries` is either `true` or
`undefined`
2. Create and save a dashboard with at least one options list control
3. Navigate to the console and set `search.allow_expensive_queries` to
`false` - **DO NOT REFRESH**
4. Go back to the dashboard
5. Open up the options list control you created in step 2
6. Fetch a new, uncached request, either by scrolling to the bottom and
fetching more (assuming these values aren't already in the cache) or by
performing a search with a string you haven't tried before
7. ⚠️ **The options list control _should_ have a fatal error** ⚠️<br>The
Elasticsearch server knows that `search.allow_expensive_queries` is now
`false` but, because we only fetch this value on the first load on the
client side, it has not yet been updated - this means the options list
service still tries to fetch the suggestions using the expensive version
of the queries despite the fact that Elasticsearch will now reject this
request. The most graceful way to handle this is to simply throw a fatal
error.
8. Refreshing the browser will make things sync up again and you should
now get the expected results when opening the options list control.
### Flaky Test Runner
<a
href="https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1845"><img
src="https://user-images.githubusercontent.com/8698078/215894267-97f07e59-6660-4117-bda7-18f63cb19af6.png"/></a>
### Checklist
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
> **Note**
> Technically, it actually does - however, it is due to an [EUI
bug](https://github.com/elastic/eui/issues/6565) from adding the group
label to the bottom of the list.
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
### Notes for reviewers
- Currently, autocomplete suggestions for fields don't work with
wildcards and data streams due to the
[bug](https://github.com/elastic/kibana/issues/149496) in the `main`. It
should be addressed separately.
### How to test
In order to spot the loading behaviour, ideally you should create an
index with a heavy mappings definition.
Afterwards, write a query that requires a field from this index, e.g.:
```
GET <my-index>/_search
{
"aggs": {
"my_agg": {
"terms": {
"field": "",
"size": 10
}
}
}
}
```
Place a cursor next to the `field` property, it should trigger mappings
fetch. After that, the mappings definition for this index will be cached
and accessed synchronously.
You can also open the browser's dev tools and enable Network throttling.
It allows noticing loading behaviour for any index.
--------------------
Resolves https://github.com/elastic/kibana/issues/146855
Instead of fetching all mappings upfront, requests mapping definition on
demand per index according to the cursor position.
Considering there is a maximum response size limit of 10MB in the
`/autocomplete_entities` endpoint, field autocompletion wasn't working
at all if the overall mappings definition exceeded this size. Retrieving
mappings per index tackles this and improves the init time.
### Checklist
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
---------
Co-authored-by: Jean-Louis Leysens <jloleysens@gmail.com>
## Summary
Fixes https://github.com/elastic/kibana/issues/149691
Follow up to https://github.com/elastic/kibana/pull/149528
This PR adds 3 separate guide ID values `appSearch`, `websiteSearch` and
`databaseSearch` instead of previously used `search`. With that change,
each card for Search on the landing page has its own independent guide
that can be completed.
Example plugin for guided onboarding is also updated so that all 3
search guides can be managed there. Otherwise there should not be any
UI/UX changes.
Currently, all 3 ids are registered with the same config and that can be
easily updated by the team in the future.
#### Screenshot
<img width="485" alt="Screenshot 2023-01-31 at 18 24 12"
src="https://user-images.githubusercontent.com/6585477/215840343-f9a65be3-0568-40ca-b458-5de92bfe8aaf.png">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Currently, changes to a data view require a round trip to management
when you're in discover. This PR allows editing of data views via flyout
from within discover.
Closes https://github.com/elastic/kibana/issues/144801
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Matthias Wilhelm <ankertal@gmail.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Fixes https://github.com/elastic/kibana/issues/89073 and
https://github.com/elastic/kibana/issues/106547
Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1785
Issue with flaky maps_integration.ts is that there was not await when
filtering maps list by tags. This resulted in list page refreshing after
`const links = await
this.find.allByCssSelector('.euiTableRow-isSelectable .euiLink');` and
then getting an elemented unmounted on future `await
links[i].getVisibleText()` calls.
A [similar fix](https://github.com/elastic/kibana/pull/82930) was
implemented for visualize and dashboard listing pages.
https://github.com/elastic/kibana/pull/82930 introduced
`listingTable.waitUntilTableIsLoaded` but did not introduce the method
in a consistent way. Other methods that search table were not updated to
use `listingTable.waitUntilTableIsLoaded`, but instead used
`this.header.waitUntilLoadingHasFinished()`. This PR resolved this issue
by updating all listingTable methods that search to use
`listingTable.waitUntilTableIsLoaded` and then updated
`listingTable.waitUntilTableIsLoaded` with a call to
`this.header.waitUntilLoadingHasFinished()`
https://github.com/elastic/kibana/pull/82930 did not update
maps_integration tests, only resolving the issue for visualize and
dashboard. To avoid future situations where fixes only resolve a few
usages, this PR moves selectFilterTags into listing_table and replaces
all implementations of selectFilterTags with
listingTable.selectFilterTags.
While investigating dashboard_integrations test, I found
`x-pack/test/functional/apps/dashboard/group2/dashboard_tagging.ts`,
which duplicated most of dashboard_integrations test. This PR removes
x-pack/test/functional/apps/dashboard/group2/dashboard_tagging.ts adds
the unique test case to dashboard_integrations
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR removes the deprecated field list sampling setting which was
marked as deprecated on 8.1. This
https://github.com/elastic/kibana/pull/139828 was the PR that added the
deprecation callout.
This PR moves the test guide used for testing and dev work to the guided
onboarding package. It is now being registered when the guided
onboarding example plugin is being setup. For that a server side is
added to the example plugin.
Registering the test guide from the example plugin demonstrates how the
"register guide" function can be used for production guides. The test
config needs to be in the `kbn-guided-onboarding` package because
example plugins have some problems importing files from other plugins.
That way the test config is available to the example plugin and to the
guided onboarding plugin for unit and functional tests.
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Fixes an error where the `data-title` attribute in the `DiscoverGrid`
component was not getting updated when using a custom panel title via
the saved search embeddable `searchTitle` prop.
[This functional test was returning a false
positive](https://github.com/elastic/kibana/blob/main/test/functional/apps/dashboard/group5/data_shared_attributes.ts#L110-L124).
The previous test left a context menu open so the test was changing the
panel title of the first embeddable (a Lens embeddable) rather than the
saved search embeddable. As a result the bug in the saved search
embeddable was unnoticed.
Epic: https://github.com/elastic/kibana/issues/144943
## Summary
Moving the existing CellActions implementation to a new home. The
`kbn-cell-actions` package contains components and hooks that are going
to be used by solutions to show data cell actions with a consistent UI
across them.
Security Solution is going to start using it by migrating all
"hover-actions" to the unified implementation, but the usage is not
restricted to it. Any plugin can register and attach its own actions to
a trigger via uiActions, and use this package to render the CellActions
components in a consistent way.
The initial implementation was placed in the uiActions plugin itself due
to a types constraints
(https://github.com/elastic/kibana/tree/main/src/plugins/ui_actions/public/cell_actions),
the constraint has been solved so we are creating the package for it as
planned.
This PR only moves that implementation to the new package, with small
directory changes. The exported components are not being used anywhere
currently, so the implementation may change during the migration phase.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
