mirror of
https://github.com/elastic/kibana.git
synced 2025-04-24 17:59:23 -04:00
Your window into the Elastic Stack
55b66e20fe
Closes https://github.com/elastic/kibana/issues/140175
Closes https://github.com/elastic/kibana/issues/143580
## Summary
Oh, boy! Get ready for a doozy of a PR, folks! Let's talk about the
three major things that were accomplished here:
### 1) Pagination
Originally, this PR was meant to add traditional pagination to the
options list control. However, after implementing a version of this, it
became apparent that, not only was UI becoming uncomfortably messy, it
also had some UX concerns because we were deviating from the usual
pagination pattern by showing the cardinality rather than the number of
pages:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214687041-f8950d3a-2b29-41d5-b656-c79d9575d744.gif"/></p>
So, instead of traditional pagination, we decided to take a different
approach (which was made possible by
https://github.com/elastic/kibana/pull/148420) - **load more options
when the user scrolls to the bottom!** Here it is in action:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214688854-06c7e8a9-7b8c-4dc0-9846-00ccf5e5f771.gif"/></p>
It is important that the first query remains **fast** - that is why we
still only request the top 10 options when the control first loads. So,
having a "load more" is the best approach that allows users to see more
suggestions while also ensuring that the performance of options lists
(especially with respect to chaining) is not impacted.
Note that it is **not possible** to grab every single value of a field -
the limit is `10,000`. However, since it is impractical that a user
would want to scroll through `10,000` suggestions (and potentially very
slow to fetch), we have instead made the limit of this "show more"
functionality `1,000`. To make this clear, if the field has more than
`1,000` values and the user scrolls all the way to the bottom, they will
get the following message:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214920302-1e3574dc-f2b6-4845-be69-f9ba04177e7f.png"/></p>
### 2) Cardinality
Previously, the cardinality of the options list control was **only**
shown as part of the control placeholder text - this meant that, once
the user entered their search term, they could no longer see the
cardinality of the returned options. This PR changes this functionality
by placing the cardinality in a badge **beside** the search bar - this
value now changes as the user types, so they can very clearly see how
many options match their search:
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214689739-9670719c-5878-4e8b-806c-0b5a6f6f907f.gif"/></p>
> **Note**
> After some initial feedback, we have removed both the cardinality and
invalid selections badges in favour of displaying the cardinality below
the search bar, like so:
>
> <p align="center"><img
src="https://user-images.githubusercontent.com/8698078/216473930-e99366a3-86df-4777-a3d8-cf2d41e550fb.gif"/></p>
>
> So, please be aware that the screenshots above are outdated.
### 3) Changes to Queries
This is where things get.... messy! Essentially, our previous queries
were all built with the expectation that the Elasticsearch setting
`search.allow_expensive_queries` was **off** - this meant that they
worked regardless of the value of this setting. However, when trying to
get the cardinality to update based on a search term, it became apparent
that this was not possible if we kept the same assumptions -
specifically, if `search.allow_expensive_queries` is off, there is
absolutely no way for the cardinality of **keyword only fields** to
respond to a search term.
After a whole lot of discussion, we decided that the updating
cardinality was a feature important enough to justify having **two
separate versions** of the queries:
1. **Queries for when `search.allow_expensive_queries` is off**:
These are essentially the same as our old queries - however, since we
can safely assume that this setting is **usually** on (it defaults on,
and there is no UI to easily change it), we opted to simplify them a
bit.
First of all, we used to create a special object for tracking the
parent/child relationship of fields that are mapped as keyword+text -
this was so that, if a user created a control on these fields, we could
support case-insensitive search. We no longer do this - if
`search.allow_expensive_queries` is off and you create a control on a
text+keyword field, the search will be case sensitive. This helps clean
up our code quite a bit.
Second, we are no longer returning **any** cardinality. Since the
cardinality is now displayed as a badge beside the search bar, users
would expect that this value would change as they type - however, since
it's impossible to make this happen for keyword-only fields and to keep
behaviour consistent, we have opted to simply remove this badge when
`search.allow_expensive_queries` is off **regardless** of the field
type. So, there is no longer a need to include the `cardinality` query
when grabbing the suggestions.
Finally, we do not support "load more" when
`search.allow_expensive_queries` is off. While this would theoretically
be possible, because we are no longer grabbing the cardinality, we would
have to always fetch `1,000` results when the user loads more, even if
the true cardinality is much smaller. Again, we are pretty confident
that **more often than not**, the `search.allow_expensive_queries` is
on; therefore, we are choosing to favour developer experience in this
instance because the impact should be quite small.
2. **Queries for when `search.allow_expensive_queries` is on**:
When this setting is on, we now have access to the prefix query, which
greatly simplifies how our queries are handled - now, rather than having
separate queries for keyword-only, keyword+text, and nested fields,
these have all been combined into a single query! And even better -
⭐ now **all** string-based fields support case-insensitive search!
⭐ Yup, that's right - even keyword-only fields 💃
There has been [discussion on the Elasticsearch side
](https://github.com/elastic/elasticsearch/issues/90898) about whether
or not this setting is even **practical**, and so it is possible that,
in the near future, this distinction will no longer be necessary. With
this in mind, I have made these two versions of our queries **completely
separate** from each other - while this introduces some code
duplication, it makes the cleanup that may follow much, much easier.
Well, that was sure fun, hey?
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214921985-49058ff0-42f2-4b01-8ae3-0a4d259d1075.gif"/></p>
## How to Test
I've created a quick little Python program to ingest some good testing
data for this PR:
```python
import random
import time
import pandas as pd
from faker import Faker
from elasticsearch import Elasticsearch
SIZE = 10000
ELASTIC_PASSWORD = "changeme"
INDEX_NAME = 'test_large_index'
Faker.seed(time.time())
faker = Faker()
hundredRandomSentences = [faker.sentence(random.randint(5, 35)) for _ in range(100)]
thousandRandomIps = [faker.ipv4() if random.randint(0, 99) < 50 else faker.ipv6() for _ in range(1000)]
client = Elasticsearch(
"http://localhost:9200",
basic_auth=("elastic", ELASTIC_PASSWORD),
)
if(client.indices.exists(index=INDEX_NAME)):
client.indices.delete(index=INDEX_NAME)
client.indices.create(index=INDEX_NAME, mappings={"properties":{"keyword_field":{"type":"keyword"},"id":{"type":"long"},"ip_field":{"type":"ip"},"boolean_field":{"type":"boolean"},"keyword_text_field":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"nested_field":{"type":"nested","properties":{"first":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"last":{"type":"text","fields":{"keyword":{"type":"keyword"}}}}},"long_keyword_text_field":{"type":"text","fields":{"keyword":{"type":"keyword"}}}}})
print('Generating data', end='')
for i in range(SIZE):
name1 = faker.name();
[first_name1, last_name1] = name1.split(' ', 1)
name2 = faker.name();
[first_name2, last_name2] = name2.split(' ', 1)
response = client.create(index=INDEX_NAME, id=i, document={
'keyword_field': faker.country(),
'id': i,
'boolean_field': faker.boolean(),
'ip_field': thousandRandomIps[random.randint(0, 999)],
'keyword_text_field': faker.name(),
'nested_field': [
{ 'first': first_name1, 'last': last_name1},
{ 'first': first_name2, 'last': last_name2}
],
'long_keyword_text_field': hundredRandomSentences[random.randint(0, 99)]
})
print('.', end='')
print(' Done!')
```
However, if you don't have Python up and running, here's a CSV with a
smaller version of this data:
[testNewQueriesData.csv](
|
||
|---|---|---|
| .buildkite | ||
| .ci | ||
| .github | ||
| api_docs | ||
| config | ||
| dev_docs | ||
| docs | ||
| examples | ||
| kbn_pm | ||
| legacy_rfcs | ||
| licenses | ||
| packages | ||
| plugins | ||
| scripts | ||
| src | ||
| test | ||
| typings | ||
| vars | ||
| x-pack | ||
| .backportrc.json | ||
| .bazelignore | ||
| .bazeliskversion | ||
| .bazelrc | ||
| .bazelrc.common | ||
| .bazelversion | ||
| .browserslistrc | ||
| .editorconfig | ||
| .eslintignore | ||
| .eslintrc.js | ||
| .gitattributes | ||
| .gitignore | ||
| .i18nrc.json | ||
| .node-version | ||
| .npmrc | ||
| .nvmrc | ||
| .prettierignore | ||
| .prettierrc | ||
| .stylelintignore | ||
| .stylelintrc | ||
| .telemetryrc.json | ||
| .yarnrc | ||
| BUILD.bazel | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| FAQ.md | ||
| fleet_packages.json | ||
| github_checks_reporter.json | ||
| Jenkinsfile | ||
| kibana.d.ts | ||
| LICENSE.txt | ||
| nav-kibana-dev.docnav.json | ||
| NOTICE.txt | ||
| package.json | ||
| preinstall_check.js | ||
| README.md | ||
| renovate.json | ||
| RISK_MATRIX.mdx | ||
| SECURITY.md | ||
| STYLEGUIDE.mdx | ||
| tsconfig.base.json | ||
| tsconfig.browser.json | ||
| tsconfig.browser_bazel.json | ||
| tsconfig.json | ||
| TYPESCRIPT.md | ||
| versions.json | ||
| WORKSPACE.bazel | ||
| yarn.lock | ||
Kibana
Kibana is your window into the Elastic Stack. Specifically, it's a browser-based analytics and search dashboard for Elasticsearch.
- Getting Started
- Documentation
- Version Compatibility with Elasticsearch
- Questions? Problems? Suggestions?
Getting Started
If you just want to try Kibana out, check out the Elastic Stack Getting Started Page to give it a whirl.
If you're interested in diving a bit deeper and getting a taste of Kibana's capabilities, head over to the Kibana Getting Started Page.
Using a Kibana Release
If you want to use a Kibana release in production, give it a test run, or just play around:
- Download the latest version on the Kibana Download Page.
- Learn more about Kibana's features and capabilities on the Kibana Product Page.
- We also offer a hosted version of Kibana on our Cloud Service.
Building and Running Kibana, and/or Contributing Code
You might want to build Kibana locally to contribute some code, test out the latest features, or try out an open PR:
- CONTRIBUTING.md will help you get Kibana up and running.
- If you would like to contribute code, please follow our STYLEGUIDE.mdx.
- For all other questions, check out the FAQ.md and wiki.
Documentation
Visit Elastic.co for the full Kibana documentation.
For information about building the documentation, see the README in elastic/docs.
Version Compatibility with Elasticsearch
Ideally, you should be running Elasticsearch and Kibana with matching version numbers. If your Elasticsearch has an older version number or a newer major number than Kibana, then Kibana will fail to run. If Elasticsearch has a newer minor or patch number than Kibana, then the Kibana Server will log a warning.
Note: The version numbers below are only examples, meant to illustrate the relationships between different types of version numbers.
| Situation | Example Kibana version | Example ES version | Outcome |
|---|---|---|---|
| Versions are the same. | 7.15.1 | 7.15.1 | 💚 OK |
| ES patch number is newer. | 7.15.0 | 7.15.1 | ⚠️ Logged warning |
| ES minor number is newer. | 7.14.2 | 7.15.0 | ⚠️ Logged warning |
| ES major number is newer. | 7.15.1 | 8.0.0 | 🚫 Fatal error |
| ES patch number is older. | 7.15.1 | 7.15.0 | ⚠️ Logged warning |
| ES minor number is older. | 7.15.1 | 7.14.2 | 🚫 Fatal error |
| ES major number is older. | 8.0.0 | 7.15.1 | 🚫 Fatal error |
Questions? Problems? Suggestions?
- If you've found a bug or want to request a feature, please create a GitHub Issue. Please check to make sure someone else hasn't already created an issue for the same topic.
- Need help using Kibana? Ask away on our Kibana Discuss Forum and a fellow community member or Elastic engineer will be glad to help you out.