github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-24 01:38:56 -04:00
Code Issues Projects Releases Packages Wiki Activity

[Dashboard] [Controls] Load more options list suggestions on scroll (#148331)

Browse source 
Closes https://github.com/elastic/kibana/issues/140175
Closes https://github.com/elastic/kibana/issues/143580

## Summary

Oh, boy! Get ready for a doozy of a PR, folks! Let's talk about the
three major things that were accomplished here:

### 1) Pagination
Originally, this PR was meant to add traditional pagination to the
options list control. However, after implementing a version of this, it
became apparent that, not only was UI becoming uncomfortably messy, it
also had some UX concerns because we were deviating from the usual
pagination pattern by showing the cardinality rather than the number of
pages:
    
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214687041-f8950d3a-2b29-41d5-b656-c79d9575d744.gif"/></p>
    
So, instead of traditional pagination, we decided to take a different
approach (which was made possible by
https://github.com/elastic/kibana/pull/148420) - **load more options
when the user scrolls to the bottom!** Here it is in action:
    
<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214688854-06c7e8a9-7b8c-4dc0-9846-00ccf5e5f771.gif"/></p>

It is important that the first query remains **fast** - that is why we
still only request the top 10 options when the control first loads. So,
having a "load more" is the best approach that allows users to see more
suggestions while also ensuring that the performance of options lists
(especially with respect to chaining) is not impacted.

Note that it is **not possible** to grab every single value of a field -
the limit is `10,000`. However, since it is impractical that a user
would want to scroll through `10,000` suggestions (and potentially very
slow to fetch), we have instead made the limit of this "show more"
functionality `1,000`. To make this clear, if the field has more than
`1,000` values and the user scrolls all the way to the bottom, they will
get the following message:


<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214920302-1e3574dc-f2b6-4845-be69-f9ba04177e7f.png"/></p>


### 2) Cardinality
Previously, the cardinality of the options list control was **only**
shown as part of the control placeholder text - this meant that, once
the user entered their search term, they could no longer see the
cardinality of the returned options. This PR changes this functionality
by placing the cardinality in a badge **beside** the search bar - this
value now changes as the user types, so they can very clearly see how
many options match their search:

<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214689739-9670719c-5878-4e8b-806c-0b5a6f6f907f.gif"/></p>

> **Note**
> After some initial feedback, we have removed both the cardinality and
invalid selections badges in favour of displaying the cardinality below
the search bar, like so:
> 
> <p align="center"><img
src="https://user-images.githubusercontent.com/8698078/216473930-e99366a3-86df-4777-a3d8-cf2d41e550fb.gif"/></p>
> 
> So,  please be aware that the screenshots above are outdated.


### 3) Changes to Queries
This is where things get.... messy! Essentially, our previous queries
were all built with the expectation that the Elasticsearch setting
`search.allow_expensive_queries` was **off** - this meant that they
worked regardless of the value of this setting. However, when trying to
get the cardinality to update based on a search term, it became apparent
that this was not possible if we kept the same assumptions -
specifically, if `search.allow_expensive_queries` is off, there is
absolutely no way for the cardinality of **keyword only fields** to
respond to a search term.

After a whole lot of discussion, we decided that the updating
cardinality was a feature important enough to justify having **two
separate versions** of the queries:
1. **Queries for when `search.allow_expensive_queries` is off**:
These are essentially the same as our old queries - however, since we
can safely assume that this setting is **usually** on (it defaults on,
and there is no UI to easily change it), we opted to simplify them a
bit.
     
First of all, we used to create a special object for tracking the
parent/child relationship of fields that are mapped as keyword+text -
this was so that, if a user created a control on these fields, we could
support case-insensitive search. We no longer do this - if
`search.allow_expensive_queries` is off and you create a control on a
text+keyword field, the search will be case sensitive. This helps clean
up our code quite a bit.
     
Second, we are no longer returning **any** cardinality. Since the
cardinality is now displayed as a badge beside the search bar, users
would expect that this value would change as they type - however, since
it's impossible to make this happen for keyword-only fields and to keep
behaviour consistent, we have opted to simply remove this badge when
`search.allow_expensive_queries` is off **regardless** of the field
type. So, there is no longer a need to include the `cardinality` query
when grabbing the suggestions.

Finally, we do not support "load more" when
`search.allow_expensive_queries` is off. While this would theoretically
be possible, because we are no longer grabbing the cardinality, we would
have to always fetch `1,000` results when the user loads more, even if
the true cardinality is much smaller. Again, we are pretty confident
that **more often than not**, the `search.allow_expensive_queries` is
on; therefore, we are choosing to favour developer experience in this
instance because the impact should be quite small.
     
2. **Queries for when `search.allow_expensive_queries` is on**:
When this setting is on, we now have access to the prefix query, which
greatly simplifies how our queries are handled - now, rather than having
separate queries for keyword-only, keyword+text, and nested fields,
these have all been combined into a single query! And even better -
⭐ now **all** string-based fields support case-insensitive search!
⭐ Yup, that's right - even keyword-only fields 💃

There has been [discussion on the Elasticsearch side
](https://github.com/elastic/elasticsearch/issues/90898) about whether
or not this setting is even **practical**, and so it is possible that,
in the near future, this distinction will no longer be necessary. With
this in mind, I have made these two versions of our queries **completely
separate** from each other - while this introduces some code
duplication, it makes the cleanup that may follow much, much easier.

Well, that was sure fun, hey?

<p align="center"><img
src="https://user-images.githubusercontent.com/8698078/214921985-49058ff0-42f2-4b01-8ae3-0a4d259d1075.gif"/></p>


## How to Test
I've created a quick little Python program to ingest some good testing
data for this PR:

```python
import random
import time
import pandas as pd
from faker import Faker
from elasticsearch import Elasticsearch

SIZE = 10000
ELASTIC_PASSWORD = "changeme"
INDEX_NAME = 'test_large_index'

Faker.seed(time.time())
faker = Faker()
hundredRandomSentences = [faker.sentence(random.randint(5, 35)) for _ in range(100)]
thousandRandomIps = [faker.ipv4() if random.randint(0, 99) < 50 else faker.ipv6() for _ in range(1000)]

client = Elasticsearch(
    "http://localhost:9200",
    basic_auth=("elastic", ELASTIC_PASSWORD),
)

if(client.indices.exists(index=INDEX_NAME)):
    client.indices.delete(index=INDEX_NAME)
client.indices.create(index=INDEX_NAME, mappings={"properties":{"keyword_field":{"type":"keyword"},"id":{"type":"long"},"ip_field":{"type":"ip"},"boolean_field":{"type":"boolean"},"keyword_text_field":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"nested_field":{"type":"nested","properties":{"first":{"type":"text","fields":{"keyword":{"type":"keyword"}}},"last":{"type":"text","fields":{"keyword":{"type":"keyword"}}}}},"long_keyword_text_field":{"type":"text","fields":{"keyword":{"type":"keyword"}}}}})

print('Generating data', end='')
for i in range(SIZE):
    name1 = faker.name();
    [first_name1, last_name1] = name1.split(' ', 1)
    name2 = faker.name();
    [first_name2, last_name2] = name2.split(' ', 1)
    response = client.create(index=INDEX_NAME, id=i, document={
        'keyword_field': faker.country(),
        'id': i,
        'boolean_field': faker.boolean(),
        'ip_field': thousandRandomIps[random.randint(0, 999)],
        'keyword_text_field': faker.name(),
        'nested_field': [
            { 'first': first_name1, 'last': last_name1},
            { 'first': first_name2, 'last': last_name2}
        ],
        'long_keyword_text_field': hundredRandomSentences[random.randint(0, 99)]
    })
    print('.', end='')
print(' Done!')
```
However, if you don't have Python up and running, here's a CSV with a
smaller version of this data:
[testNewQueriesData.csv](10538537/testNewQueriesData.csv)

> **Warning**
> When uploading, make sure to update the mappings of the CSV data to
the mappings included as part of the Python script above (which you can
find as part of the `client.indices.create` call). You'll notice,
however, that **none of the CSV documents have a nested field**.
Unfortunately, there doesn't seem to be a way to able to ingest nested
data through uploading a CSV, so the above data does not include one -
in order to test the nested data type, you'd have to add some of your
own documents
>
> Here's a sample nested field document, for your convenience:
> ```json
> {
>     "keyword_field": "Russian Federation",
>     "id": 0,
>     "boolean_field": true,
>     "ip_field": "121.149.70.251",
>     "keyword_text_field": "Michael Foster",
>     "nested_field": [
>       {
>         "first": "Rachel",
>         "last": "Wright"
>       },
>       {
>         "first": "Gary",
>         "last": "Reyes"
>       }
>     ],
> "long_keyword_text_field": "Color hotel indicate appear since well
sure right yet individual easy often test enough left a usually
attention."
> }
> ```
> 

### Testing Notes
Because there are now two versions of the queries, thorough testing
should be done for both when `search.allow_expensive_queries` is `true`
and when it is `false` for every single field type that is currently
supported. Use the following call to the cluster settings API to toggle
this value back and forth:

```php
PUT _cluster/settings
{
  "transient": {
	"search.allow_expensive_queries": <value> // true or false
  }
}
```

You should pay super special attention to the behaviour that happens
when toggling this value from `true` to `false` - for example, consider
the following:
1. Ensure `search.allow_expensive_queries` is either `true` or
`undefined`
2. Create and save a dashboard with at least one options list control
3. Navigate to the console and set `search.allow_expensive_queries` to
`false` - **DO NOT REFRESH**
4. Go back to the dashboard
5. Open up the options list control you created in step 2
6. Fetch a new, uncached request, either by scrolling to the bottom and
fetching more (assuming these values aren't already in the cache) or by
performing a search with a string you haven't tried before
7. ⚠️ **The options list control _should_ have a fatal error** ⚠️<br>The
Elasticsearch server knows that `search.allow_expensive_queries` is now
`false` but, because we only fetch this value on the first load on the
client side, it has not yet been updated - this means the options list
service still tries to fetch the suggestions using the expensive version
of the queries despite the fact that Elasticsearch will now reject this
request. The most graceful way to handle this is to simply throw a fatal
error.
8. Refreshing the browser will make things sync up again and you should
now get the expected results when opening the options list control.

### Flaky Test Runner

<a
href="https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1845"><img
src="https://user-images.githubusercontent.com/8698078/215894267-97f07e59-6660-4117-bda7-18f63cb19af6.png"/></a>

### Checklist

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [x] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
     > **Note**
> Technically, it actually does - however, it is due to an [EUI
bug](https://github.com/elastic/eui/issues/6565) from adding the group
label to the bottom of the list.
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Hannah Mudge 2023-02-06 16:13:28 -07:00 • committed by GitHub
parent 78b16cbe12
commit 55b66e20fe
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
51 changed files with 2379 additions and 948 deletions
Download patch file Download diff file Expand all files Collapse all files

39
src/plugins/controls/common/options_list/types.ts
View file

@ -9,8 +9,8 @@
import { FieldSpec, DataView, RuntimeFieldSpec } from '@kbn/data-views-plugin/common';
import type { Filter, Query, BoolQuery, TimeRange } from '@kbn/es-query';
import { OptionsListSortingType } from './suggestions_sorting';
import { DataControlInput } from '../types';
import type { OptionsListSortingType } from './suggestions_sorting';
import type { DataControlInput } from '../types';
export const OPTIONS_LIST_CONTROL = 'optionsListControl';
@ -20,20 +20,14 @@ export interface OptionsListEmbeddableInput extends DataControlInput {
existsSelected?: boolean;
runPastTimeout?: boolean;
singleSelect?: boolean;
hideActionBar?: boolean;
hideExclude?: boolean;
hideExists?: boolean;
hideSort?: boolean;
hideActionBar?: boolean;
exclude?: boolean;
placeholder?: string;
}
export type OptionsListField = FieldSpec & {
textFieldName?: string;
parentFieldName?: string;
childFieldName?: string;
};
export interface OptionsListSuggestions {
[key: string]: { doc_count: number };
}
@ -41,13 +35,27 @@ export interface OptionsListSuggestions {
/**
* The Options list response is returned from the serverside Options List route.
*/
export interface OptionsListResponse {
rejected: boolean;
export interface OptionsListSuccessResponse {
suggestions: OptionsListSuggestions;
totalCardinality: number;
totalCardinality?: number; // total cardinality will be undefined when `useExpensiveQueries` is `false`
invalidSelections?: string[];
}
/**
* The invalid selections are parsed **after** the server returns with the result from the ES client; so, the
* suggestion aggregation parser only returns the suggestions list + the cardinality of the result
*/
export type OptionsListParsedSuggestions = Pick<
OptionsListSuccessResponse,
'suggestions' | 'totalCardinality'
>;
export interface OptionsListFailureResponse {
error: 'aborted' | Error;
}
export type OptionsListResponse = OptionsListSuccessResponse | OptionsListFailureResponse;
/**
* The Options list request type taken in by the public Options List service.
*/
@ -55,11 +63,12 @@ export type OptionsListRequest = Omit<
OptionsListRequestBody,
'filters' | 'fieldName' | 'fieldSpec' | 'textFieldName'
> & {
allowExpensiveQueries: boolean;
timeRange?: TimeRange;
field: OptionsListField;
runPastTimeout?: boolean;
dataView: DataView;
filters?: Filter[];
field: FieldSpec;
query?: Query;
};
@ -68,13 +77,13 @@ export type OptionsListRequest = Omit<
*/
export interface OptionsListRequestBody {
runtimeFieldMap?: Record<string, RuntimeFieldSpec>;
allowExpensiveQueries: boolean;
sort?: OptionsListSortingType;
filters?: Array<{ bool: BoolQuery }>;
selectedOptions?: string[];
runPastTimeout?: boolean;
parentFieldName?: string;
textFieldName?: string;
searchString?: string;
fieldSpec?: FieldSpec;
fieldName: string;
size: number;
}

2
src/plugins/controls/common/types.ts
View file

@ -30,7 +30,5 @@ export type ControlInput = EmbeddableInput & {
export type DataControlInput = ControlInput & {
fieldName: string;
parentFieldName?: string;
childFieldName?: string;
dataViewId: string;
};

1
src/plugins/controls/public/__stories__/controls.stories.tsx
View file

@ -61,7 +61,6 @@ const storybookStubOptionsListRequest = async (
{}
),
totalCardinality: 100,
rejected: false,
}),
120
)

31
src/plugins/controls/public/control_group/component/control_frame_component.tsx
View file

@ -9,15 +9,13 @@
import React, { useEffect, useMemo, useState } from 'react';
import classNames from 'classnames';
import {
EuiButtonEmpty,
EuiButtonIcon,
EuiFormControlLayout,
EuiFormLabel,
EuiFormRow,
EuiIcon,
EuiLink,
EuiLoadingChart,
EuiPopover,
EuiText,
EuiToolTip,
} from '@elastic/eui';
@ -40,25 +38,26 @@ interface ControlFrameErrorProps {
const ControlFrameError = ({ error }: ControlFrameErrorProps) => {
const [isPopoverOpen, setPopoverOpen] = useState(false);
const popoverButton = (
<EuiText className="errorEmbeddableCompact__button" size="xs">
<EuiLink
className="eui-textTruncate"
color="subdued"
onClick={() => setPopoverOpen((open) => !open)}
>
<EuiIcon type="alert" color="danger" />
<FormattedMessage
id="controls.frame.error.message"
defaultMessage="An error has occurred. Read more"
/>
</EuiLink>
</EuiText>
<EuiButtonEmpty
color="danger"
iconSize="m"
iconType={'alert'}
onClick={() => setPopoverOpen((open) => !open)}
className={'errorEmbeddableCompact__button'}
textProps={{ className: 'errorEmbeddableCompact__text' }}
>
<FormattedMessage
id="controls.frame.error.message"
defaultMessage="An error occurred. View more"
/>
</EuiButtonEmpty>
);
return (
<EuiPopover
button={popoverButton}
isOpen={isPopoverOpen}
className="errorEmbeddableCompact__popover"
anchorClassName="errorEmbeddableCompact__popoverAnchor"
closePopover={() => setPopoverOpen(false)}
>

3
src/plugins/controls/public/control_group/editor/control_editor.tsx
View file

@ -199,11 +199,8 @@ export const ControlEditor = ({
selectedFieldName={selectedField}
dataView={dataView}
onSelectField={(field) => {
const { parentFieldName, childFieldName } = fieldRegistry?.[field.name] ?? {};
onTypeEditorChange({
fieldName: field.name,
...(parentFieldName && { parentFieldName }),
...(childFieldName && { childFieldName }),
});
const newDefaultTitle = field.displayName ?? field.name;
setDefaultTitle(newDefaultTitle);

53
src/plugins/controls/public/control_group/editor/data_control_editor_tools.ts
View file

@ -8,11 +8,10 @@
import { memoize } from 'lodash';
import { IFieldSubTypeMulti } from '@kbn/es-query';
import { DataView } from '@kbn/data-views-plugin/common';
import { pluginServices } from '../../services';
import { DataControlFieldRegistry, IEditableControlFactory } from '../../types';
import { DataControlField, DataControlFieldRegistry, IEditableControlFactory } from '../../types';
export const getDataControlFieldRegistry = memoize(
async (dataView: DataView) => {
@ -21,50 +20,30 @@ export const getDataControlFieldRegistry = memoize(
(dataView: DataView) => [dataView.id, JSON.stringify(dataView.fields.getAll())].join('|')
);
const doubleLinkFields = (dataView: DataView) => {
// double link the parent-child relationship specifically for case-sensitivity support for options lists
const fieldRegistry: DataControlFieldRegistry = {};
for (const field of dataView.fields.getAll()) {
if (!fieldRegistry[field.name]) {
fieldRegistry[field.name] = { field, compatibleControlTypes: [] };
}
const parentFieldName = (field.subType as IFieldSubTypeMulti)?.multi?.parent;
if (parentFieldName) {
fieldRegistry[field.name].parentFieldName = parentFieldName;
const parentField = dataView.getFieldByName(parentFieldName);
if (!fieldRegistry[parentFieldName] && parentField) {
fieldRegistry[parentFieldName] = { field: parentField, compatibleControlTypes: [] };
}
fieldRegistry[parentFieldName].childFieldName = field.name;
}
}
return fieldRegistry;
};
const loadFieldRegistryFromDataView = async (
dataView: DataView
): Promise<DataControlFieldRegistry> => {
const {
controls: { getControlTypes, getControlFactory },
} = pluginServices.getServices();
const newFieldRegistry: DataControlFieldRegistry = doubleLinkFields(dataView);
const controlFactories = getControlTypes().map(
(controlType) => getControlFactory(controlType) as IEditableControlFactory
);
dataView.fields.map((dataViewField) => {
for (const factory of controlFactories) {
if (factory.isFieldCompatible) {
factory.isFieldCompatible(newFieldRegistry[dataViewField.name]);
const fieldRegistry: DataControlFieldRegistry = dataView.fields
.getAll()
.reduce((registry, field) => {
const test: DataControlField = { field, compatibleControlTypes: [] };
for (const factory of controlFactories) {
if (factory.isFieldCompatible) {
factory.isFieldCompatible(test);
}
}
}
if (test.compatibleControlTypes.length === 0) {
return { ...registry };
}
return { ...registry, [field.name]: test };
}, {});
if (newFieldRegistry[dataViewField.name]?.compatibleControlTypes.length === 0) {
delete newFieldRegistry[dataViewField.name];
}
});
return newFieldRegistry;
return fieldRegistry;
};

31
src/plugins/controls/public/options_list/components/options_list.scss
View file

@ -9,8 +9,18 @@
.optionsList__actions {
padding: $euiSizeS;
padding-bottom: 0;
border-bottom: $euiBorderThin;
border-color: darken($euiColorLightestShade, 2%);
.optionsList__actionsRow {
margin: ($euiSizeS / 2) 0 !important;
.optionsList__actionBarDivider {
height: $euiSize;
border-right: $euiBorderThin;
}
}
}
.optionsList__popoverTitle {
@ -30,13 +40,17 @@
font-style: italic;
}
.optionsList__loadMore {
font-style: italic;
}
.optionsList__negateLabel {
font-weight: bold;
font-size: $euiSizeM;
color: $euiColorDanger;
}
.optionsList__ignoredBadge {
.optionsList__actionBarFirstBadge {
margin-left: $euiSizeS;
}
@ -86,3 +100,18 @@
.optionsList--sortPopover {
width: $euiSizeXL * 7;
}
.optionslist--loadingMoreGroupLabel {
text-align: center;
padding: $euiSizeM;
font-style: italic;
height: $euiSizeXXL !important;
}
.optionslist--endOfOptionsGroupLabel {
text-align: center;
font-size: $euiSizeM;
height: auto !important;
color: $euiTextSubduedColor;
padding: $euiSizeM;
}

1
src/plugins/controls/public/options_list/components/options_list_control.test.tsx
View file

@ -20,6 +20,7 @@ import { BehaviorSubject } from 'rxjs';
describe('Options list control', () => {
const defaultProps = {
typeaheadSubject: new BehaviorSubject(''),
loadMoreSubject: new BehaviorSubject(10),
};
interface MountOptions {

33
src/plugins/controls/public/options_list/components/options_list_control.tsx
View file

@ -17,20 +17,24 @@ import { useReduxEmbeddableContext } from '@kbn/presentation-util-plugin/public'
import { OptionsListStrings } from './options_list_strings';
import { OptionsListPopover } from './options_list_popover';
import { optionsListReducers } from '../options_list_reducers';
import { OptionsListReduxState } from '../types';
import { MAX_OPTIONS_LIST_REQUEST_SIZE, OptionsListReduxState } from '../types';
import './options_list.scss';
export const OptionsListControl = ({ typeaheadSubject }: { typeaheadSubject: Subject<string> }) => {
const [isPopoverOpen, setIsPopoverOpen] = useState(false);
export const OptionsListControl = ({
typeaheadSubject,
loadMoreSubject,
}: {
typeaheadSubject: Subject<string>;
loadMoreSubject: Subject<number>;
}) => {
const resizeRef = useRef(null);
const dimensions = useResizeObserver(resizeRef.current);
// Redux embeddable Context
const {
useEmbeddableDispatch,
actions: { replaceSelection, setSearchString },
actions: { replaceSelection, setSearchString, setPopoverOpen },
useEmbeddableSelector: select,
} = useReduxEmbeddableContext<OptionsListReduxState, typeof optionsListReducers>();
const dispatch = useEmbeddableDispatch();
@ -38,6 +42,7 @@ export const OptionsListControl = ({ typeaheadSubject }: { typeaheadSubject: Sub
// Select current state from Redux using multiple selectors to avoid rerenders.
const invalidSelections = select((state) => state.componentState.invalidSelections);
const validSelections = select((state) => state.componentState.validSelections);
const isPopoverOpen = select((state) => state.componentState.popoverOpen);
const selectedOptions = select((state) => state.explicitInput.selectedOptions);
const existsSelected = select((state) => state.explicitInput.existsSelected);
@ -51,6 +56,12 @@ export const OptionsListControl = ({ typeaheadSubject }: { typeaheadSubject: Sub
const loading = select((state) => state.output.loading);
useEffect(() => {
return () => {
dispatch(setPopoverOpen(false)); // on unmount, close the popover
};
}, [dispatch, setPopoverOpen]);
// debounce loading state so loading doesn't flash when user types
const [debouncedLoading, setDebouncedLoading] = useState(true);
const debounceSetLoading = useMemo(
@ -77,6 +88,13 @@ export const OptionsListControl = ({ typeaheadSubject }: { typeaheadSubject: Sub
[typeaheadSubject, dispatch, setSearchString]
);
const loadMoreSuggestions = useCallback(
(cardinality: number) => {
loadMoreSubject.next(Math.min(cardinality, MAX_OPTIONS_LIST_REQUEST_SIZE));
},
[loadMoreSubject]
);
const { hasSelections, selectionDisplayNode, validSelectionsCount } = useMemo(() => {
return {
hasSelections: !isEmpty(validSelections) || !isEmpty(invalidSelections),
@ -123,7 +141,7 @@ export const OptionsListControl = ({ typeaheadSubject }: { typeaheadSubject: Sub
'optionsList--filterBtnPlaceholder': !hasSelections,
})}
data-test-subj={`optionsList-control-${id}`}
onClick={() => setIsPopoverOpen((openState) => !openState)}
onClick={() => dispatch(setPopoverOpen(!isPopoverOpen))}
isSelected={isPopoverOpen}
numActiveFilters={validSelectionsCount}
hasActiveFilters={Boolean(validSelectionsCount)}
@ -149,7 +167,7 @@ export const OptionsListControl = ({ typeaheadSubject }: { typeaheadSubject: Sub
panelPaddingSize="none"
anchorPosition="downCenter"
className="optionsList__popoverOverride"
closePopover={() => setIsPopoverOpen(false)}
closePopover={() => dispatch(setPopoverOpen(false))}
anchorClassName="optionsList__anchorOverride"
aria-label={OptionsListStrings.popover.getAriaLabel(fieldName)}
>
@ -157,6 +175,7 @@ export const OptionsListControl = ({ typeaheadSubject }: { typeaheadSubject: Sub
width={dimensions.width}
isLoading={debouncedLoading}
updateSearchString={updateSearchString}
loadMoreSuggestions={loadMoreSuggestions}
/>
</EuiPopover>
</EuiFilterGroup>

30
src/plugins/controls/public/options_list/components/options_list_popover.test.tsx
View file

@ -11,11 +11,11 @@ import { ReactWrapper } from 'enzyme';
import { mountWithIntl } from '@kbn/test-jest-helpers';
import { findTestSubject } from '@elastic/eui/lib/test';
import { FieldSpec } from '@kbn/data-views-plugin/common';
import { OptionsListPopover, OptionsListPopoverProps } from './options_list_popover';
import { OptionsListComponentState, OptionsListReduxState } from '../types';
import { mockOptionsListReduxEmbeddableTools } from '../../../common/mocks';
import { OptionsListField } from '../../../common/options_list/types';
import { ControlOutput, OptionsListEmbeddableInput } from '../..';
describe('Options list popover', () => {
@ -23,6 +23,7 @@ describe('Options list popover', () => {
width: 500,
isLoading: false,
updateSearchString: jest.fn(),
loadMoreSuggestions: jest.fn(),
};
interface MountOptions {
@ -63,7 +64,7 @@ describe('Options list popover', () => {
// the div cannot be smaller than 301 pixels wide
popover = await mountComponent({ popoverProps: { width: 300 } });
popoverDiv = findTestSubject(popover, 'optionsList-control-available-options');
expect(popoverDiv.getDOMNode().getAttribute('style')).toBe(null);
expect(popoverDiv.getDOMNode().getAttribute('style')).toBe('width: 100%; height: 100%;');
});
test('no available options', async () => {
@ -237,7 +238,7 @@ describe('Options list popover', () => {
test('when sorting suggestions, show both sorting types for keyword field', async () => {
const popover = await mountComponent({
componentState: {
field: { name: 'Test keyword field', type: 'keyword' } as OptionsListField,
field: { name: 'Test keyword field', type: 'keyword' } as FieldSpec,
},
});
const sortButton = findTestSubject(popover, 'optionsListControl__sortingOptionsButton');
@ -252,7 +253,7 @@ describe('Options list popover', () => {
const popover = await mountComponent({
explicitInput: { sort: { by: '_key', direction: 'asc' } },
componentState: {
field: { name: 'Test keyword field', type: 'keyword' } as OptionsListField,
field: { name: 'Test keyword field', type: 'keyword' } as FieldSpec,
},
});
const sortButton = findTestSubject(popover, 'optionsListControl__sortingOptionsButton');
@ -270,7 +271,7 @@ describe('Options list popover', () => {
test('when sorting suggestions, only show document count sorting for IP fields', async () => {
const popover = await mountComponent({
componentState: { field: { name: 'Test IP field', type: 'ip' } as OptionsListField },
componentState: { field: { name: 'Test IP field', type: 'ip' } as FieldSpec },
});
const sortButton = findTestSubject(popover, 'optionsListControl__sortingOptionsButton');
sortButton.simulate('click');
@ -280,6 +281,25 @@ describe('Options list popover', () => {
expect(optionsText).toEqual(['By document count - Checked option.']);
});
test('ensure warning icon does not show up when testAllowExpensiveQueries = true/undefined', async () => {
const popover = await mountComponent({
componentState: { field: { name: 'Test keyword field', type: 'keyword' } as FieldSpec },
});
const warning = findTestSubject(popover, 'optionsList-allow-expensive-queries-warning');
expect(warning).toEqual({});
});
test('ensure warning icon shows up when testAllowExpensiveQueries = false', async () => {
const popover = await mountComponent({
componentState: {
field: { name: 'Test keyword field', type: 'keyword' } as FieldSpec,
allowExpensiveQueries: false,
},
});
const warning = findTestSubject(popover, 'optionsList-allow-expensive-queries-warning');
expect(warning.getDOMNode()).toBeInstanceOf(HTMLDivElement);
});
describe('Test advanced settings', () => {
const ensureComponentIsHidden = async ({
explicitInput,

56
src/plugins/controls/public/options_list/components/options_list_popover.tsx
View file

@ -9,12 +9,12 @@
import React, { useState } from 'react';
import { isEmpty } from 'lodash';
import { EuiPopoverTitle } from '@elastic/eui';
import { useReduxEmbeddableContext } from '@kbn/presentation-util-plugin/public';
import { OptionsListReduxState } from '../types';
import { OptionsListStrings } from './options_list_strings';
import { optionsListReducers } from '../options_list_reducers';
import { OptionsListPopoverTitle } from './options_list_popover_title';
import { OptionsListPopoverFooter } from './options_list_popover_footer';
import { OptionsListPopoverActionBar } from './options_list_popover_action_bar';
import { OptionsListPopoverSuggestions } from './options_list_popover_suggestions';
@ -23,6 +23,7 @@ import { OptionsListPopoverInvalidSelections } from './options_list_popover_inva
export interface OptionsListPopoverProps {
width: number;
isLoading: boolean;
loadMoreSuggestions: (cardinality: number) => void;
updateSearchString: (newSearchString: string) => void;
}
@ -30,6 +31,7 @@ export const OptionsListPopover = ({
width,
isLoading,
updateSearchString,
loadMoreSuggestions,
}: OptionsListPopoverProps) => {
// Redux embeddable container Context
const { useEmbeddableSelector: select } = useReduxEmbeddableContext<
@ -42,39 +44,45 @@ export const OptionsListPopover = ({
const availableOptions = select((state) => state.componentState.availableOptions);
const field = select((state) => state.componentState.field);
const hideExclude = select((state) => state.explicitInput.hideExclude);
const hideActionBar = select((state) => state.explicitInput.hideActionBar);
const hideExclude = select((state) => state.explicitInput.hideExclude);
const fieldName = select((state) => state.explicitInput.fieldName);
const title = select((state) => state.explicitInput.title);
const id = select((state) => state.explicitInput.id);
const [showOnlySelected, setShowOnlySelected] = useState(false);
return (
<div
id={`control-popover-${id}`}
style={{ width, minWidth: 300 }}
data-test-subj={`optionsList-control-popover`}
aria-label={OptionsListStrings.popover.getAriaLabel(fieldName)}
>
<EuiPopoverTitle paddingSize="s">{title}</EuiPopoverTitle>
{field?.type !== 'boolean' && !hideActionBar && (
<OptionsListPopoverActionBar
showOnlySelected={showOnlySelected}
setShowOnlySelected={setShowOnlySelected}
updateSearchString={updateSearchString}
/>
)}
<>
<div
data-test-subj={`optionsList-control-available-options`}
data-option-count={isLoading ? 0 : Object.keys(availableOptions ?? {}).length}
id={`control-popover-${id}`}
style={{ width, minWidth: 300 }}
data-test-subj={`optionsList-control-popover`}
aria-label={OptionsListStrings.popover.getAriaLabel(fieldName)}
>
<OptionsListPopoverSuggestions showOnlySelected={showOnlySelected} isLoading={isLoading} />
{!showOnlySelected && invalidSelections && !isEmpty(invalidSelections) && (
<OptionsListPopoverInvalidSelections />
<OptionsListPopoverTitle />
{field?.type !== 'boolean' && !hideActionBar && (
<OptionsListPopoverActionBar
showOnlySelected={showOnlySelected}
updateSearchString={updateSearchString}
setShowOnlySelected={setShowOnlySelected}
/>
)}
<div
data-test-subj={`optionsList-control-available-options`}
data-option-count={isLoading ? 0 : Object.keys(availableOptions ?? {}).length}
style={{ width: '100%', height: '100%' }}
>
<OptionsListPopoverSuggestions
loadMoreSuggestions={loadMoreSuggestions}
showOnlySelected={showOnlySelected}
/>
{!showOnlySelected && invalidSelections && !isEmpty(invalidSelections) && (
<OptionsListPopoverInvalidSelections />
)}
</div>
{!hideExclude && <OptionsListPopoverFooter isLoading={isLoading} />}
</div>
{!hideExclude && <OptionsListPopoverFooter />}
</div>
</>
);
};

152
src/plugins/controls/public/options_list/components/options_list_popover_action_bar.tsx
View file

@ -15,7 +15,7 @@ import {
EuiFlexItem,
EuiFormRow,
EuiToolTip,
EuiBadge,
EuiText,
} from '@elastic/eui';
import { useReduxEmbeddableContext } from '@kbn/presentation-util-plugin/public';
@ -26,8 +26,8 @@ import { OptionsListPopoverSortingButton } from './options_list_popover_sorting_
interface OptionsListPopoverProps {
showOnlySelected: boolean;
setShowOnlySelected: (value: boolean) => void;
updateSearchString: (newSearchString: string) => void;
setShowOnlySelected: (value: boolean) => void;
}
export const OptionsListPopoverActionBar = ({
@ -44,23 +44,17 @@ export const OptionsListPopoverActionBar = ({
const dispatch = useEmbeddableDispatch();
// Select current state from Redux using multiple selectors to avoid rerenders.
const allowExpensiveQueries = select((state) => state.componentState.allowExpensiveQueries);
const invalidSelections = select((state) => state.componentState.invalidSelections);
const totalCardinality = select((state) => state.componentState.totalCardinality);
const totalCardinality = select((state) => state.componentState.totalCardinality) ?? 0;
const searchString = select((state) => state.componentState.searchString);
const hideSort = select((state) => state.explicitInput.hideSort);
return (
<div className="optionsList__actions">
<EuiFormRow fullWidth>
<EuiFlexGroup
gutterSize="xs"
direction="row"
justifyContent="spaceBetween"
alignItems="center"
responsive={false}
>
<EuiFlexItem>
<EuiFlexGroup gutterSize="xs" alignItems="center" responsive={false}>
<EuiFlexItem grow={true}>
<EuiFieldSearch
isInvalid={!searchString.valid}
compressed
@ -69,71 +63,93 @@ export const OptionsListPopoverActionBar = ({
onChange={(event) => updateSearchString(event.target.value)}
value={searchString.value}
data-test-subj="optionsList-control-search-input"
placeholder={
totalCardinality
? OptionsListStrings.popover.getTotalCardinalityPlaceholder(totalCardinality)
: undefined
}
placeholder={OptionsListStrings.popover.getSearchPlaceholder()}
autoFocus={true}
/>
</EuiFlexItem>
<EuiFlexItem grow={false}>
{(invalidSelections?.length ?? 0) > 0 && (
<EuiToolTip
content={OptionsListStrings.popover.getInvalidSelectionsTooltip(
invalidSelections?.length ?? 0
)}
>
<EuiBadge className="optionsList__ignoredBadge" color="warning">
{invalidSelections?.length}
</EuiBadge>
</EuiToolTip>
)}
</EuiFlexItem>
{!hideSort && (
<EuiFlexItem grow={false}>
<OptionsListPopoverSortingButton showOnlySelected={showOnlySelected} />
</EuiFlexItem>
)}
<EuiFlexItem grow={false}>
<EuiToolTip
position="top"
content={
showOnlySelected
? OptionsListStrings.popover.getAllOptionsButtonTitle()
: OptionsListStrings.popover.getSelectedOptionsButtonTitle()
}
</EuiFlexGroup>
</EuiFormRow>
<EuiFormRow className="optionsList__actionsRow" fullWidth>
<EuiFlexGroup
justifyContent="spaceBetween"
alignItems="center"
gutterSize="s"
responsive={false}
>
{allowExpensiveQueries && (
<EuiFlexItem grow={false}>
<EuiText size="xs" color="subdued" data-test-subj="optionsList-cardinality-label">
{OptionsListStrings.popover.getCardinalityLabel(totalCardinality)}
</EuiText>
</EuiFlexItem>
)}
{invalidSelections && invalidSelections.length > 0 && (
<>
{allowExpensiveQueries && (
<EuiFlexItem grow={false}>
<div className="optionsList__actionBarDivider" />
</EuiFlexItem>
)}
<EuiFlexItem grow={false}>
<EuiText size="xs" color="subdued">
{OptionsListStrings.popover.getInvalidSelectionsLabel(invalidSelections.length)}
</EuiText>
</EuiFlexItem>
</>
)}
<EuiFlexItem grow={true}>
<EuiFlexGroup
gutterSize="xs"
alignItems="center"
justifyContent="flexEnd"
responsive={false}
>
<EuiButtonIcon
size="s"
iconType="list"
aria-pressed={showOnlySelected}
color={showOnlySelected ? 'primary' : 'text'}
display={showOnlySelected ? 'base' : 'empty'}
onClick={() => setShowOnlySelected(!showOnlySelected)}
data-test-subj="optionsList-control-show-only-selected"
aria-label={
showOnlySelected
? OptionsListStrings.popover.getAllOptionsButtonTitle()
: OptionsListStrings.popover.getSelectedOptionsButtonTitle()
}
/>
</EuiToolTip>
</EuiFlexItem>
<EuiFlexItem grow={false}>
<EuiToolTip
position="top"
content={OptionsListStrings.popover.getClearAllSelectionsButtonTitle()}
>
<EuiButtonIcon
size="s"
color="danger"
iconType="eraser"
onClick={() => dispatch(clearSelections({}))}
data-test-subj="optionsList-control-clear-all-selections"
aria-label={OptionsListStrings.popover.getClearAllSelectionsButtonTitle()}
/>
</EuiToolTip>
<EuiFlexItem grow={false}>
<EuiToolTip
position="top"
content={
showOnlySelected
? OptionsListStrings.popover.getAllOptionsButtonTitle()
: OptionsListStrings.popover.getSelectedOptionsButtonTitle()
}
>
<EuiButtonIcon
size="xs"
iconType="list"
aria-pressed={showOnlySelected}
color={showOnlySelected ? 'primary' : 'text'}
display={showOnlySelected ? 'base' : 'empty'}
onClick={() => setShowOnlySelected(!showOnlySelected)}
data-test-subj="optionsList-control-show-only-selected"
aria-label={
showOnlySelected
? OptionsListStrings.popover.getAllOptionsButtonTitle()
: OptionsListStrings.popover.getSelectedOptionsButtonTitle()
}
/>
</EuiToolTip>
</EuiFlexItem>
<EuiFlexItem grow={false}>
<EuiToolTip
position="top"
content={OptionsListStrings.popover.getClearAllSelectionsButtonTitle()}
>
<EuiButtonIcon
size="xs"
color="danger"
iconType="eraser"
onClick={() => dispatch(clearSelections({}))}
data-test-subj="optionsList-control-clear-all-selections"
aria-label={OptionsListStrings.popover.getClearAllSelectionsButtonTitle()}
/>
</EuiToolTip>
</EuiFlexItem>
</EuiFlexGroup>
</EuiFlexItem>
</EuiFlexGroup>
</EuiFormRow>

41
src/plugins/controls/public/options_list/components/options_list_popover_footer.tsx
View file

@ -7,7 +7,13 @@
*/
import React from 'react';
import { EuiPopoverFooter, EuiButtonGroup, useEuiBackgroundColor } from '@elastic/eui';
import {
useEuiBackgroundColor,
useEuiPaddingSize,
EuiPopoverFooter,
EuiButtonGroup,
EuiProgress,
} from '@elastic/eui';
import { css } from '@emotion/react';
import { useReduxEmbeddableContext } from '@kbn/presentation-util-plugin/public';
@ -26,7 +32,7 @@ const aggregationToggleButtons = [
},
];
export const OptionsListPopoverFooter = () => {
export const OptionsListPopoverFooter = ({ isLoading }: { isLoading: boolean }) => {
// Redux embeddable container Context
const {
useEmbeddableDispatch,
@ -41,19 +47,32 @@ export const OptionsListPopoverFooter = () => {
return (
<>
<EuiPopoverFooter
paddingSize="s"
paddingSize="none"
css={css`
background-color: ${useEuiBackgroundColor('subdued')};
`}
>
<EuiButtonGroup
legend={OptionsListStrings.popover.getIncludeExcludeLegend()}
options={aggregationToggleButtons}
idSelected={exclude ? 'optionsList__excludeResults' : 'optionsList__includeResults'}
onChange={(optionId) => dispatch(setExclude(optionId === 'optionsList__excludeResults'))}
buttonSize="compressed"
data-test-subj="optionsList__includeExcludeButtonGroup"
/>
{isLoading && (
<div style={{ position: 'absolute', width: '100%' }}>
<EuiProgress size="xs" color="accent" />
</div>
)}
<div
css={css`
padding: ${useEuiPaddingSize('s')};
`}
>
<EuiButtonGroup
legend={OptionsListStrings.popover.getIncludeExcludeLegend()}
options={aggregationToggleButtons}
idSelected={exclude ? 'optionsList__excludeResults' : 'optionsList__includeResults'}
onChange={(optionId) =>
dispatch(setExclude(optionId === 'optionsList__excludeResults'))
}
buttonSize="compressed"
data-test-subj="optionsList__includeExcludeButtonGroup"
/>
</div>
</EuiPopoverFooter>
</>
);

43
src/plugins/controls/public/options_list/components/options_list_popover_sorting_button.tsx
View file

@ -12,14 +12,14 @@ import {
EuiButtonGroupOptionProps,
EuiSelectableOption,
EuiPopoverTitle,
EuiButtonEmpty,
EuiButtonGroup,
EuiButtonIcon,
EuiSelectable,
EuiFlexGroup,
EuiFlexItem,
EuiToolTip,
EuiPopover,
Direction,
EuiToolTip,
} from '@elastic/eui';
import { useReduxEmbeddableContext } from '@kbn/presentation-util-plugin/public';
@ -91,25 +91,32 @@ export const OptionsListPopoverSortingButton = ({
}
};
const SortButton = () => (
<EuiButtonEmpty
size="s"
color="text"
iconSide="right"
iconType="arrowDown"
disabled={showOnlySelected}
data-test-subj="optionsListControl__sortingOptionsButton"
onClick={() => setIsSortingPopoverOpen(!isSortingPopoverOpen)}
className="euiFilterGroup" // this gives the button a nice border
aria-label={OptionsListStrings.popover.getSortPopoverDescription()}
>
{OptionsListStrings.popover.getSortPopoverTitle()}
</EuiButtonEmpty>
);
return (
<EuiPopover
button={
<EuiToolTip
position="top"
content={
showOnlySelected
? OptionsListStrings.popover.getSortDisabledTooltip()
: OptionsListStrings.popover.getSortPopoverDescription()
}
>
<EuiButtonIcon
iconType="sortable"
disabled={showOnlySelected}
data-test-subj="optionsListControl__sortingOptionsButton"
onClick={() => setIsSortingPopoverOpen(!isSortingPopoverOpen)}
aria-label={OptionsListStrings.popover.getSortPopoverDescription()}
/>
</EuiToolTip>
showOnlySelected ? (
<EuiToolTip position="top" content={OptionsListStrings.popover.getSortDisabledTooltip()}>
<SortButton />
</EuiToolTip>
) : (
<SortButton />
)
}
panelPaddingSize="none"
isOpen={isSortingPopoverOpen}

140
src/plugins/controls/public/options_list/components/options_list_popover_suggestions.tsx
View file

@ -6,26 +6,27 @@
* Side Public License, v 1.
*/
import React, { useEffect, useMemo, useState } from 'react';
import React, { useCallback, useEffect, useMemo, useRef, useState } from 'react';
import { EuiLoadingSpinner, EuiSelectable, EuiSpacer } from '@elastic/eui';
import { euiThemeVars } from '@kbn/ui-theme';
import { EuiSelectable } from '@elastic/eui';
import { useReduxEmbeddableContext } from '@kbn/presentation-util-plugin/public';
import { EuiSelectableOption } from '@elastic/eui/src/components/selectable/selectable_option';
import { OptionsListReduxState } from '../types';
import { OptionsListStrings } from './options_list_strings';
import { optionsListReducers } from '../options_list_reducers';
import { MAX_OPTIONS_LIST_REQUEST_SIZE, OptionsListReduxState } from '../types';
import { OptionsListPopoverEmptyMessage } from './options_list_popover_empty_message';
import { OptionsListPopoverSuggestionBadge } from './options_list_popover_suggestion_badge';
interface OptionsListPopoverSuggestionsProps {
isLoading: boolean;
showOnlySelected: boolean;
loadMoreSuggestions: (cardinality: number) => void;
}
export const OptionsListPopoverSuggestions = ({
isLoading,
showOnlySelected,
loadMoreSuggestions,
}: OptionsListPopoverSuggestionsProps) => {
// Redux embeddable container Context
const {
@ -38,12 +39,27 @@ export const OptionsListPopoverSuggestions = ({
// Select current state from Redux using multiple selectors to avoid rerenders.
const invalidSelections = select((state) => state.componentState.invalidSelections);
const availableOptions = select((state) => state.componentState.availableOptions);
const totalCardinality = select((state) => state.componentState.totalCardinality);
const searchString = select((state) => state.componentState.searchString);
const selectedOptions = select((state) => state.explicitInput.selectedOptions);
const existsSelected = select((state) => state.explicitInput.existsSelected);
const singleSelect = select((state) => state.explicitInput.singleSelect);
const hideExists = select((state) => state.explicitInput.hideExists);
const isLoading = select((state) => state.output.loading) ?? false;
const fieldName = select((state) => state.explicitInput.fieldName);
const sort = select((state) => state.explicitInput.sort);
const listRef = useRef<HTMLDivElement>(null);
const canLoadMoreSuggestions = useMemo(
() =>
totalCardinality
? Object.keys(availableOptions ?? {}).length <
Math.min(totalCardinality, MAX_OPTIONS_LIST_REQUEST_SIZE)
: false,
[availableOptions, totalCardinality]
);
// track selectedOptions and invalidSelections in sets for more efficient lookup
const selectedOptionsSet = useMemo(() => new Set<string>(selectedOptions), [selectedOptions]);
@ -51,7 +67,6 @@ export const OptionsListPopoverSuggestions = ({
() => new Set<string>(invalidSelections),
[invalidSelections]
);
const suggestions = useMemo(() => {
return showOnlySelected ? selectedOptions : Object.keys(availableOptions ?? {});
}, [availableOptions, selectedOptions, showOnlySelected]);
@ -87,10 +102,23 @@ export const OptionsListPopoverSuggestions = ({
) : undefined,
};
});
const suggestionsSelectableOptions = existsSelectableOption
? [existsSelectableOption, ...options]
: options;
setSelectableOptions(suggestionsSelectableOptions);
if (canLoadMoreSuggestions) {
options.push({
key: 'loading-option',
className: 'optionslist--loadingMoreGroupLabel',
label: OptionsListStrings.popover.getLoadingMoreMessage(),
isGroupLabel: true,
});
} else if (options.length === MAX_OPTIONS_LIST_REQUEST_SIZE) {
options.push({
key: 'no-more-option',
className: 'optionslist--endOfOptionsGroupLabel',
label: OptionsListStrings.popover.getAtEndOfOptionsMessage(),
isGroupLabel: true,
});
}
setSelectableOptions(existsSelectableOption ? [existsSelectableOption, ...options] : options);
}, [
suggestions,
availableOptions,
@ -98,42 +126,66 @@ export const OptionsListPopoverSuggestions = ({
selectedOptionsSet,
invalidSelectionsSet,
existsSelectableOption,
canLoadMoreSuggestions,
]);
return (
<EuiSelectable
isLoading={isLoading}
loadingMessage={
<span data-test-subj="optionsList-control-popover-loading">
<EuiLoadingSpinner size="m" />
<EuiSpacer size="xs" />
{OptionsListStrings.popover.getLoadingMessage()}
</span>
}
options={selectableOptions}
listProps={{ onFocusBadge: false }}
aria-label={OptionsListStrings.popover.getSuggestionsAriaLabel(
fieldName,
selectableOptions.length
)}
emptyMessage={<OptionsListPopoverEmptyMessage showOnlySelected={showOnlySelected} />}
onChange={(newSuggestions, _, changedOption) => {
setSelectableOptions(newSuggestions);
const loadMoreOptions = useCallback(() => {
const listbox = listRef.current?.querySelector('.euiSelectableList__list');
if (!listbox) return;
const key = changedOption.key ?? changedOption.label;
// the order of these checks matters, so be careful if rearranging them
if (key === 'exists-option') {
dispatch(selectExists(!Boolean(existsSelected)));
} else if (showOnlySelected || selectedOptionsSet.has(key)) {
dispatch(deselectOption(key));
} else if (singleSelect) {
dispatch(replaceSelection(key));
} else {
dispatch(selectOption(key));
}
}}
>
{(list) => list}
</EuiSelectable>
const { scrollTop, scrollHeight, clientHeight } = listbox;
if (scrollTop + clientHeight >= scrollHeight - parseInt(euiThemeVars.euiSizeXXL, 10)) {
// reached the "bottom" of the list, where euiSizeXXL acts as a "margin of error" so that the user doesn't
// have to scroll **all the way** to the bottom in order to load more options
loadMoreSuggestions(totalCardinality ?? MAX_OPTIONS_LIST_REQUEST_SIZE);
}
}, [loadMoreSuggestions, totalCardinality]);
useEffect(() => {
const container = listRef.current;
if (!isLoading && canLoadMoreSuggestions) {
container?.addEventListener('scroll', loadMoreOptions, true);
return () => {
container?.removeEventListener('scroll', loadMoreOptions, true);
};
}
}, [loadMoreOptions, isLoading, canLoadMoreSuggestions]);
useEffect(() => {
// scroll back to the top when changing the sorting or the search string
const listbox = listRef.current?.querySelector('.euiSelectableList__list');
listbox?.scrollTo({ top: 0 });
}, [sort, searchString]);
return (
<>
<div ref={listRef}>
<EuiSelectable
options={selectableOptions}
listProps={{ onFocusBadge: false }}
aria-label={OptionsListStrings.popover.getSuggestionsAriaLabel(
fieldName,
selectableOptions.length
)}
emptyMessage={<OptionsListPopoverEmptyMessage showOnlySelected={showOnlySelected} />}
onChange={(newSuggestions, _, changedOption) => {
const key = changedOption.key ?? changedOption.label;
setSelectableOptions(newSuggestions);
// the order of these checks matters, so be careful if rearranging them
if (key === 'exists-option') {
dispatch(selectExists(!Boolean(existsSelected)));
} else if (showOnlySelected || selectedOptionsSet.has(key)) {
dispatch(deselectOption(key));
} else if (singleSelect) {
dispatch(replaceSelection(key));
} else {
dispatch(selectOption(key));
}
}}
>
{(list) => list}
</EuiSelectable>
</div>
</>
);
};

46
src/plugins/controls/public/options_list/components/options_list_popover_title.tsx Normal file
View file

@ -0,0 +1,46 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import React from 'react';
import { EuiFlexGroup, EuiFlexItem, EuiPopoverTitle, EuiIconTip } from '@elastic/eui';
import { useReduxEmbeddableContext } from '@kbn/presentation-util-plugin/public';
import { OptionsListReduxState } from '../types';
import { OptionsListStrings } from './options_list_strings';
import { optionsListReducers } from '../options_list_reducers';
export const OptionsListPopoverTitle = () => {
// Redux embeddable container Context
const { useEmbeddableSelector: select } = useReduxEmbeddableContext<
OptionsListReduxState,
typeof optionsListReducers
>();
// Select current state from Redux using multiple selectors to avoid rerenders.
const allowExpensiveQueries = select((state) => state.componentState.allowExpensiveQueries);
const title = select((state) => state.explicitInput.title);
return (
<EuiPopoverTitle paddingSize="s">
<EuiFlexGroup gutterSize="xs" alignItems="center">
<EuiFlexItem grow={false}>{title}</EuiFlexItem>
{!allowExpensiveQueries && (
<EuiFlexItem data-test-subj="optionsList-allow-expensive-queries-warning" grow={false}>
<EuiIconTip
aria-label="Warning"
type="alert"
color="warning"
content={OptionsListStrings.popover.getAllowExpensiveQueriesWarning()}
/>
</EuiFlexItem>
)}
</EuiFlexGroup>
</EuiPopoverTitle>
);
};

32
src/plugins/controls/public/options_list/components/options_list_strings.ts
View file

@ -54,9 +54,19 @@ export const OptionsListStrings = {
'Available {optionCount, plural, one {option} other {options}} for {fieldName}',
values: { fieldName, optionCount },
}),
getLoadingMessage: () =>
i18n.translate('controls.optionsList.popover.loading', {
defaultMessage: 'Loading options',
getAllowExpensiveQueriesWarning: () =>
i18n.translate('controls.optionsList.popover.allowExpensiveQueriesWarning', {
defaultMessage:
'The cluster setting to allow expensive queries is off, so some features are disabled.',
}),
getLoadingMoreMessage: () =>
i18n.translate('controls.optionsList.popover.loadingMore', {
defaultMessage: 'Loading more options...',
}),
getAtEndOfOptionsMessage: () =>
i18n.translate('controls.optionsList.popover.endOfOptions', {
defaultMessage:
'The top 1,000 available options are displayed. View more options by searching for the name.',
}),
getEmptyMessage: () =>
i18n.translate('controls.optionsList.popover.empty', {
@ -78,10 +88,14 @@ export const OptionsListStrings = {
i18n.translate('controls.optionsList.popover.clearAllSelectionsTitle', {
defaultMessage: 'Clear selections',
}),
getTotalCardinalityPlaceholder: (totalOptions: number) =>
i18n.translate('controls.optionsList.popover.cardinalityPlaceholder', {
getSearchPlaceholder: () =>
i18n.translate('controls.optionsList.popover.searchPlaceholder', {
defaultMessage: 'Search',
}),
getCardinalityLabel: (totalOptions: number) =>
i18n.translate('controls.optionsList.popover.cardinalityLabel', {
defaultMessage:
'Search {totalOptions} available {totalOptions, plural, one {option} other {options}}',
'{totalOptions, number} {totalOptions, plural, one {option} other {options}}',
values: { totalOptions },
}),
getInvalidSelectionsSectionAriaLabel: (fieldName: string, invalidSelectionCount: number) =>
@ -96,10 +110,10 @@ export const OptionsListStrings = {
'Ignored {invalidSelectionCount, plural, one {selection} other {selections}}',
values: { invalidSelectionCount },
}),
getInvalidSelectionsTooltip: (selectedOptions: number) =>
i18n.translate('controls.optionsList.popover.invalidSelectionsTooltip', {
getInvalidSelectionsLabel: (selectedOptions: number) =>
i18n.translate('controls.optionsList.popover.invalidSelectionsLabel', {
defaultMessage:
'{selectedOptions} selected {selectedOptions, plural, one {option} other {options}} {selectedOptions, plural, one {is} other {are}} ignored because {selectedOptions, plural, one {it is} other {they are}} no longer in the data.',
'{selectedOptions} {selectedOptions, plural, one {selection} other {selections}} ignored',
values: { selectedOptions },
}),
getInvalidSelectionScreenReaderText: () =>

113
src/plugins/controls/public/options_list/embeddable/options_list_embeddable.tsx
View file

@ -24,11 +24,11 @@ import {
buildExistsFilter,
} from '@kbn/es-query';
import { ReduxEmbeddableTools, ReduxEmbeddablePackage } from '@kbn/presentation-util-plugin/public';
import { DataView } from '@kbn/data-views-plugin/public';
import { DataView, FieldSpec } from '@kbn/data-views-plugin/public';
import { Embeddable, IContainer } from '@kbn/embeddable-plugin/public';
import { KibanaThemeProvider } from '@kbn/kibana-react-plugin/public';
import { OptionsListReduxState } from '../types';
import { MIN_OPTIONS_LIST_REQUEST_SIZE, OptionsListReduxState } from '../types';
import { pluginServices } from '../../services';
import {
ControlInput,
@ -40,7 +40,6 @@ import { getDefaultComponentState, optionsListReducers } from '../options_list_r
import { OptionsListControl } from '../components/options_list_control';
import { ControlsDataViewsService } from '../../services/data_views/types';
import { ControlsOptionsListService } from '../../services/options_list/types';
import { OptionsListField } from '../../../common/options_list/types';
const diffDataFetchProps = (
last?: OptionsListDataFetchProps,
@ -76,9 +75,10 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
// Internal data fetching state for this input control.
private typeaheadSubject: Subject<string> = new Subject<string>();
private loadMoreSubject: Subject<number> = new Subject<number>();
private abortController?: AbortController;
private dataView?: DataView;
private field?: OptionsListField;
private field?: FieldSpec;
private reduxEmbeddableTools: ReduxEmbeddableTools<
OptionsListReduxState,
@ -98,6 +98,7 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
pluginServices.getServices());
this.typeaheadSubject = new Subject<string>();
this.loadMoreSubject = new Subject<number>();
// build redux embeddable tools
this.reduxEmbeddableTools = reduxEmbeddablePackage.createTools<
@ -115,6 +116,13 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
private initialize = async () => {
const { selectedOptions: initialSelectedOptions } = this.getInput();
if (!initialSelectedOptions) this.setInitializationFinished();
const {
actions: { setAllowExpensiveQueries },
dispatch,
} = this.reduxEmbeddableTools;
dispatch(setAllowExpensiveQueries(await this.optionsListService.getAllowExpensiveQueries()));
this.runOptionsListQuery().then(async () => {
if (initialSelectedOptions) {
await this.buildFilter();
@ -144,12 +152,21 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
// debounce typeahead pipe to slow down search string related queries
const typeaheadPipe = this.typeaheadSubject.pipe(debounceTime(100));
const loadMorePipe = this.loadMoreSubject.pipe(debounceTime(100));
// fetch available options when input changes or when search string has changed
this.subscriptions.add(
merge(dataFetchPipe, typeaheadPipe)
.pipe(skip(1)) // Skip the first input update because options list query will be run by initialize.
.subscribe(this.runOptionsListQuery)
.subscribe(() => {
this.runOptionsListQuery();
})
);
// fetch more options when reaching the bottom of the available options
this.subscriptions.add(
loadMorePipe.subscribe((size) => {
this.runOptionsListQuery(size);
})
);
/**
@ -203,7 +220,7 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
private getCurrentDataViewAndField = async (): Promise<{
dataView?: DataView;
field?: OptionsListField;
field?: FieldSpec;
}> => {
const {
dispatch,
@ -212,7 +229,7 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
} = this.reduxEmbeddableTools;
const {
explicitInput: { dataViewId, fieldName, parentFieldName, childFieldName },
explicitInput: { dataViewId, fieldName },
} = getState();
if (!this.dataView || this.dataView.id !== dataViewId) {
@ -244,21 +261,7 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
);
}
// pair up keyword / text fields for case insensitive search
const childField =
(childFieldName && this.dataView.getFieldByName(childFieldName)) || undefined;
const parentField =
(parentFieldName && this.dataView.getFieldByName(parentFieldName)) || undefined;
const textFieldName = childField?.esTypes?.includes('text')
? childField.name
: parentField?.esTypes?.includes('text')
? parentField.name
: undefined;
const optionsListField: OptionsListField = originalField.toSpec();
optionsListField.textFieldName = textFieldName;
this.field = optionsListField;
this.field = originalField.toSpec();
} catch (e) {
this.onFatalError(e);
}
@ -268,13 +271,12 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
return { dataView: this.dataView, field: this.field! };
};
private runOptionsListQuery = async () => {
private runOptionsListQuery = async (size: number = MIN_OPTIONS_LIST_REQUEST_SIZE) => {
const {
dispatch,
getState,
actions: { setLoading, publishFilters, setSearchString, updateQueryResults },
} = this.reduxEmbeddableTools;
const previousFieldName = this.field?.name;
const { dataView, field } = await this.getCurrentDataViewAndField();
if (!dataView || !field) return;
@ -284,7 +286,7 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
}
const {
componentState: { searchString },
componentState: { searchString, allowExpensiveQueries },
explicitInput: { selectedOptions, runPastTimeout, existsSelected, sort },
} = getState();
dispatch(setLoading(true));
@ -307,27 +309,34 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
mode: 'absolute' as 'absolute',
}
: globalTimeRange;
const { suggestions, invalidSelections, totalCardinality, rejected } =
await this.optionsListService.runOptionsListRequest(
{
sort,
field,
query,
filters,
dataView,
timeRange,
searchString: searchString.value,
runPastTimeout,
selectedOptions,
},
this.abortController.signal
);
if (rejected) {
// This prevents a rejected request (which can happen, for example, when a user types a search string too quickly)
// from prematurely setting loading to `false` and updating the suggestions to show "No results"
const response = await this.optionsListService.runOptionsListRequest(
{
sort,
size,
field,
query,
filters,
dataView,
timeRange,
runPastTimeout,
selectedOptions,
allowExpensiveQueries,
searchString: searchString.value,
},
this.abortController.signal
);
if (this.optionsListService.optionsListResponseWasFailure(response)) {
if (response.error === 'aborted') {
// This prevents an aborted request (which can happen, for example, when a user types a search string too quickly)
// from prematurely setting loading to `false` and updating the suggestions to show "No results"
return;
}
this.onFatalError(response.error);
return;
}
const { suggestions, invalidSelections, totalCardinality } = response;
if (
(!selectedOptions && !existsSelected) ||
isEmpty(invalidSelections) ||
@ -411,11 +420,24 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
this.runOptionsListQuery();
};
public onFatalError = (e: Error) => {
const {
dispatch,
actions: { setPopoverOpen, setLoading },
} = this.reduxEmbeddableTools;
batch(() => {
dispatch(setLoading(false));
dispatch(setPopoverOpen(false));
});
super.onFatalError(e);
};
public destroy = () => {
super.destroy();
this.abortController?.abort();
this.subscriptions.unsubscribe();
this.reduxEmbeddableTools.cleanup();
if (this.node) ReactDOM.unmountComponentAtNode(this.node);
};
public render = (node: HTMLElement) => {
@ -427,7 +449,10 @@ export class OptionsListEmbeddable extends Embeddable<OptionsListEmbeddableInput
ReactDOM.render(
<KibanaThemeProvider theme$={pluginServices.getServices().theme.theme$}>
<OptionsListReduxWrapper>
<OptionsListControl typeaheadSubject={this.typeaheadSubject} />
<OptionsListControl
typeaheadSubject={this.typeaheadSubject}
loadMoreSubject={this.loadMoreSubject}
/>
</OptionsListReduxWrapper>
</KibanaThemeProvider>,
node

20
src/plugins/controls/public/options_list/options_list_reducers.ts
View file

@ -9,9 +9,9 @@ import { PayloadAction } from '@reduxjs/toolkit';
import { WritableDraft } from 'immer/dist/types/types-external';
import { Filter } from '@kbn/es-query';
import { FieldSpec } from '@kbn/data-views-plugin/common';
import { OptionsListReduxState, OptionsListComponentState } from './types';
import { OptionsListField } from '../../common/options_list/types';
import { getIpRangeQuery } from '../../common/options_list/ip_search';
import {
OPTIONS_LIST_DEFAULT_SORT,
@ -19,6 +19,8 @@ import {
} from '../../common/options_list/suggestions_sorting';
export const getDefaultComponentState = (): OptionsListReduxState['componentState'] => ({
popoverOpen: false,
allowExpensiveQueries: true,
searchString: { value: '', valid: true },
});
@ -41,6 +43,15 @@ export const optionsListReducers = {
state.componentState.searchString.valid = getIpRangeQuery(action.payload).validSearch;
}
},
setAllowExpensiveQueries: (
state: WritableDraft<OptionsListReduxState>,
action: PayloadAction<boolean>
) => {
state.componentState.allowExpensiveQueries = action.payload;
},
setPopoverOpen: (state: WritableDraft<OptionsListReduxState>, action: PayloadAction<boolean>) => {
state.componentState.popoverOpen = action.payload;
},
setSort: (
state: WritableDraft<OptionsListReduxState>,
action: PayloadAction<Partial<OptionsListSortingType>>
@ -97,7 +108,7 @@ export const optionsListReducers = {
},
setField: (
state: WritableDraft<OptionsListReduxState>,
action: PayloadAction<OptionsListField | undefined>
action: PayloadAction<FieldSpec | undefined>
) => {
state.componentState.field = action.payload;
},
@ -110,7 +121,10 @@ export const optionsListReducers = {
>
>
) => {
state.componentState = { ...(state.componentState ?? {}), ...action.payload };
state.componentState = {
...(state.componentState ?? {}),
...action.payload,
};
},
publishFilters: (
state: WritableDraft<OptionsListReduxState>,

14
src/plugins/controls/public/options_list/types.ts
View file

@ -7,13 +7,17 @@
*/
import { ReduxEmbeddableState } from '@kbn/presentation-util-plugin/public';
import { FieldSpec } from '@kbn/data-views-plugin/common';
import { ControlOutput } from '../types';
import {
OptionsListField,
OptionsListSuggestions,
OptionsListEmbeddableInput,
} from '../../common/options_list/types';
export const MIN_OPTIONS_LIST_REQUEST_SIZE = 10;
export const MAX_OPTIONS_LIST_REQUEST_SIZE = 1000;
interface SearchString {
value: string;
valid: boolean;
@ -21,12 +25,14 @@ interface SearchString {
// Component state is only used by public components.
export interface OptionsListComponentState {
field?: OptionsListField;
totalCardinality?: number;
availableOptions?: OptionsListSuggestions;
allowExpensiveQueries: boolean;
invalidSelections?: string[];
validSelections?: string[];
searchString: SearchString;
validSelections?: string[];
totalCardinality?: number;
popoverOpen: boolean;
field?: FieldSpec;
}
// public only - redux embeddable state type

1
src/plugins/controls/public/services/http/http.stub.ts
View file

@ -13,5 +13,6 @@ import { ControlsHTTPService } from './types';
type HttpServiceFactory = PluginServiceFactory<ControlsHTTPService>;
export const httpServiceFactory: HttpServiceFactory = () => ({
get: async () => ({} as unknown as HttpResponse),
fetch: async () => ({} as unknown as HttpResponse),
});

3
src/plugins/controls/public/services/http/http_service.ts
View file

@ -16,10 +16,11 @@ export type HttpServiceFactory = KibanaPluginServiceFactory<
>;
export const httpServiceFactory: HttpServiceFactory = ({ coreStart }) => {
const {
http: { fetch },
http: { get, fetch },
} = coreStart;
return {
get,
fetch,
};
};

1
src/plugins/controls/public/services/http/types.ts
View file

@ -9,5 +9,6 @@
import { CoreSetup } from '@kbn/core/public';
export interface ControlsHTTPService {
get: CoreSetup['http']['get'];
fetch: CoreSetup['http']['fetch'];
}

11
src/plugins/controls/public/services/options_list/options_list.story.ts
View file

@ -20,14 +20,11 @@ let optionsListRequestMethod = async (request: OptionsListRequest, abortSignal:
r({
suggestions: {},
totalCardinality: 100,
rejected: false,
}),
120
)
);
const clearOptionsListCacheMock = () => {};
export const replaceOptionsListMethod = (
newMethod: (request: OptionsListRequest, abortSignal: AbortSignal) => Promise<OptionsListResponse>
) => (optionsListRequestMethod = newMethod);
@ -35,6 +32,12 @@ export const replaceOptionsListMethod = (
export const optionsListServiceFactory: OptionsListServiceFactory = () => {
return {
runOptionsListRequest: optionsListRequestMethod,
clearOptionsListCache: clearOptionsListCacheMock,
clearOptionsListCache: jest.fn(),
getAllowExpensiveQueries: jest.fn().mockReturnValue(Promise.resolve(true)),
optionsListResponseWasFailure: jest
.fn()
.mockReturnValue(
false
) as unknown as ControlsOptionsListService['optionsListResponseWasFailure'],
};
};

31
src/plugins/controls/public/services/options_list/options_list_service.ts
View file

@ -16,7 +16,7 @@ import {
OptionsListRequest,
OptionsListResponse,
OptionsListRequestBody,
OptionsListField,
OptionsListFailureResponse,
} from '../../../common/options_list/types';
import { ControlsHTTPService } from '../http/types';
import { ControlsDataService } from '../data/types';
@ -38,6 +38,7 @@ class OptionsListService implements ControlsOptionsListService {
private optionsListCacheResolver = (request: OptionsListRequest) => {
const {
size,
sort,
query,
filters,
@ -59,6 +60,7 @@ class OptionsListService implements ControlsOptionsListService {
dataViewTitle,
searchString,
fieldName,
size,
].join('|');
};
@ -90,18 +92,39 @@ class OptionsListService implements ControlsOptionsListService {
filters: esFilters,
fieldName: field.name,
fieldSpec: field,
textFieldName: (field as OptionsListField).textFieldName,
runtimeFieldMap: dataView.toSpec().runtimeFieldMap,
};
};
private cachedAllowExpensiveQueries = memoize(async () => {
const { allowExpensiveQueries } = await this.http.get<{
allowExpensiveQueries: boolean;
}>('/api/kibana/controls/optionsList/getClusterSettings');
return allowExpensiveQueries;
});
public getAllowExpensiveQueries = async (): Promise<boolean> => {
try {
return await this.cachedAllowExpensiveQueries();
} catch (error) {
return false;
}
};
public optionsListResponseWasFailure = (
response: OptionsListResponse
): response is OptionsListFailureResponse => {
return (response as OptionsListFailureResponse).error !== undefined;
};
public runOptionsListRequest = async (request: OptionsListRequest, abortSignal: AbortSignal) => {
try {
return await this.cachedOptionsListRequest(request, abortSignal);
} catch (error) {
} catch (error: any) {
// Remove rejected results from memoize cache
this.cachedOptionsListRequest.cache.delete(this.optionsListCacheResolver(request));
return { rejected: true } as OptionsListResponse;
if (error.name === 'AbortError') return { error: 'aborted' } as OptionsListFailureResponse;
return { error } as OptionsListFailureResponse;
}
};

11
src/plugins/controls/public/services/options_list/types.ts
View file

@ -6,13 +6,20 @@
* Side Public License, v 1.
*/
import { OptionsListRequest, OptionsListResponse } from '../../../common/options_list/types';
import {
OptionsListFailureResponse,
OptionsListRequest,
OptionsListResponse,
} from '../../../common/options_list/types';
export interface ControlsOptionsListService {
runOptionsListRequest: (
request: OptionsListRequest,
abortSignal: AbortSignal
) => Promise<OptionsListResponse>;
clearOptionsListCache: () => void;
optionsListResponseWasFailure: (
response: OptionsListResponse
) => response is OptionsListFailureResponse;
getAllowExpensiveQueries: () => Promise<boolean>;
}

2
src/plugins/controls/public/types.ts
View file

@ -63,8 +63,6 @@ export interface ControlEditorProps<T extends ControlInput = ControlInput> {
export interface DataControlField {
field: DataViewField;
parentFieldName?: string;
childFieldName?: string;
compatibleControlTypes: string[];
}

519
src/plugins/controls/server/options_list/options_list_queries.test.ts → src/plugins/controls/server/options_list/options_list_cheap_suggestion_queries.test.ts
View file

@ -9,13 +9,10 @@
import { FieldSpec } from '@kbn/data-views-plugin/common';
import { SearchResponse } from '@elastic/elasticsearch/lib/api/types';
import {
getSuggestionAggregationBuilder,
getValidationAggregationBuilder,
} from './options_list_queries';
import { getCheapSuggestionAggregationBuilder } from './options_list_cheap_suggestion_queries';
import { OptionsListRequestBody } from '../../common/options_list/types';
describe('options list queries', () => {
describe('options list cheap queries', () => {
let rawSearchResponseMock: SearchResponse = {} as SearchResponse;
beforeEach(() => {
@ -37,146 +34,85 @@ describe('options list queries', () => {
};
});
describe('validation aggregation and parsing', () => {
test('creates validation aggregation when given selections', () => {
const validationAggBuilder = getValidationAggregationBuilder();
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolTestField',
selectedOptions: ['coolOption1', 'coolOption2', 'coolOption3'],
};
expect(validationAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"filters": Object {
"filters": Object {
"coolOption1": Object {
"match": Object {
"coolTestField": "coolOption1",
},
},
"coolOption2": Object {
"match": Object {
"coolTestField": "coolOption2",
},
},
"coolOption3": Object {
"match": Object {
"coolTestField": "coolOption3",
},
},
},
},
}
`);
});
test('returns undefined when not given selections', () => {
const validationAggBuilder = getValidationAggregationBuilder();
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolTestField',
};
expect(validationAggBuilder.buildAggregation(optionsListRequestBodyMock)).toBeUndefined();
});
test('parses validation result', () => {
const validationAggBuilder = getValidationAggregationBuilder();
rawSearchResponseMock.aggregations = {
validation: {
buckets: {
cool1: { doc_count: 0 },
cool2: { doc_count: 15 },
cool3: { doc_count: 0 },
cool4: { doc_count: 2 },
cool5: { doc_count: 112 },
cool6: { doc_count: 0 },
},
},
};
expect(validationAggBuilder.parse(rawSearchResponseMock)).toMatchInlineSnapshot(`
Array [
"cool1",
"cool3",
"cool6",
]
`);
});
});
describe('suggestion aggregation', () => {
describe('text / keyword field', () => {
test('with a search string, creates case insensitive aggregation', () => {
describe('keyword or text+keyword field', () => {
test('without a search string, creates keyword aggregation', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
allowExpensiveQueries: false,
fieldName: 'coolTestField.keyword',
textFieldName: 'coolTestField',
searchString: 'cooool',
sort: { by: '_count', direction: 'asc' },
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"aggs": Object {
"keywordSuggestions": Object {
"suggestions": Object {
"terms": Object {
"field": "coolTestField.keyword",
"include": ".*",
"order": Object {
"_count": "asc",
},
"shard_size": 10,
},
},
}
`);
});
test('with a search string, creates case sensitive keyword aggregation', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
searchString: 'cooool',
allowExpensiveQueries: false,
fieldName: 'coolTestField.keyword',
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"terms": Object {
"field": "coolTestField.keyword",
"include": "cooool.*",
"order": Object {
"_count": "desc",
},
"shard_size": 10,
},
},
},
"filter": Object {
"match_phrase_prefix": Object {
"coolTestField": "cooool",
},
},
}
`);
});
test('without a search string, creates keyword aggregation', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolTestField.keyword',
textFieldName: 'coolTestField',
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
sort: { by: '_count', direction: 'asc' },
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"terms": Object {
"execution_hint": "map",
"field": "coolTestField.keyword",
"include": ".*",
"order": Object {
"_count": "asc",
},
"shard_size": 10,
},
}
`);
}
`);
});
});
test('creates boolean aggregation for boolean field', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'coolean',
fieldSpec: { type: 'boolean' } as unknown as FieldSpec,
allowExpensiveQueries: false,
sort: { by: '_key', direction: 'desc' },
fieldSpec: { type: 'boolean' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(optionsListRequestBodyMock);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"terms": Object {
"execution_hint": "map",
"field": "coolean",
"order": Object {
"_key": "desc",
"suggestions": Object {
"terms": Object {
"field": "coolean",
"order": Object {
"_key": "desc",
},
"shard_size": 10,
},
"shard_size": 10,
},
}
`);
@ -184,53 +120,33 @@ describe('options list queries', () => {
test('creates nested aggregation for nested field', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolNestedField',
size: 10,
searchString: 'cooool',
fieldSpec: { subType: { nested: { path: 'path.to.nested' } } } as unknown as FieldSpec,
allowExpensiveQueries: false,
fieldName: 'coolNestedField',
sort: { by: '_key', direction: 'asc' },
fieldSpec: { subType: { nested: { path: 'path.to.nested' } } } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(optionsListRequestBodyMock);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"aggs": Object {
"nestedSuggestions": Object {
"terms": Object {
"execution_hint": "map",
"field": "coolNestedField",
"include": "cooool.*",
"order": Object {
"_key": "asc",
"nestedSuggestions": Object {
"aggs": Object {
"suggestions": Object {
"terms": Object {
"field": "coolNestedField",
"include": "cooool.*",
"order": Object {
"_key": "asc",
},
"shard_size": 10,
},
"shard_size": 10,
},
},
},
"nested": Object {
"path": "path.to.nested",
},
}
`);
});
test('creates keyword only aggregation', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolTestField.keyword',
searchString: 'cooool',
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"terms": Object {
"execution_hint": "map",
"field": "coolTestField.keyword",
"include": "cooool.*",
"order": Object {
"_count": "desc",
"nested": Object {
"path": "path.to.nested",
},
"shard_size": 10,
},
}
`);
@ -239,36 +155,41 @@ describe('options list queries', () => {
describe('IP field', () => {
test('without a search string, creates IP range aggregation with default range', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
allowExpensiveQueries: false,
sort: { by: '_count', direction: 'asc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"execution_hint": "map",
"field": "clientip",
"order": Object {
"_count": "asc",
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_count": "asc",
},
"shard_size": 10,
},
"shard_size": 10,
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "::",
"key": "ipv6",
"to": "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
},
],
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "::",
"key": "ipv6",
"to": "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
},
],
},
},
}
`);
@ -276,36 +197,41 @@ describe('options list queries', () => {
test('full IPv4 in the search string, creates IP range aggregation with CIDR mask', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
allowExpensiveQueries: false,
searchString: '41.77.243.255',
sort: { by: '_key', direction: 'desc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"execution_hint": "map",
"field": "clientip",
"order": Object {
"_key": "desc",
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_key": "desc",
},
"shard_size": 10,
},
"shard_size": 10,
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"key": "ipv4",
"mask": "41.77.243.255/32",
},
],
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"key": "ipv4",
"mask": "41.77.243.255/32",
},
],
},
},
}
`);
@ -313,36 +239,41 @@ describe('options list queries', () => {
test('full IPv6 in the search string, creates IP range aggregation with CIDR mask', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
allowExpensiveQueries: false,
sort: { by: '_key', direction: 'asc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
searchString: 'f688:fb50:6433:bba2:604:f2c:194a:d3c5',
sort: { by: '_key', direction: 'asc' },
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"execution_hint": "map",
"field": "clientip",
"order": Object {
"_key": "asc",
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_key": "asc",
},
"shard_size": 10,
},
"shard_size": 10,
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"key": "ipv6",
"mask": "f688:fb50:6433:bba2:604:f2c:194a:d3c5/128",
},
],
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"key": "ipv6",
"mask": "f688:fb50:6433:bba2:604:f2c:194a:d3c5/128",
},
],
},
},
}
`);
@ -350,36 +281,41 @@ describe('options list queries', () => {
test('partial IPv4 in the search string, creates IP range aggregation with min and max', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
searchString: '41.77',
allowExpensiveQueries: false,
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"execution_hint": "map",
"field": "clientip",
"order": Object {
"_count": "desc",
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_count": "desc",
},
"shard_size": 10,
},
"shard_size": 10,
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "41.77.0.0",
"key": "ipv4",
"to": "41.77.255.255",
},
],
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "41.77.0.0",
"key": "ipv4",
"to": "41.77.255.255",
},
],
},
},
}
`);
@ -387,37 +323,42 @@ describe('options list queries', () => {
test('partial IPv46 in the search string, creates IP range aggregation with min and max', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
searchString: 'cdb6:',
allowExpensiveQueries: false,
sort: { by: '_count', direction: 'desc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"execution_hint": "map",
"field": "clientip",
"order": Object {
"_count": "desc",
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_count": "desc",
},
"shard_size": 10,
},
"shard_size": 10,
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "cdb6::",
"key": "ipv6",
"to": "cdb6:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
},
],
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "cdb6::",
"key": "ipv6",
"to": "cdb6:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
},
],
},
},
}
`);
@ -428,24 +369,25 @@ describe('options list queries', () => {
describe('suggestion parsing', () => {
test('parses keyword / text result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolTestField.keyword',
textFieldName: 'coolTestField',
size: 10,
searchString: 'cooool',
allowExpensiveQueries: false,
fieldName: 'coolTestField.keyword',
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(optionsListRequestBodyMock);
rawSearchResponseMock.aggregations = {
suggestions: {
keywordSuggestions: {
buckets: [
{ doc_count: 5, key: 'cool1' },
{ doc_count: 15, key: 'cool2' },
{ doc_count: 10, key: 'cool3' },
],
},
buckets: [
{ doc_count: 5, key: 'cool1' },
{ doc_count: 15, key: 'cool2' },
{ doc_count: 10, key: 'cool3' },
],
},
};
expect(suggestionAggBuilder.parse(rawSearchResponseMock)).toMatchInlineSnapshot(`
expect(
suggestionAggBuilder.parse(rawSearchResponseMock, optionsListRequestBodyMock).suggestions
).toMatchInlineSnapshot(`
Object {
"cool1": Object {
"doc_count": 5,
@ -462,10 +404,12 @@ describe('options list queries', () => {
test('parses boolean result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'coolean',
allowExpensiveQueries: false,
fieldSpec: { type: 'boolean' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(optionsListRequestBodyMock);
rawSearchResponseMock.aggregations = {
suggestions: {
buckets: [
@ -474,7 +418,9 @@ describe('options list queries', () => {
],
},
};
expect(suggestionAggBuilder.parse(rawSearchResponseMock)).toMatchInlineSnapshot(`
expect(
suggestionAggBuilder.parse(rawSearchResponseMock, optionsListRequestBodyMock).suggestions
).toMatchInlineSnapshot(`
Object {
"false": Object {
"doc_count": 55,
@ -488,14 +434,16 @@ describe('options list queries', () => {
test('parses nested result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolNestedField',
size: 10,
searchString: 'cooool',
fieldName: 'coolNestedField',
allowExpensiveQueries: false,
fieldSpec: { subType: { nested: { path: 'path.to.nested' } } } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(optionsListRequestBodyMock);
rawSearchResponseMock.aggregations = {
suggestions: {
nestedSuggestions: {
nestedSuggestions: {
suggestions: {
buckets: [
{ doc_count: 5, key: 'cool1' },
{ doc_count: 15, key: 'cool2' },
@ -504,7 +452,9 @@ describe('options list queries', () => {
},
},
};
expect(suggestionAggBuilder.parse(rawSearchResponseMock)).toMatchInlineSnapshot(`
expect(
suggestionAggBuilder.parse(rawSearchResponseMock, optionsListRequestBodyMock).suggestions
).toMatchInlineSnapshot(`
Object {
"cool1": Object {
"doc_count": 5,
@ -521,11 +471,13 @@ describe('options list queries', () => {
test('parses keyword only result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
fieldName: 'coolTestField.keyword',
size: 10,
searchString: 'cooool',
allowExpensiveQueries: false,
fieldName: 'coolTestField.keyword',
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(optionsListRequestBodyMock);
rawSearchResponseMock.aggregations = {
suggestions: {
buckets: [
@ -535,7 +487,9 @@ describe('options list queries', () => {
],
},
};
expect(suggestionAggBuilder.parse(rawSearchResponseMock)).toMatchInlineSnapshot(`
expect(
suggestionAggBuilder.parse(rawSearchResponseMock, optionsListRequestBodyMock).suggestions
).toMatchInlineSnapshot(`
Object {
"cool1": Object {
"doc_count": 5,
@ -553,10 +507,12 @@ describe('options list queries', () => {
test('parses mixed IPv4 and IPv6 result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
allowExpensiveQueries: false,
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getSuggestionAggregationBuilder(optionsListRequestBodyMock);
const suggestionAggBuilder = getCheapSuggestionAggregationBuilder(optionsListRequestBodyMock);
rawSearchResponseMock.aggregations = {
suggestions: {
buckets: {
@ -592,7 +548,10 @@ describe('options list queries', () => {
},
};
const parsed = suggestionAggBuilder.parse(rawSearchResponseMock);
const parsed = suggestionAggBuilder.parse(
rawSearchResponseMock,
optionsListRequestBodyMock
).suggestions;
/** first, verify that the sorting worked as expected */
expect(Object.keys(parsed)).toMatchInlineSnapshot(`
Array [

201
src/plugins/controls/server/options_list/options_list_cheap_suggestion_queries.ts Normal file
View file

@ -0,0 +1,201 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { get } from 'lodash';
import { getFieldSubtypeNested } from '@kbn/data-views-plugin/common';
import { OptionsListRequestBody, OptionsListSuggestions } from '../../common/options_list/types';
import { getIpRangeQuery, type IpRangeQuery } from '../../common/options_list/ip_search';
import { EsBucket, OptionsListSuggestionAggregationBuilder } from './types';
import {
getEscapedQuery,
getIpBuckets,
getSortType,
} from './options_list_suggestion_query_helpers';
/**
* Suggestion aggregations
*/
export const getCheapSuggestionAggregationBuilder = ({ fieldSpec }: OptionsListRequestBody) => {
if (fieldSpec?.type === 'boolean') {
return cheapSuggestionAggSubtypes.boolean;
}
if (fieldSpec?.type === 'ip') {
return cheapSuggestionAggSubtypes.ip;
}
if (fieldSpec && getFieldSubtypeNested(fieldSpec)) {
return cheapSuggestionAggSubtypes.subtypeNested;
}
return cheapSuggestionAggSubtypes.keywordOrText;
};
const cheapSuggestionAggSubtypes: { [key: string]: OptionsListSuggestionAggregationBuilder } = {
/**
* The "textOrKeyword" query / parser should be used whenever the field is built on some type non-nested string field
* (such as a keyword field or a keyword+text multi-field)
*/
keywordOrText: {
buildAggregation: ({ fieldName, searchString, sort }: OptionsListRequestBody) => ({
suggestions: {
terms: {
field: fieldName,
include: `${getEscapedQuery(searchString)}.*`,
shard_size: 10,
order: getSortType(sort),
},
},
}),
parse: (rawEsResult) => ({
suggestions: get(rawEsResult, 'aggregations.suggestions.buckets').reduce(
(suggestions: OptionsListSuggestions, suggestion: EsBucket) => {
return { ...suggestions, [suggestion.key]: { doc_count: suggestion.doc_count } };
},
{}
),
}),
},
/**
* the "Boolean" query / parser should be used when the options list is built on a field of type boolean. The query is slightly different than a keyword query.
*/
boolean: {
buildAggregation: ({ fieldName, sort }: OptionsListRequestBody) => ({
suggestions: {
terms: {
field: fieldName,
shard_size: 10,
order: getSortType(sort),
},
},
}),
parse: (rawEsResult) => ({
suggestions: get(rawEsResult, 'aggregations.suggestions.buckets')?.reduce(
(suggestions: OptionsListSuggestions, suggestion: EsBucket & { key_as_string: string }) => {
return {
...suggestions,
[suggestion.key_as_string]: { doc_count: suggestion.doc_count },
};
},
{}
),
}),
},
/**
* the "IP" query / parser should be used when the options list is built on a field of type IP.
*/
ip: {
buildAggregation: ({ fieldName, searchString, sort }: OptionsListRequestBody) => {
let ipRangeQuery: IpRangeQuery = {
validSearch: true,
rangeQuery: [
{
key: 'ipv6',
from: '::',
to: 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff',
},
],
};
if (searchString) {
ipRangeQuery = getIpRangeQuery(searchString);
if (!ipRangeQuery.validSearch) {
// ideally should be prevented on the client side but, if somehow an invalid search gets through to the server,
// simply don't return an aggregation query for the ES search request
return undefined;
}
}
return {
suggestions: {
ip_range: {
field: fieldName,
ranges: ipRangeQuery.rangeQuery,
keyed: true,
},
aggs: {
filteredSuggestions: {
terms: {
field: fieldName,
shard_size: 10,
order: getSortType(sort),
},
},
},
},
};
},
parse: (rawEsResult, { sort }) => {
if (!Boolean(rawEsResult.aggregations?.suggestions)) {
// if this is happens, that means there is an invalid search that snuck through to the server side code;
// so, might as well early return with no suggestions
return { suggestions: {} };
}
const buckets: EsBucket[] = [];
getIpBuckets(rawEsResult, buckets, 'ipv4'); // modifies buckets array directly, i.e. "by reference"
getIpBuckets(rawEsResult, buckets, 'ipv6');
const sortedSuggestions =
sort?.direction === 'asc'
? buckets.sort(
(bucketA: EsBucket, bucketB: EsBucket) => bucketA.doc_count - bucketB.doc_count
)
: buckets.sort(
(bucketA: EsBucket, bucketB: EsBucket) => bucketB.doc_count - bucketA.doc_count
);
return {
suggestions: sortedSuggestions
.slice(0, 10) // only return top 10 results
.reduce((suggestions, suggestion: EsBucket) => {
return { ...suggestions, [suggestion.key]: { doc_count: suggestion.doc_count } };
}, {}),
};
},
},
/**
* the "Subtype Nested" query / parser should be used when the options list is built on a field with subtype nested.
*/
subtypeNested: {
buildAggregation: (req: OptionsListRequestBody) => {
const { fieldSpec, fieldName, searchString, sort } = req;
const subTypeNested = fieldSpec && getFieldSubtypeNested(fieldSpec);
if (!subTypeNested) {
// if this field is not subtype nested, fall back to keywordOnly
return cheapSuggestionAggSubtypes.keywordOnly.buildAggregation(req);
}
return {
nestedSuggestions: {
nested: {
path: subTypeNested.nested.path,
},
aggs: {
suggestions: {
terms: {
field: fieldName,
include: `${getEscapedQuery(searchString)}.*`,
shard_size: 10,
order: getSortType(sort),
},
},
},
},
};
},
parse: (rawEsResult) => ({
suggestions: get(rawEsResult, 'aggregations.nestedSuggestions.suggestions.buckets').reduce(
(suggestions: OptionsListSuggestions, suggestion: EsBucket) => {
return { ...suggestions, [suggestion.key]: { doc_count: suggestion.doc_count } };
},
{}
),
}),
},
};

47
src/plugins/controls/server/options_list/options_list_cluster_settings_route.ts Normal file
View file

@ -0,0 +1,47 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { getKbnServerError, reportServerError } from '@kbn/kibana-utils-plugin/server';
import { CoreSetup } from '@kbn/core/server';
export const setupOptionsListClusterSettingsRoute = ({ http }: CoreSetup) => {
const router = http.createRouter();
router.get(
{
path: '/api/kibana/controls/optionsList/getClusterSettings',
validate: false,
},
async (context, _, response) => {
try {
const esClient = (await context.core).elasticsearch.client.asCurrentUser;
const settings = await esClient.cluster.getSettings({
include_defaults: true,
filter_path: '**.allow_expensive_queries',
});
// priority: transient -> persistent -> default
const allowExpensiveQueries: string =
settings.transient?.search?.allow_expensive_queries ??
settings.persistent?.search?.allow_expensive_queries ??
settings.defaults?.search?.allow_expensive_queries ??
// by default, the allowExpensiveQueries cluster setting is undefined; so, we need to treat this the same
// as `true` since that's the way other applications (such as the dashboard listing page) handle this.
'true';
return response.ok({
body: {
allowExpensiveQueries: allowExpensiveQueries === 'true',
},
});
} catch (e) {
const kbnErr = getKbnServerError(e);
return reportServerError(response, kbnErr);
}
}
);
};

676
src/plugins/controls/server/options_list/options_list_expensive_suggestion_queries.test.ts Normal file
View file

@ -0,0 +1,676 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { FieldSpec } from '@kbn/data-views-plugin/common';
import { SearchResponse } from '@elastic/elasticsearch/lib/api/types';
import { getExpensiveSuggestionAggregationBuilder } from './options_list_expensive_suggestion_queries';
import { OptionsListRequestBody } from '../../common/options_list/types';
describe('options list expensive queries', () => {
let rawSearchResponseMock: SearchResponse = {} as SearchResponse;
beforeEach(() => {
rawSearchResponseMock = {
hits: {
total: 10,
max_score: 10,
hits: [],
},
took: 10,
timed_out: false,
_shards: {
failed: 0,
successful: 1,
total: 1,
skipped: 0,
},
aggregations: {},
};
});
describe('suggestion aggregation', () => {
describe('string (keyword, text+keyword, or nested) field', () => {
test('test keyword field, without a search string', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
allowExpensiveQueries: true,
fieldName: 'coolTestField.keyword',
sort: { by: '_key', direction: 'asc' },
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"terms": Object {
"field": "coolTestField.keyword",
"order": Object {
"_key": "asc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "coolTestField.keyword",
},
},
}
`);
});
test('test keyword field, with a search string', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
searchString: 'cooool',
allowExpensiveQueries: true,
fieldName: 'coolTestField.keyword',
sort: { by: '_key', direction: 'desc' },
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"filteredSuggestions": Object {
"aggs": Object {
"suggestions": Object {
"terms": Object {
"field": "coolTestField.keyword",
"order": Object {
"_key": "desc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "coolTestField.keyword",
},
},
},
"filter": Object {
"prefix": Object {
"coolTestField.keyword": Object {
"case_insensitive": true,
"value": "cooool",
},
},
},
},
}
`);
});
test('test nested field, with a search string', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
searchString: 'cooool',
allowExpensiveQueries: true,
fieldName: 'coolNestedField',
sort: { by: '_count', direction: 'asc' },
fieldSpec: { subType: { nested: { path: 'path.to.nested' } } } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"nestedSuggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"aggs": Object {
"suggestions": Object {
"terms": Object {
"field": "coolNestedField",
"order": Object {
"_count": "asc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "coolNestedField",
},
},
},
"filter": Object {
"prefix": Object {
"coolNestedField": Object {
"case_insensitive": true,
"value": "cooool",
},
},
},
},
},
"nested": Object {
"path": "path.to.nested",
},
},
}
`);
});
});
test('boolean field', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'coolean',
allowExpensiveQueries: false,
sort: { by: '_key', direction: 'desc' },
fieldSpec: { type: 'boolean' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"terms": Object {
"field": "coolean",
"order": Object {
"_key": "desc",
},
"shard_size": 10,
},
},
}
`);
});
describe('IP field', () => {
test('without a search string, creates IP range aggregation with default range', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
allowExpensiveQueries: true,
sort: { by: '_key', direction: 'asc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_key": "asc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "clientip",
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "::",
"key": "ipv6",
"to": "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
},
],
},
},
}
`);
});
test('full IPv4 in the search string, creates IP range aggregation with CIDR mask', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
allowExpensiveQueries: true,
searchString: '41.77.243.255',
sort: { by: '_count', direction: 'asc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_count": "asc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "clientip",
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"key": "ipv4",
"mask": "41.77.243.255/32",
},
],
},
},
}
`);
});
test('full IPv6 in the search string, creates IP range aggregation with CIDR mask', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
allowExpensiveQueries: true,
sort: { by: '_key', direction: 'asc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
searchString: 'f688:fb50:6433:bba2:604:f2c:194a:d3c5',
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_key": "asc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "clientip",
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"key": "ipv6",
"mask": "f688:fb50:6433:bba2:604:f2c:194a:d3c5/128",
},
],
},
},
}
`);
});
test('partial IPv4 in the search string, creates IP range aggregation with min and max', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
searchString: '41.77',
allowExpensiveQueries: true,
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_count": "desc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "clientip",
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "41.77.0.0",
"key": "ipv4",
"to": "41.77.255.255",
},
],
},
},
}
`);
});
test('partial IPv46 in the search string, creates IP range aggregation with min and max', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
searchString: 'cdb6:',
allowExpensiveQueries: true,
sort: { by: '_count', direction: 'desc' },
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
expect(suggestionAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"aggs": Object {
"filteredSuggestions": Object {
"terms": Object {
"field": "clientip",
"order": Object {
"_count": "desc",
},
"shard_size": 10,
"size": 10,
},
},
"unique_terms": Object {
"cardinality": Object {
"field": "clientip",
},
},
},
"ip_range": Object {
"field": "clientip",
"keyed": true,
"ranges": Array [
Object {
"from": "cdb6::",
"key": "ipv6",
"to": "cdb6:ffff:ffff:ffff:ffff:ffff:ffff:ffff",
},
],
},
},
}
`);
});
});
});
describe('suggestion parsing', () => {
test('parses string (keyword, text+keyword, or nested) result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
allowExpensiveQueries: true,
fieldName: 'coolTestField.keyword',
fieldSpec: { aggregatable: true } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
rawSearchResponseMock.aggregations = {
suggestions: {
buckets: [
{ doc_count: 5, key: 'cool1' },
{ doc_count: 15, key: 'cool2' },
{ doc_count: 10, key: 'cool3' },
],
},
unique_terms: {
value: 3,
},
};
expect(suggestionAggBuilder.parse(rawSearchResponseMock, optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"cool1": Object {
"doc_count": 5,
},
"cool2": Object {
"doc_count": 15,
},
"cool3": Object {
"doc_count": 10,
},
},
"totalCardinality": 3,
}
`);
});
test('parses boolean result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'coolean',
allowExpensiveQueries: true,
fieldSpec: { type: 'boolean' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
rawSearchResponseMock.aggregations = {
suggestions: {
buckets: [
{ doc_count: 55, key_as_string: 'false' },
{ doc_count: 155, key_as_string: 'true' },
],
},
};
expect(suggestionAggBuilder.parse(rawSearchResponseMock, optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"false": Object {
"doc_count": 55,
},
"true": Object {
"doc_count": 155,
},
},
"totalCardinality": 2,
}
`);
});
test('parses nested result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
searchString: 'co',
fieldName: 'coolNestedField',
allowExpensiveQueries: true,
fieldSpec: { subType: { nested: { path: 'path.to.nested' } } } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
rawSearchResponseMock.aggregations = {
nestedSuggestions: {
filteredSuggestions: {
suggestions: {
buckets: [
{ doc_count: 5, key: 'cool1' },
{ doc_count: 15, key: 'cool2' },
{ doc_count: 10, key: 'cool3' },
],
},
unique_terms: {
value: 3,
},
},
},
};
expect(suggestionAggBuilder.parse(rawSearchResponseMock, optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"suggestions": Object {
"cool1": Object {
"doc_count": 5,
},
"cool2": Object {
"doc_count": 15,
},
"cool3": Object {
"doc_count": 10,
},
},
"totalCardinality": 3,
}
`);
});
test('parses mixed IPv4 and IPv6 result', () => {
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'clientip',
allowExpensiveQueries: true,
fieldSpec: { type: 'ip' } as unknown as FieldSpec,
};
const suggestionAggBuilder = getExpensiveSuggestionAggregationBuilder(
optionsListRequestBodyMock
);
rawSearchResponseMock.aggregations = {
suggestions: {
buckets: {
ipv4: {
from: '0.0.0.0',
to: '255.255.255.255',
filteredSuggestions: {
buckets: [
{ doc_count: 8, key: '21.35.91.62' },
{ doc_count: 8, key: '21.35.91.61' },
{ doc_count: 11, key: '111.52.174.2' },
{ doc_count: 1, key: '56.73.58.63' },
{ doc_count: 9, key: '23.216.241.120' },
{ doc_count: 10, key: '196.162.13.39' },
{ doc_count: 7, key: '203.88.33.151' },
],
},
},
ipv6: {
from: '::',
to: 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff',
filteredSuggestions: {
buckets: [
{ doc_count: 12, key: '52:ae76:5947:5e2a:551:fe6a:712a:c72' },
{ doc_count: 1, key: 'fd:4aa0:c27c:b04:997f:2de1:51b4:8418' },
{ doc_count: 9, key: '28c7:c9a4:42fd:16b0:4de5:e41e:28d9:9172' },
{ doc_count: 6, key: '1ec:aa98:b0a6:d07c:590:18a0:8a33:2eb8' },
{ doc_count: 10, key: 'f7a9:640b:b5a0:1219:8d75:ed94:3c3e:2e63' },
],
},
},
},
},
unique_terms: {
buckets: {
ipv4: {
value: 7,
},
ipv6: {
value: 5,
},
},
},
};
const parsed = suggestionAggBuilder.parse(
rawSearchResponseMock,
optionsListRequestBodyMock
).suggestions;
/** first, verify that the sorting worked as expected */
expect(Object.keys(parsed)).toMatchInlineSnapshot(`
Array [
"52:ae76:5947:5e2a:551:fe6a:712a:c72",
"111.52.174.2",
"196.162.13.39",
"f7a9:640b:b5a0:1219:8d75:ed94:3c3e:2e63",
"23.216.241.120",
"28c7:c9a4:42fd:16b0:4de5:e41e:28d9:9172",
"21.35.91.62",
"21.35.91.61",
"203.88.33.151",
"1ec:aa98:b0a6:d07c:590:18a0:8a33:2eb8",
]
`);
/** then, make sure the object is structured properly */
expect(parsed).toMatchInlineSnapshot(`
Object {
"111.52.174.2": Object {
"doc_count": 11,
},
"196.162.13.39": Object {
"doc_count": 10,
},
"1ec:aa98:b0a6:d07c:590:18a0:8a33:2eb8": Object {
"doc_count": 6,
},
"203.88.33.151": Object {
"doc_count": 7,
},
"21.35.91.61": Object {
"doc_count": 8,
},
"21.35.91.62": Object {
"doc_count": 8,
},
"23.216.241.120": Object {
"doc_count": 9,
},
"28c7:c9a4:42fd:16b0:4de5:e41e:28d9:9172": Object {
"doc_count": 9,
},
"52:ae76:5947:5e2a:551:fe6a:712a:c72": Object {
"doc_count": 12,
},
"f7a9:640b:b5a0:1219:8d75:ed94:3c3e:2e63": Object {
"doc_count": 10,
},
}
`);
});
});
});

217
src/plugins/controls/server/options_list/options_list_expensive_suggestion_queries.ts Normal file
View file

@ -0,0 +1,217 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { get } from 'lodash';
import { getFieldSubtypeNested } from '@kbn/data-views-plugin/common';
import { OptionsListRequestBody, OptionsListSuggestions } from '../../common/options_list/types';
import { getIpRangeQuery, type IpRangeQuery } from '../../common/options_list/ip_search';
import { EsBucket, OptionsListSuggestionAggregationBuilder } from './types';
import { getIpBuckets, getSortType } from './options_list_suggestion_query_helpers';
/**
* Suggestion aggregations
*/
export const getExpensiveSuggestionAggregationBuilder = ({ fieldSpec }: OptionsListRequestBody) => {
if (fieldSpec?.type === 'boolean') {
return expensiveSuggestionAggSubtypes.boolean;
}
if (fieldSpec?.type === 'ip') {
return expensiveSuggestionAggSubtypes.ip;
}
return expensiveSuggestionAggSubtypes.textOrKeywordOrNested;
};
const expensiveSuggestionAggSubtypes: { [key: string]: OptionsListSuggestionAggregationBuilder } = {
/**
* The "textOrKeywordOrNested" query / parser should be used whenever the field is built on some type of string field,
* regardless of if it is keyword only, keyword+text, or some nested keyword/keyword+text field.
*/
textOrKeywordOrNested: {
buildAggregation: ({
searchString,
fieldName,
fieldSpec,
sort,
size,
}: OptionsListRequestBody) => {
const subTypeNested = fieldSpec && getFieldSubtypeNested(fieldSpec);
let textOrKeywordQuery: any = {
suggestions: {
terms: {
size,
field: fieldName,
shard_size: 10,
order: getSortType(sort),
},
},
unique_terms: {
cardinality: {
field: fieldName,
},
},
};
if (searchString) {
textOrKeywordQuery = {
filteredSuggestions: {
filter: {
prefix: {
[fieldName]: {
value: searchString,
case_insensitive: true,
},
},
},
aggs: { ...textOrKeywordQuery },
},
};
}
if (subTypeNested) {
textOrKeywordQuery = {
nestedSuggestions: {
nested: {
path: subTypeNested.nested.path,
},
aggs: {
...textOrKeywordQuery,
},
},
};
}
return textOrKeywordQuery;
},
parse: (rawEsResult, request) => {
let basePath = 'aggregations';
const isNested = request.fieldSpec && getFieldSubtypeNested(request.fieldSpec);
basePath += isNested ? '.nestedSuggestions' : '';
basePath += request.searchString ? '.filteredSuggestions' : '';
const suggestions = get(rawEsResult, `${basePath}.suggestions.buckets`)?.reduce(
(acc: OptionsListSuggestions, suggestion: EsBucket) => {
return { ...acc, [suggestion.key]: { doc_count: suggestion.doc_count } };
},
{}
);
return {
suggestions,
totalCardinality: get(rawEsResult, `${basePath}.unique_terms.value`),
};
},
},
/**
* the "Boolean" query / parser should be used when the options list is built on a field of type boolean. The query is slightly different than a keyword query.
*/
boolean: {
buildAggregation: ({ fieldName, sort }: OptionsListRequestBody) => ({
suggestions: {
terms: {
field: fieldName,
shard_size: 10,
order: getSortType(sort),
},
},
}),
parse: (rawEsResult) => {
const suggestions = get(rawEsResult, 'aggregations.suggestions.buckets')?.reduce(
(acc: OptionsListSuggestions, suggestion: EsBucket & { key_as_string: string }) => {
return {
...acc,
[suggestion.key_as_string]: { doc_count: suggestion.doc_count },
};
},
{}
);
return { suggestions, totalCardinality: Object.keys(suggestions).length }; // cardinality is only ever 0, 1, or 2 so safe to use length here
},
},
/**
* the "IP" query / parser should be used when the options list is built on a field of type IP.
*/
ip: {
buildAggregation: ({ fieldName, searchString, sort, size }: OptionsListRequestBody) => {
let ipRangeQuery: IpRangeQuery = {
validSearch: true,
rangeQuery: [
{
key: 'ipv6',
from: '::',
to: 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff',
},
],
};
if (searchString) {
ipRangeQuery = getIpRangeQuery(searchString);
if (!ipRangeQuery.validSearch) {
// ideally should be prevented on the client side but, if somehow an invalid search gets through to the server,
// simply don't return an aggregation query for the ES search request
return undefined;
}
}
return {
suggestions: {
ip_range: {
field: fieldName,
ranges: ipRangeQuery.rangeQuery,
keyed: true,
},
aggs: {
filteredSuggestions: {
terms: {
size,
field: fieldName,
shard_size: 10,
order: getSortType(sort),
},
},
unique_terms: {
cardinality: {
field: fieldName,
},
},
},
},
};
},
parse: (rawEsResult, request) => {
if (!Boolean(rawEsResult.aggregations?.suggestions)) {
// if this is happens, that means there is an invalid search that snuck through to the server side code;
// so, might as well early return with no suggestions
return { suggestions: {}, totalCardinality: 0 };
}
const buckets: EsBucket[] = [];
getIpBuckets(rawEsResult, buckets, 'ipv4'); // modifies buckets array directly, i.e. "by reference"
getIpBuckets(rawEsResult, buckets, 'ipv6');
const sortedSuggestions =
request.sort?.direction === 'asc'
? buckets.sort(
(bucketA: EsBucket, bucketB: EsBucket) => bucketA.doc_count - bucketB.doc_count
)
: buckets.sort(
(bucketA: EsBucket, bucketB: EsBucket) => bucketB.doc_count - bucketA.doc_count
);
const suggestions: OptionsListSuggestions = sortedSuggestions
.slice(0, request.size)
.reduce((acc: OptionsListSuggestions, suggestion: EsBucket) => {
return { ...acc, [suggestion.key]: { doc_count: suggestion.doc_count } };
}, {});
const totalCardinality =
(get(rawEsResult, `aggregations.suggestions.buckets.ipv4.unique_terms.value`) ?? 0) +
(get(rawEsResult, `aggregations.suggestions.buckets.ipv6.unique_terms.value`) ?? 0);
return {
suggestions,
totalCardinality,
};
},
},
};

289
src/plugins/controls/server/options_list/options_list_queries.ts
View file

@ -1,289 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { get, isEmpty } from 'lodash';
import { SearchResponse } from '@elastic/elasticsearch/lib/api/types';
import { getFieldSubtypeNested } from '@kbn/data-views-plugin/common';
import { OptionsListRequestBody, OptionsListSuggestions } from '../../common/options_list/types';
import {
OPTIONS_LIST_DEFAULT_SORT,
OptionsListSortingType,
} from '../../common/options_list/suggestions_sorting';
import { getIpRangeQuery, type IpRangeQuery } from '../../common/options_list/ip_search';
export interface OptionsListValidationAggregationBuilder {
buildAggregation: (req: OptionsListRequestBody) => unknown;
parse: (response: SearchResponse) => string[];
}
export interface OptionsListSuggestionAggregationBuilder {
buildAggregation: (req: OptionsListRequestBody) => unknown;
parse: (response: SearchResponse) => OptionsListSuggestions;
}
interface EsBucket {
key: string;
doc_count: number;
}
const getSortType = (sort?: OptionsListSortingType) => {
return sort
? { [sort.by]: sort.direction }
: { [OPTIONS_LIST_DEFAULT_SORT.by]: OPTIONS_LIST_DEFAULT_SORT.direction };
};
/**
* Validation aggregations
*/
export const getValidationAggregationBuilder: () => OptionsListValidationAggregationBuilder =
() => ({
buildAggregation: ({ selectedOptions, fieldName }: OptionsListRequestBody) => {
let selectedOptionsFilters;
if (selectedOptions) {
selectedOptionsFilters = selectedOptions.reduce((acc, currentOption) => {
acc[currentOption] = { match: { [fieldName]: currentOption } };
return acc;
}, {} as { [key: string]: { match: { [key: string]: string } } });
}
return selectedOptionsFilters && !isEmpty(selectedOptionsFilters)
? {
filters: {
filters: selectedOptionsFilters,
},
}
: undefined;
},
parse: (rawEsResult) => {
const rawInvalidSuggestions = get(rawEsResult, 'aggregations.validation.buckets');
return rawInvalidSuggestions && !isEmpty(rawInvalidSuggestions)
? Object.keys(rawInvalidSuggestions).filter(
(key) => rawInvalidSuggestions[key].doc_count === 0
)
: [];
},
});
/**
* Suggestion aggregations
*/
export const getSuggestionAggregationBuilder = ({
fieldSpec,
textFieldName,
searchString,
}: OptionsListRequestBody) => {
if (textFieldName && fieldSpec?.aggregatable && searchString) {
return suggestionAggSubtypes.keywordAndText;
}
if (fieldSpec?.type === 'boolean') {
return suggestionAggSubtypes.boolean;
}
if (fieldSpec?.type === 'ip') {
return suggestionAggSubtypes.ip;
}
if (fieldSpec && getFieldSubtypeNested(fieldSpec)) {
return suggestionAggSubtypes.subtypeNested;
}
return suggestionAggSubtypes.keywordOnly;
};
const getEscapedQuery = (q: string = '') =>
q.replace(/[.?+*|{}[\]()"\\#@&<>~]/g, (match) => `\\${match}`);
const getIpBuckets = (rawEsResult: any, combinedBuckets: EsBucket[], type: 'ipv4' | 'ipv6') => {
const results = get(
rawEsResult,
`aggregations.suggestions.buckets.${type}.filteredSuggestions.buckets`
);
if (results) {
results.forEach((suggestion: EsBucket) => combinedBuckets.push(suggestion));
}
};
const suggestionAggSubtypes: { [key: string]: OptionsListSuggestionAggregationBuilder } = {
/**
* the "Keyword only" query / parser should be used when the options list is built on a field which has only keyword mappings.
*/
keywordOnly: {
buildAggregation: ({ fieldName, searchString, sort }: OptionsListRequestBody) => ({
terms: {
field: fieldName,
include: `${getEscapedQuery(searchString)}.*`,
execution_hint: 'map',
shard_size: 10,
order: getSortType(sort),
},
}),
parse: (rawEsResult) =>
get(rawEsResult, 'aggregations.suggestions.buckets').reduce(
(suggestions: OptionsListSuggestions, suggestion: EsBucket) => {
return { ...suggestions, [suggestion.key]: { doc_count: suggestion.doc_count } };
},
{}
),
},
/**
* the "Keyword and text" query / parser should be used when the options list is built on a multi-field which has both keyword and text mappings. It supports case-insensitive searching
*/
keywordAndText: {
buildAggregation: (req: OptionsListRequestBody) => {
if (!req.textFieldName) {
// if there is no textFieldName specified, or if there is no search string yet fall back to keywordOnly
return suggestionAggSubtypes.keywordOnly.buildAggregation(req);
}
const { fieldName, searchString, textFieldName, sort } = req;
return {
filter: {
match_phrase_prefix: {
[textFieldName]: searchString,
},
},
aggs: {
keywordSuggestions: {
terms: {
field: fieldName,
shard_size: 10,
order: getSortType(sort),
},
},
},
};
},
parse: (rawEsResult) =>
get(rawEsResult, 'aggregations.suggestions.keywordSuggestions.buckets').reduce(
(suggestions: OptionsListSuggestions, suggestion: EsBucket) => {
return { ...suggestions, [suggestion.key]: { doc_count: suggestion.doc_count } };
},
{}
),
},
/**
* the "Boolean" query / parser should be used when the options list is built on a field of type boolean. The query is slightly different than a keyword query.
*/
boolean: {
buildAggregation: ({ fieldName, sort }: OptionsListRequestBody) => ({
terms: {
field: fieldName,
execution_hint: 'map',
shard_size: 10,
order: getSortType(sort),
},
}),
parse: (rawEsResult) =>
get(rawEsResult, 'aggregations.suggestions.buckets')?.reduce(
(suggestions: OptionsListSuggestions, suggestion: EsBucket & { key_as_string: string }) => {
return {
...suggestions,
[suggestion.key_as_string]: { doc_count: suggestion.doc_count },
};
},
{}
),
},
/**
* the "IP" query / parser should be used when the options list is built on a field of type IP.
*/
ip: {
buildAggregation: ({ fieldName, searchString, sort }: OptionsListRequestBody) => {
let ipRangeQuery: IpRangeQuery = {
validSearch: true,
rangeQuery: [
{
key: 'ipv6',
from: '::',
to: 'ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff',
},
],
};
if (searchString) {
ipRangeQuery = getIpRangeQuery(searchString);
if (!ipRangeQuery.validSearch) {
// ideally should be prevented on the client side but, if somehow an invalid search gets through to the server,
// simply don't return an aggregation query for the ES search request
return undefined;
}
}
return {
ip_range: {
field: fieldName,
ranges: ipRangeQuery.rangeQuery,
keyed: true,
},
aggs: {
filteredSuggestions: {
terms: {
field: fieldName,
execution_hint: 'map',
shard_size: 10,
order: getSortType(sort),
},
},
},
};
},
parse: (rawEsResult) => {
if (!Boolean(rawEsResult.aggregations?.suggestions)) {
// if this is happens, that means there is an invalid search that snuck through to the server side code;
// so, might as well early return with no suggestions
return [];
}
const buckets: EsBucket[] = [];
getIpBuckets(rawEsResult, buckets, 'ipv4'); // modifies buckets array directly, i.e. "by reference"
getIpBuckets(rawEsResult, buckets, 'ipv6');
return buckets
.sort((bucketA: EsBucket, bucketB: EsBucket) => bucketB.doc_count - bucketA.doc_count)
.slice(0, 10) // only return top 10 results
.reduce((suggestions, suggestion: EsBucket) => {
return { ...suggestions, [suggestion.key]: { doc_count: suggestion.doc_count } };
}, {});
},
},
/**
* the "Subtype Nested" query / parser should be used when the options list is built on a field with subtype nested.
*/
subtypeNested: {
buildAggregation: (req: OptionsListRequestBody) => {
const { fieldSpec, fieldName, searchString, sort } = req;
const subTypeNested = fieldSpec && getFieldSubtypeNested(fieldSpec);
if (!subTypeNested) {
// if this field is not subtype nested, fall back to keywordOnly
return suggestionAggSubtypes.keywordOnly.buildAggregation(req);
}
return {
nested: {
path: subTypeNested.nested.path,
},
aggs: {
nestedSuggestions: {
terms: {
field: fieldName,
include: `${getEscapedQuery(searchString)}.*`,
execution_hint: 'map',
shard_size: 10,
order: getSortType(sort),
},
},
},
};
},
parse: (rawEsResult) =>
get(rawEsResult, 'aggregations.suggestions.nestedSuggestions.buckets').reduce(
(suggestions: OptionsListSuggestions, suggestion: EsBucket) => {
return { ...suggestions, [suggestion.key]: { doc_count: suggestion.doc_count } };
},
{}
),
},
};

38
src/plugins/controls/server/options_list/options_list_suggestion_query_helpers.ts Normal file
View file

@ -0,0 +1,38 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { get } from 'lodash';
import { EsBucket } from './types';
import {
OPTIONS_LIST_DEFAULT_SORT,
OptionsListSortingType,
} from '../../common/options_list/suggestions_sorting';
export const getSortType = (sort?: OptionsListSortingType) => {
return sort
? { [sort.by]: sort.direction }
: { [OPTIONS_LIST_DEFAULT_SORT.by]: OPTIONS_LIST_DEFAULT_SORT.direction };
};
export const getEscapedQuery = (q: string = '') =>
q.replace(/[.?+*|{}[\]()"\\#@&<>~]/g, (match) => `\\${match}`);
export const getIpBuckets = (
rawEsResult: any,
combinedBuckets: EsBucket[],
type: 'ipv4' | 'ipv6'
) => {
const results = get(
rawEsResult,
`aggregations.suggestions.buckets.${type}.filteredSuggestions.buckets`
);
if (results) {
results.forEach((suggestion: EsBucket) => combinedBuckets.push(suggestion));
}
};

41
src/plugins/controls/server/options_list/options_list_suggestions_route.ts
View file

@ -7,7 +7,6 @@
*/
import { Observable } from 'rxjs';
import { get } from 'lodash';
import { PluginSetup as UnifiedSearchPluginSetup } from '@kbn/unified-search-plugin/server';
import { getKbnServerError, reportServerError } from '@kbn/kibana-utils-plugin/server';
@ -16,10 +15,10 @@ import { SearchRequest } from '@kbn/data-plugin/common';
import { schema } from '@kbn/config-schema';
import { OptionsListRequestBody, OptionsListResponse } from '../../common/options_list/types';
import {
getSuggestionAggregationBuilder,
getValidationAggregationBuilder,
} from './options_list_queries';
import { getValidationAggregationBuilder } from './options_list_validation_queries';
import { getExpensiveSuggestionAggregationBuilder } from './options_list_expensive_suggestion_queries';
import { getCheapSuggestionAggregationBuilder } from './options_list_cheap_suggestion_queries';
import { OptionsListSuggestionAggregationBuilder } from './types';
export const setupOptionsListSuggestionsRoute = (
{ http }: CoreSetup,
@ -39,10 +38,12 @@ export const setupOptionsListSuggestionsRoute = (
),
body: schema.object(
{
size: schema.number(),
fieldName: schema.string(),
sort: schema.maybe(schema.any()),
filters: schema.maybe(schema.any()),
fieldSpec: schema.maybe(schema.any()),
allowExpensiveQueries: schema.boolean(),
searchString: schema.maybe(schema.string()),
selectedOptions: schema.maybe(schema.arrayOf(schema.string())),
},
@ -55,6 +56,7 @@ export const setupOptionsListSuggestionsRoute = (
const suggestionRequest: OptionsListRequestBody = request.body;
const { index } = request.params;
const esClient = (await context.core).elasticsearch.client.asCurrentUser;
const suggestionsResponse = await getOptionsListSuggestions({
abortedEvent$: request.events.aborted$,
request: suggestionRequest,
@ -86,21 +88,21 @@ export const setupOptionsListSuggestionsRoute = (
/**
* Build ES Query
*/
const { runPastTimeout, filters, fieldName, runtimeFieldMap } = request;
const { runPastTimeout, filters, runtimeFieldMap, allowExpensiveQueries } = request;
const { terminateAfter, timeout } = getAutocompleteSettings();
const timeoutSettings = runPastTimeout
? {}
: { timeout: `${timeout}ms`, terminate_after: terminateAfter };
const suggestionBuilder = getSuggestionAggregationBuilder(request);
let suggestionBuilder: OptionsListSuggestionAggregationBuilder;
if (allowExpensiveQueries) {
suggestionBuilder = getExpensiveSuggestionAggregationBuilder(request);
} else {
suggestionBuilder = getCheapSuggestionAggregationBuilder(request);
}
const validationBuilder = getValidationAggregationBuilder();
const builtSuggestionAggregation = suggestionBuilder.buildAggregation(request);
const suggestionAggregation = builtSuggestionAggregation
? {
suggestions: builtSuggestionAggregation,
}
: {};
const suggestionAggregation: any = suggestionBuilder.buildAggregation(request) ?? {};
const builtValidationAggregation = validationBuilder.buildAggregation(request);
const validationAggregations = builtValidationAggregation
? {
@ -118,17 +120,11 @@ export const setupOptionsListSuggestionsRoute = (
aggs: {
...suggestionAggregation,
...validationAggregations,
unique_terms: {
cardinality: {
field: fieldName,
},
},
},
runtime_mappings: {
...runtimeFieldMap,
},
};
/**
* Run ES query
*/
@ -137,14 +133,13 @@ export const setupOptionsListSuggestionsRoute = (
/**
* Parse ES response into Options List Response
*/
const totalCardinality = get(rawEsResult, 'aggregations.unique_terms.value');
const suggestions = suggestionBuilder.parse(rawEsResult);
const results = suggestionBuilder.parse(rawEsResult, request);
const totalCardinality = results.totalCardinality;
const invalidSelections = validationBuilder.parse(rawEsResult);
return {
suggestions,
suggestions: results.suggestions,
totalCardinality,
invalidSelections,
rejected: false,
};
};
};

104
src/plugins/controls/server/options_list/options_list_validation_queries.test.ts Normal file
View file

@ -0,0 +1,104 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { SearchResponse } from '@elastic/elasticsearch/lib/api/types';
import { OptionsListRequestBody } from '../../common/options_list/types';
import { getValidationAggregationBuilder } from './options_list_validation_queries';
describe('options list queries', () => {
let rawSearchResponseMock: SearchResponse = {} as SearchResponse;
beforeEach(() => {
rawSearchResponseMock = {
hits: {
total: 10,
max_score: 10,
hits: [],
},
took: 10,
timed_out: false,
_shards: {
failed: 0,
successful: 1,
total: 1,
skipped: 0,
},
aggregations: {},
};
});
describe('validation aggregation and parsing', () => {
test('creates validation aggregation when given selections', () => {
const validationAggBuilder = getValidationAggregationBuilder();
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'coolTestField',
allowExpensiveQueries: true,
selectedOptions: ['coolOption1', 'coolOption2', 'coolOption3'],
};
expect(validationAggBuilder.buildAggregation(optionsListRequestBodyMock))
.toMatchInlineSnapshot(`
Object {
"filters": Object {
"filters": Object {
"coolOption1": Object {
"match": Object {
"coolTestField": "coolOption1",
},
},
"coolOption2": Object {
"match": Object {
"coolTestField": "coolOption2",
},
},
"coolOption3": Object {
"match": Object {
"coolTestField": "coolOption3",
},
},
},
},
}
`);
});
test('returns undefined when not given selections', () => {
const validationAggBuilder = getValidationAggregationBuilder();
const optionsListRequestBodyMock: OptionsListRequestBody = {
size: 10,
fieldName: 'coolTestField',
allowExpensiveQueries: true,
};
expect(validationAggBuilder.buildAggregation(optionsListRequestBodyMock)).toBeUndefined();
});
test('parses validation result', () => {
const validationAggBuilder = getValidationAggregationBuilder();
rawSearchResponseMock.aggregations = {
validation: {
buckets: {
cool1: { doc_count: 0 },
cool2: { doc_count: 15 },
cool3: { doc_count: 0 },
cool4: { doc_count: 2 },
cool5: { doc_count: 112 },
cool6: { doc_count: 0 },
},
},
};
expect(validationAggBuilder.parse(rawSearchResponseMock)).toMatchInlineSnapshot(`
Array [
"cool1",
"cool3",
"cool6",
]
`);
});
});
});

43
src/plugins/controls/server/options_list/options_list_validation_queries.ts Normal file
View file

@ -0,0 +1,43 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { get, isEmpty } from 'lodash';
import { OptionsListRequestBody } from '../../common/options_list/types';
import { OptionsListValidationAggregationBuilder } from './types';
/**
* Validation aggregations
*/
export const getValidationAggregationBuilder: () => OptionsListValidationAggregationBuilder =
() => ({
buildAggregation: ({ selectedOptions, fieldName }: OptionsListRequestBody) => {
let selectedOptionsFilters;
if (selectedOptions) {
selectedOptionsFilters = selectedOptions.reduce((acc, currentOption) => {
acc[currentOption] = { match: { [fieldName]: currentOption } };
return acc;
}, {} as { [key: string]: { match: { [key: string]: string } } });
}
return selectedOptionsFilters && !isEmpty(selectedOptionsFilters)
? {
filters: {
filters: selectedOptionsFilters,
},
}
: undefined;
},
parse: (rawEsResult) => {
const rawInvalidSuggestions = get(rawEsResult, 'aggregations.validation.buckets');
return rawInvalidSuggestions && !isEmpty(rawInvalidSuggestions)
? Object.keys(rawInvalidSuggestions).filter(
(key) => rawInvalidSuggestions[key].doc_count === 0
)
: [];
},
});

28
src/plugins/controls/server/options_list/types.ts Normal file
View file

@ -0,0 +1,28 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { SearchResponse } from '@elastic/elasticsearch/lib/api/types';
import {
OptionsListRequestBody,
OptionsListParsedSuggestions,
} from '../../common/options_list/types';
export interface EsBucket {
key: string;
doc_count: number;
}
export interface OptionsListValidationAggregationBuilder {
buildAggregation: (req: OptionsListRequestBody) => unknown;
parse: (response: SearchResponse) => string[];
}
export interface OptionsListSuggestionAggregationBuilder {
buildAggregation: (req: OptionsListRequestBody) => unknown;
parse: (response: SearchResponse, req: OptionsListRequestBody) => OptionsListParsedSuggestions;
}

3
src/plugins/controls/server/plugin.ts
View file

@ -15,6 +15,7 @@ import { controlGroupContainerPersistableStateServiceFactory } from './control_g
import { optionsListPersistableStateServiceFactory } from './options_list/options_list_embeddable_factory';
import { rangeSliderPersistableStateServiceFactory } from './range_slider/range_slider_embeddable_factory';
import { timeSliderPersistableStateServiceFactory } from './time_slider/time_slider_embeddable_factory';
import { setupOptionsListClusterSettingsRoute } from './options_list/options_list_cluster_settings_route';
interface SetupDeps {
embeddable: EmbeddableSetup;
@ -30,7 +31,7 @@ export class ControlsPlugin implements Plugin<object, object, SetupDeps> {
embeddable.registerEmbeddableFactory(optionsListPersistableStateServiceFactory());
embeddable.registerEmbeddableFactory(rangeSliderPersistableStateServiceFactory());
embeddable.registerEmbeddableFactory(timeSliderPersistableStateServiceFactory());
setupOptionsListClusterSettingsRoute(core);
setupOptionsListSuggestionsRoute(core, unifiedSearch.autocomplete.getAutocompleteSettings);
return {};
}

1
src/plugins/controls/tsconfig.json
View file

@ -32,6 +32,7 @@
"@kbn/test-jest-helpers",
"@kbn/config-schema",
"@kbn/storybook",
"@kbn/ui-theme",
],
"exclude": [
"target/**/*",

2
src/plugins/dashboard/public/dashboard_container/embeddable/dashboard_container.tsx
View file

@ -96,7 +96,7 @@ export class DashboardContainer extends Container<InheritedChildInput, Dashboard
public controlGroup?: ControlGroupContainer;
// Dashboard State
private onDestroyControlGroup?: () => void;
public onDestroyControlGroup?: () => void;
private subscriptions: Subscription = new Subscription();
private initialized$: BehaviorSubject<boolean> = new BehaviorSubject<boolean>(false);

11
src/plugins/dashboard/public/dashboard_container/embeddable/integrations/controls/dashboard_control_group_integration.ts
View file

@ -65,12 +65,19 @@ export async function startControlGroupIntegration(
return;
}
this.untilInitialized().then(() => startSyncingDashboardControlGroup.bind(this)());
this.untilInitialized().then(() => {
const stopSyncingControlGroup =
startSyncingDashboardControlGroup.bind(this)()?.stopSyncingWithControlGroup;
this.onDestroyControlGroup = () => {
stopSyncingControlGroup?.();
this.controlGroup?.destroy();
};
});
await controlGroup.untilInitialized();
return controlGroup;
}
async function startSyncingDashboardControlGroup(this: DashboardContainer) {
function startSyncingDashboardControlGroup(this: DashboardContainer) {
if (!this.controlGroup) return;
const subscriptions = new Subscription();

10
src/plugins/embeddable/public/lib/embeddables/error_embedabble.scss
View file

@ -1,10 +0,0 @@
.errorEmbeddableCompact__popoverAnchor {
max-width: 100%;
}
.errorEmbeddableCompact__button {
padding-left: 0 !important;
.euiIcon {
margin-right: $euiSizeXS * .5;
}
}

20
src/plugins/embeddable/public/lib/embeddables/error_embeddable.scss Normal file
View file

@ -0,0 +1,20 @@
.errorEmbeddableCompact__popover {
height: 100%;
}
.errorEmbeddableCompact__popoverAnchor {
max-width: 100%;
}
.errorEmbeddableCompact__button {
:hover {
text-decoration: underline;
text-decoration-color: $euiTextSubduedColor !important;
}
}
.errorEmbeddableCompact__text {
font-size: $euiSizeM;
color: $euiTextSubduedColor;
}

1
src/plugins/embeddable/public/lib/embeddables/error_embeddable.tsx
View file

@ -11,6 +11,7 @@ import { EmbeddablePanelError } from '../panel/embeddable_panel_error';
import { Embeddable } from './embeddable';
import { EmbeddableInput, EmbeddableOutput, IEmbeddable } from './i_embeddable';
import { IContainer } from '../containers';
import './error_embeddable.scss';
export const ERROR_EMBEDDABLE_TYPE = 'error';

20
test/functional/apps/dashboard_elements/controls/options_list/index.ts
View file

@ -14,16 +14,12 @@ export default function ({ loadTestFile, getService, getPageObjects }: FtrProvid
const elasticChart = getService('elasticChart');
const security = getService('security');
const { timePicker, dashboard } = getPageObjects([
'dashboardControls',
'timePicker',
'dashboard',
'common',
]);
const { timePicker, dashboard, common } = getPageObjects(['timePicker', 'dashboard', 'common']);
async function setup() {
const setup = async () => {
await security.testUser.setRoles(['kibana_admin', 'test_logstash_reader', 'animals']);
await common.navigateToApp('dashboard');
await dashboard.gotoDashboardLandingPage();
await dashboard.clickNewDashboard();
await timePicker.setDefaultDataRange();
@ -32,13 +28,13 @@ export default function ({ loadTestFile, getService, getPageObjects }: FtrProvid
exitFromEditMode: false,
storeTimeWithDashboard: true,
});
}
};
async function teardown() {
const teardown = async () => {
await security.testUser.restoreDefaults();
}
};
describe('Options list control', function () {
describe('Options list control', async () => {
before(setup);
after(teardown);
@ -46,5 +42,7 @@ export default function ({ loadTestFile, getService, getPageObjects }: FtrProvid
loadTestFile(require.resolve('./options_list_dashboard_interaction'));
loadTestFile(require.resolve('./options_list_suggestions'));
loadTestFile(require.resolve('./options_list_validation'));
loadTestFile(require.resolve('./options_list_allow_expensive_queries_off.ts'));
});
}

96
test/functional/apps/dashboard_elements/controls/options_list/options_list_allow_expensive_queries_off.ts Normal file
View file

@ -0,0 +1,96 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { OPTIONS_LIST_CONTROL } from '@kbn/controls-plugin/common';
import expect from '@kbn/expect';
import { OPTIONS_LIST_ANIMAL_SOUND_SUGGESTIONS } from '../../../../page_objects/dashboard_page_controls';
import { FtrProviderContext } from '../../../../ftr_provider_context';
export default function ({ getService, getPageObjects }: FtrProviderContext) {
const browser = getService('browser');
const { dashboardControls, timePicker, console, common, dashboard, header } = getPageObjects([
'dashboardControls',
'timePicker',
'dashboard',
'console',
'common',
'header',
]);
const setAllowExpensiveQueries = async (value: boolean) => {
await common.navigateToApp('console');
await console.closeHelpIfExists();
await console.clearTextArea();
await console.enterRequest(
'\nPUT _cluster/settings\n{"transient": {"search.allow_expensive_queries": ' + value + '}}'
);
await console.clickPlay();
await header.waitUntilLoadingHasFinished();
await browser.refresh();
};
describe('Allow expensive queries setting is off', () => {
let controlId: string;
before(async () => {
await setAllowExpensiveQueries(false);
await common.navigateToApp('dashboard');
await dashboard.clickNewDashboard();
await dashboard.ensureDashboardIsInEditMode();
await timePicker.setDefaultDataRange();
await header.waitUntilLoadingHasFinished();
await dashboardControls.createControl({
controlType: OPTIONS_LIST_CONTROL,
dataViewTitle: 'animals-*',
fieldName: 'sound.keyword',
});
controlId = (await dashboardControls.getAllControlIds())[0];
});
after(async () => {
await dashboard.gotoDashboardLandingPage();
await dashboard.clickUnsavedChangesDiscard(`discard-unsaved-New-Dashboard`);
await setAllowExpensiveQueries(true);
});
it('Shows available options in options list', async () => {
await dashboardControls.ensureAvailableOptionsEqual(controlId, {
suggestions: OPTIONS_LIST_ANIMAL_SOUND_SUGGESTIONS,
invalidSelections: [],
});
});
it('Can search options list for available options', async () => {
await dashboardControls.optionsListOpenPopover(controlId);
await dashboardControls.optionsListPopoverSearchForOption('meo');
await dashboardControls.ensureAvailableOptionsEqual(
controlId,
{
suggestions: { meow: OPTIONS_LIST_ANIMAL_SOUND_SUGGESTIONS.meow },
invalidSelections: [],
},
true
);
await dashboardControls.optionsListPopoverClearSearch();
await dashboardControls.optionsListEnsurePopoverIsClosed(controlId);
});
it('Can search options list for available options - case sensitive', async () => {
await dashboardControls.optionsListOpenPopover(controlId);
await dashboardControls.optionsListPopoverSearchForOption('MEO');
const cardinality = await dashboardControls.optionsListPopoverGetAvailableOptionsCount();
expect(cardinality).to.be(0);
await dashboardControls.optionsListPopoverClearSearch();
await dashboardControls.optionsListEnsurePopoverIsClosed(controlId);
});
});
}

13
test/functional/page_objects/dashboard_page_controls.ts
View file

@ -428,9 +428,22 @@ export class DashboardPageControls extends FtrService {
await this.retry.try(async () => {
expect(await this.optionsListPopoverGetAvailableOptions()).to.eql(expectation);
});
if (await this.testSubjects.exists('optionsList-cardinality-label')) {
expect(await this.optionsListGetCardinalityValue()).to.be(
Object.keys(expectation.suggestions).length.toLocaleString()
);
}
if (!skipOpen) await this.optionsListEnsurePopoverIsClosed(controlId);
}
public async optionsListGetCardinalityValue() {
this.log.debug(`getting the value of the cardinality badge`);
const cardinalityLabel = await (
await this.testSubjects.find('optionsList-cardinality-label')
).getVisibleText();
return cardinalityLabel.split(' ')[0];
}
public async optionsListPopoverSearchForOption(search: string) {
this.log.debug(`searching for ${search} in options list`);
await this.optionsListPopoverAssertOpen();

2
x-pack/plugins/translations/translations/fr-FR.json
View file

@ -352,9 +352,7 @@
"controls.optionsList.errors.dataViewNotFound": "Impossible de localiser la vue de données : {dataViewId}",
"controls.optionsList.errors.fieldNotFound": "Impossible de localiser le champ : {fieldName}",
"controls.optionsList.popover.ariaLabel": "Fenêtre contextuelle pour le contrôle {fieldName}",
"controls.optionsList.popover.cardinalityPlaceholder": "Rechercher {totalOptions} {totalOptions, plural, one {option disponible} other {options disponibles}}",
"controls.optionsList.popover.invalidSelectionsSectionTitle": "{invalidSelectionCount, plural, one {Sélection ignorée} other {Sélections ignorées}}",
"controls.optionsList.popover.invalidSelectionsTooltip": "{selectedOptions} {selectedOptions, plural, one {option sélectionnée} other {options sélectionnées}} {selectedOptions, plural, one {est ignorée} other {sont ignorées}}, car {selectedOptions, plural, one {elle n'est plus présente} other {elles ne sont plus présentes}} dans les données.",
"controls.rangeSlider.errors.dataViewNotFound": "Impossible de localiser la vue de données : {dataViewId}",
"controls.rangeSlider.errors.fieldNotFound": "Impossible de localiser le champ : {fieldName}",
"controls.controlGroup.emptyState.addControlButtonTitle": "Ajouter un contrôle",

2
x-pack/plugins/translations/translations/ja-JP.json
View file

@ -354,9 +354,7 @@
"controls.optionsList.errors.dataViewNotFound": "データビュー{dataViewId}が見つかりませんでした",
"controls.optionsList.errors.fieldNotFound": "フィールド{fieldName}が見つかりませんでした",
"controls.optionsList.popover.ariaLabel": "{fieldName}コントロールのポップオーバー",
"controls.optionsList.popover.cardinalityPlaceholder": "{totalOptions}個の使用可能な{totalOptions, plural, other {オプション}}を検索",
"controls.optionsList.popover.invalidSelectionsSectionTitle": "{invalidSelectionCount, plural, other {個の選択項目}}",
"controls.optionsList.popover.invalidSelectionsTooltip": "{selectedOptions}個の選択した{selectedOptions, plural, other {オプション}} {selectedOptions, plural, other {が}}無視されます。{selectedOptions, plural, other {オプションが}}データに存在しません。",
"controls.rangeSlider.errors.dataViewNotFound": "データビュー{dataViewId}が見つかりませんでした",
"controls.rangeSlider.errors.fieldNotFound": "フィールド{fieldName}が見つかりませんでした",
"controls.controlGroup.emptyState.addControlButtonTitle": "コントロールを追加",

2
x-pack/plugins/translations/translations/zh-CN.json
View file

@ -354,9 +354,7 @@
"controls.optionsList.errors.dataViewNotFound": "找不到数据视图：{dataViewId}",
"controls.optionsList.errors.fieldNotFound": "找不到字段：{fieldName}",
"controls.optionsList.popover.ariaLabel": "{fieldName} 控件的弹出框",
"controls.optionsList.popover.cardinalityPlaceholder": "搜索 {totalOptions} 个可用{totalOptions, plural, other {选项}}",
"controls.optionsList.popover.invalidSelectionsSectionTitle": "已忽略{invalidSelectionCount, plural, other {个选择}}",
"controls.optionsList.popover.invalidSelectionsTooltip": "{selectedOptions} 个选定{selectedOptions, plural, other {选项}} {selectedOptions, plural, other {已}}忽略，因为{selectedOptions, plural, one {其} other {它们}}已不再在数据中。",
"controls.rangeSlider.errors.dataViewNotFound": "找不到数据视图：{dataViewId}",
"controls.rangeSlider.errors.fieldNotFound": "找不到字段：{fieldName}",
"controls.controlGroup.emptyState.addControlButtonTitle": "添加控件",