[Discover] New field stats in Discover sidebar popover (#139072)

* [UnifiedFieldList] Bootstrap a new unifiedFieldList plugin

* [UnifiedFieldList] Move backend API for field stats from Lens to UnifiedFieldList plugin

* [CI] Auto-commit changed files from 'node scripts/build_plugin_list_docs'

* [Discover] Address CI checks

* [UnifiedFieldList] Move field stats UI from Lens to UnifiedFieldList plugin

* [Discover] Integrate FieldStats into Discover field popover

* [Discover] Show both views side to side

* [Discover] Allow for some customization

* [Discover] Allow for more customization

* [UnifiedFieldList] Remove temporary code

* [UnifiedFieldList] Extract styles

* [UnifiedFieldList] Fix after merge

* [UnifiedFieldList] Extend i18n

* [UnifiedFieldList] Migrate stats API from server to public

* [UnifiedFieldList] Update types

* [UnifiedFieldList] Update Lens tests

* [UnifiedFieldList] Update Lens tests

* [UnifiedFieldList] Before merging

* [UnifiedFieldList] After merging

* [UnifiedFieldList] Refactor localization keys

* [UnifiedFieldList] Update types

* [UnifiedFieldList] Reintroduce server API for field stats and refactor integration tests

* [UnifiedFieldList] Update limits

* [UnifiedFieldList] Rename the component

* [UnifiedFieldList] Improve types

* [UnifiedFieldList] Add AbortController

* [UnifiedFieldList] Render counts in PopoverFooter in Lens

* [UnifiedFieldList] Hide new stats from Discover for now

* [UnifiedFieldList] Fix tests

* [UnifiedFieldList] Rename to loadFieldStats

* [UnifiedFieldList] Rearrange utils

* [UnifiedFieldList] Fix types

* [UnifiedFieldList] Fix references

* [UnifiedFieldList] Use emotion css

* [UnifiedFieldList] Increase limits

* [UnifiedFieldList] Add first tests

* [UnifiedFieldList] Add more tests

* [UnifiedFieldList] Refactor interface to accept services object

* [UnifiedFieldList] Update types

* [UnifiedFieldList] Add docs

* [CI] Auto-commit changed files from 'node scripts/build_plugin_list_docs'

* [UnifiedFieldList] Add missing references

* [UnifiedFieldList] Tmp

* [UnifiedFieldList] Revert changes from Discover for now

* Revert "[UnifiedFieldList] Revert changes from Discover for now"

This reverts commit 3f4ae6e395.

* [Discover] Extract top values UI into a separate component. Update colors.

* [Discover] Extract bucket UI into a separate component. Update colors.

* [Discover] Update styling

* [Discover] Fix empty values

* [Discover] Allow to customize colors

* [Discover] Add filter buttons

* [Discover] Rename props

* [Discover] Improve format

* [Discover] Add a switch in Settings. Move Visualize button into PopoverFooter.

* [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix'

* [Discover] Fix props

* [Discover] Hide filter buttons for Other section

* [Discover] Simplify default messages when analysis is not available

* [Discover] Small update

* [Discover] Remove translations

* [Discover] Update some tests

* [Discover] Fallback to old Discover logic and show examples for non-aggregatable fields

* [Discover] Exclude vector fields

* [Discover] Don't call details unless for legacy code

* [Discover] Fix types

* [Discover] Small update for stories

* [Discover] Adapt tests

* [Discover] Update tests

* [Discover] Update tests

* [Discover] Update tests

* [Discover] Update tests

* [Discover] Update tests

* [Discover] Add tooltips. Update examples sample values. Update tests.

* [Discover] Close the popover when filter is pressed

* [Discover] Add functional tests for non-aggregatable fields

* [Discover] Fix query

* [Discover] Add more tests

* [Discover] Add more tests

* [Discover] Add more tests

* [Discover] Add more tests

* [Discover] Fix time range for field stats

* [Discover] Remove sort param from examples query

* [Discover] Prevent reduntant requests

* [Discover] Increase examples size

* [Discover] Add exist filter to Discover popover

* [Discover] Update label

* [Discover] Update logic for picking a multifield

* [Discover] Fix how percentage is calculated for Examples view (non-aggregatable fields)

* [Discover] Update copy and uncomment console error

* [Discover] Add "no data" message and update field type check in examples

* [Discover] Update type checks and no-data copy

* [Discover] Update copy

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
This commit is contained in:
Julia Rechkunova 2022-09-15 14:26:41 +02:00 committed by GitHub
parent 21c36c563c
commit cae3a33de3
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
48 changed files with 1765 additions and 477 deletions

View file

@ -320,6 +320,9 @@ the minimum and maximum values of a numeric field or a map of a geo field.
[[discover:showMultiFields]]`discover:showMultiFields`::
Controls the display of multi-fields in the expanded document view.
[[discover:showLegacyFieldTopValues]]`discover:showLegacyFieldTopValues`::
To calculate the top values for a field in the sidebar using 500 instead of 5,000 records per shard, turn on this option.
[[discover-sort-defaultorder]]`discover:sort:defaultOrder`::
The default sort direction for time-based data views.

View file

@ -28,4 +28,5 @@ export const TRUNCATE_MAX_HEIGHT = 'truncate:maxHeight';
export const ROW_HEIGHT_OPTION = 'discover:rowHeightOption';
export const SEARCH_EMBEDDABLE_TYPE = 'search';
export const HIDE_ANNOUNCEMENTS = 'hideAnnouncements';
export const SHOW_LEGACY_FIELD_TOP_VALUES = 'discover:showLegacyFieldTopValues';
export const ENABLE_SQL = 'discover:enableSql';

View file

@ -17,7 +17,8 @@
"savedObjectsManagement",
"dataViewFieldEditor",
"dataViewEditor",
"expressions"
"expressions",
"unifiedFieldList"
],
"optionalPlugins": [
"home",

View file

@ -70,6 +70,10 @@ const services = {
getAbsoluteTime: () => {
return { from: '2020-05-14T11:05:13.590', to: '2020-05-14T11:20:13.590' };
},
getTime: () => ({
from: 'now-7d',
to: 'now',
}),
},
},
savedQueries: { findSavedQueries: () => Promise.resolve({ queries: [] as SavedQuery[] }) },

View file

@ -172,6 +172,7 @@ export function getDocumentsLayoutProps(dataView: DataView) {
language: 'kuery',
query: '',
},
filters: [],
},
} as unknown as DiscoverLayoutProps;
}
@ -186,6 +187,7 @@ export const getPlainRecordLayoutProps = (dataView: DataView) => {
query: {
sql: 'SELECT * FROM "kibana_sample_data_ecommerce"',
},
filters: [],
},
} as unknown as DiscoverLayoutProps;
};

View file

@ -88,7 +88,7 @@ storiesOf('components/sidebar/DiscoverFieldDetails', module)
<DiscoverFieldDetails
field={field}
dataView={dataView}
details={{ buckets: [], error: 'An error occurred', exists: 1, total: 2, columns: [] }}
details={{ buckets: [], error: 'An error occurred', exists: 1, total: 2 }}
onAddFilter={() => {}}
/>
));

View file

@ -6,15 +6,43 @@
* Side Public License, v 1.
*/
import { act } from 'react-dom/test-utils';
import { EuiPopover, EuiProgress, EuiButtonIcon } from '@elastic/eui';
import { ReactWrapper } from 'enzyme';
import React from 'react';
import { findTestSubject } from '@elastic/eui/lib/test';
import { mountWithIntl } from '@kbn/test-jest-helpers';
import { DiscoverField } from './discover_field';
import { chartPluginMock } from '@kbn/charts-plugin/public/mocks';
import { dataViewPluginMocks } from '@kbn/data-views-plugin/public/mocks';
import { fieldFormatsServiceMock } from '@kbn/field-formats-plugin/public/mocks';
import { dataPluginMock } from '@kbn/data-plugin/public/mocks';
import { DiscoverField, DiscoverFieldProps } from './discover_field';
import { DataViewField } from '@kbn/data-views-plugin/public';
import { KibanaContextProvider } from '@kbn/kibana-react-plugin/public';
import { stubDataView } from '@kbn/data-views-plugin/common/data_view.stub';
jest.mock('@kbn/unified-field-list-plugin/public/services/field_stats', () => ({
loadFieldStats: jest.fn().mockResolvedValue({
totalDocuments: 1624,
sampledDocuments: 1624,
sampledValues: 3248,
topValues: {
buckets: [
{
count: 2042,
key: 'osx',
},
{
count: 1206,
key: 'winx',
},
],
},
}),
}));
const dataServiceMock = dataPluginMock.createStartContract();
jest.mock('../../../../kibana_services', () => ({
getUiActions: jest.fn(() => {
return {
@ -23,16 +51,18 @@ jest.mock('../../../../kibana_services', () => ({
}),
}));
function getComponent({
async function getComponent({
selected = false,
showDetails = false,
showFieldStats = false,
field,
onAddFilterExists = true,
showLegacyFieldTopValues = false,
}: {
selected?: boolean;
showDetails?: boolean;
showFieldStats?: boolean;
field?: DataViewField;
onAddFilterExists?: boolean;
showLegacyFieldTopValues?: boolean;
}) {
const finalField =
field ??
@ -49,16 +79,21 @@ function getComponent({
const dataView = stubDataView;
dataView.toSpec = () => ({});
const props = {
const props: DiscoverFieldProps = {
dataView: stubDataView,
field: finalField,
getDetails: jest.fn(() => ({ buckets: [], error: '', exists: 1, total: 2, columns: [] })),
getDetails: jest.fn(() => ({ buckets: [], error: '', exists: 1, total: 2 })),
...(onAddFilterExists && { onAddFilter: jest.fn() }),
onAddField: jest.fn(),
onRemoveField: jest.fn(),
showDetails,
showFieldStats,
selected,
persistDataView: jest.fn(),
state: {
query: { query: '', language: 'lucene' },
filters: [],
},
contextualFields: [],
};
const services = {
history: () => ({
@ -76,10 +111,32 @@ function getComponent({
if (key === 'fields:popularLimit') {
return 5;
}
if (key === 'discover:showLegacyFieldTopValues') {
return showLegacyFieldTopValues;
}
},
},
data: {
...dataServiceMock,
query: {
...dataServiceMock.query,
timefilter: {
...dataServiceMock.query.timefilter,
timefilter: {
...dataServiceMock.query.timefilter.timefilter,
getAbsoluteTime: () => ({
from: '2021-08-31T22:00:00.000Z',
to: '2022-09-01T09:16:29.553Z',
}),
},
},
},
},
dataViews: dataViewPluginMocks.createStartContract(),
fieldFormats: fieldFormatsServiceMock.createStartContract(),
charts: chartPluginMock.createSetupContract(),
};
const comp = mountWithIntl(
const comp = await mountWithIntl(
<KibanaContextProvider services={services}>
<DiscoverField {...props} />
</KibanaContextProvider>
@ -88,22 +145,26 @@ function getComponent({
}
describe('discover sidebar field', function () {
it('should allow selecting fields', function () {
const { comp, props } = getComponent({});
it('should allow selecting fields', async function () {
const { comp, props } = await getComponent({});
findTestSubject(comp, 'fieldToggle-bytes').simulate('click');
expect(props.onAddField).toHaveBeenCalledWith('bytes');
});
it('should allow deselecting fields', function () {
const { comp, props } = getComponent({ selected: true });
it('should allow deselecting fields', async function () {
const { comp, props } = await getComponent({ selected: true });
findTestSubject(comp, 'fieldToggle-bytes').simulate('click');
expect(props.onRemoveField).toHaveBeenCalledWith('bytes');
});
it('should trigger getDetails', function () {
const { comp, props } = getComponent({ selected: true });
it('should trigger getDetails', async function () {
const { comp, props } = await getComponent({
selected: true,
showFieldStats: true,
showLegacyFieldTopValues: true,
});
findTestSubject(comp, 'field-bytes-showDetails').simulate('click');
expect(props.getDetails).toHaveBeenCalledWith(props.field);
});
it('should not allow clicking on _source', function () {
it('should not allow clicking on _source', async function () {
const field = new DataViewField({
name: '_source',
type: '_source',
@ -112,14 +173,15 @@ describe('discover sidebar field', function () {
aggregatable: true,
readFromDocValues: true,
});
const { comp, props } = getComponent({
const { comp, props } = await getComponent({
selected: true,
field,
showLegacyFieldTopValues: true,
});
findTestSubject(comp, 'field-_source-showDetails').simulate('click');
expect(props.getDetails).not.toHaveBeenCalled();
});
it('displays warning for conflicting fields', function () {
it('displays warning for conflicting fields', async function () {
const field = new DataViewField({
name: 'troubled_field',
type: 'conflict',
@ -128,23 +190,26 @@ describe('discover sidebar field', function () {
aggregatable: true,
readFromDocValues: false,
});
const { comp } = getComponent({
const { comp } = await getComponent({
selected: true,
field,
});
const dscField = findTestSubject(comp, 'field-troubled_field-showDetails');
expect(dscField.find('.kbnFieldButton__infoIcon').length).toEqual(1);
});
it('should not execute getDetails when rendered, since it can be expensive', function () {
const { props } = getComponent({});
expect(props.getDetails.mock.calls.length).toEqual(0);
it('should not execute getDetails when rendered, since it can be expensive', async function () {
const { props } = await getComponent({});
expect(props.getDetails).toHaveBeenCalledTimes(0);
});
it('should execute getDetails when show details is requested', function () {
const { props, comp } = getComponent({});
it('should execute getDetails when show details is requested', async function () {
const { props, comp } = await getComponent({
showFieldStats: true,
showLegacyFieldTopValues: true,
});
findTestSubject(comp, 'field-bytes-showDetails').simulate('click');
expect(props.getDetails.mock.calls.length).toEqual(1);
expect(props.getDetails).toHaveBeenCalledTimes(1);
});
it('should not return the popover if onAddFilter is not provided', function () {
it('should not return the popover if onAddFilter is not provided', async function () {
const field = new DataViewField({
name: '_source',
type: '_source',
@ -153,7 +218,7 @@ describe('discover sidebar field', function () {
aggregatable: true,
readFromDocValues: true,
});
const { comp } = getComponent({
const { comp } = await getComponent({
selected: true,
field,
onAddFilterExists: false,
@ -161,4 +226,37 @@ describe('discover sidebar field', function () {
const popover = findTestSubject(comp, 'discoverFieldListPanelPopover');
expect(popover.length).toBe(0);
});
it('should request field stats', async function () {
const field = new DataViewField({
name: 'machine.os.raw',
type: 'string',
esTypes: ['keyword'],
aggregatable: true,
searchable: true,
});
let comp: ReactWrapper;
await act(async () => {
const result = await getComponent({ showFieldStats: true, field, onAddFilterExists: true });
comp = result.comp;
await comp.update();
});
await act(async () => {
const fieldItem = findTestSubject(comp, 'field-machine.os.raw-showDetails');
await fieldItem.simulate('click');
await comp.update();
});
await comp!.update();
expect(comp!.find(EuiPopover).prop('isOpen')).toBe(true);
expect(findTestSubject(comp!, 'dscFieldStats-title').text()).toBe('Top values');
expect(findTestSubject(comp!, 'dscFieldStats-topValues-bucket')).toHaveLength(2);
expect(
findTestSubject(comp!, 'dscFieldStats-topValues-formattedFieldValue').first().text()
).toBe('osx');
expect(comp!.find(EuiProgress)).toHaveLength(2);
expect(findTestSubject(comp!, 'dscFieldStats-topValues').find(EuiButtonIcon)).toHaveLength(4);
});
});

View file

@ -19,19 +19,22 @@ import {
EuiFlexGroup,
EuiFlexItem,
EuiSpacer,
EuiHorizontalRule,
} from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import { UiCounterMetricType } from '@kbn/analytics';
import classNames from 'classnames';
import { FieldButton, FieldIcon } from '@kbn/react-field';
import type { DataViewField, DataView } from '@kbn/data-views-plugin/public';
import { FieldStats } from '@kbn/unified-field-list-plugin/public';
import { getFieldCapabilities } from '../../../../utils/get_field_capabilities';
import { getTypeForFieldIcon } from '../../../../utils/get_type_for_field_icon';
import { DiscoverFieldDetails } from './discover_field_details';
import { FieldDetails } from './types';
import { getFieldTypeName } from '../../../../utils/get_field_type_name';
import { DiscoverFieldVisualize } from './discover_field_visualize';
import type { AppState } from '../../services/discover_state';
import { useDiscoverServices } from '../../../../hooks/use_discover_services';
import { SHOW_LEGACY_FIELD_TOP_VALUES } from '../../../../../common';
function wrapOnDot(str?: string) {
// u200B is a non-width white-space character, which allows
@ -224,7 +227,7 @@ export interface DiscoverFieldProps {
/**
* Callback to add a filter to filter bar
*/
onAddFilter?: (field: DataViewField | string, value: string, type: '+' | '-') => void;
onAddFilter?: (field: DataViewField | string, value: unknown, type: '+' | '-') => void;
/**
* Callback to remove/deselect a the field
* @param fieldName
@ -264,6 +267,16 @@ export interface DiscoverFieldProps {
*/
showFieldStats?: boolean;
persistDataView: (dataView: DataView) => Promise<DataView | undefined>;
/**
* Discover App State
*/
state: AppState;
/**
* Columns
*/
contextualFields: string[];
}
function DiscoverFieldComponent({
@ -281,10 +294,25 @@ function DiscoverFieldComponent({
onDeleteField,
showFieldStats,
persistDataView,
state,
contextualFields,
}: DiscoverFieldProps) {
const services = useDiscoverServices();
const { data } = services;
const [infoIsOpen, setOpen] = useState(false);
const isDocumentRecord = !!onAddFilter;
const addFilterAndClosePopover: typeof onAddFilter | undefined = useMemo(
() =>
onAddFilter
? (...params) => {
setOpen(false);
onAddFilter?.(...params);
}
: undefined,
[setOpen, onAddFilter]
);
const toggleDisplay = useCallback(
(f: DataViewField) => {
if (selected) {
@ -325,36 +353,66 @@ function DiscoverFieldComponent({
const { canEdit, canDelete } = getFieldCapabilities(dataView, field);
const canEditField = onEditField && canEdit;
const canDeleteField = onDeleteField && canDelete;
const addExistFilterTooltip = i18n.translate(
'discover.fieldChooser.discoverField.addExistFieldLabel',
{
defaultMessage: 'Filter for field present',
}
);
const editFieldTooltip = i18n.translate('discover.fieldChooser.discoverField.editFieldLabel', {
defaultMessage: 'Edit data view field',
});
const deleteFieldTooltip = i18n.translate(
'discover.fieldChooser.discoverField.deleteFieldLabel',
{
defaultMessage: 'Delete data view field',
}
);
const popoverTitle = (
<EuiPopoverTitle style={{ textTransform: 'none' }} className="eui-textBreakWord">
<EuiFlexGroup responsive={false} gutterSize="s">
<EuiFlexItem grow={true}>
<h5>{field.displayName}</h5>
</EuiFlexItem>
{onAddFilter && !dataView.metaFields.includes(field.name) && !field.scripted && (
<EuiFlexItem grow={false} data-test-subj="discoverFieldListPanelAddExistFilterItem">
<EuiToolTip content={addExistFilterTooltip}>
<EuiButtonIcon
onClick={() => {
setOpen(false);
onAddFilter('_exists_', field.name, '+');
}}
iconType="filter"
data-test-subj={`discoverFieldListPanelAddExistFilter-${field.name}`}
aria-label={addExistFilterTooltip}
/>
</EuiToolTip>
</EuiFlexItem>
)}
{canEditField && (
<EuiFlexItem grow={false} data-test-subj="discoverFieldListPanelEditItem">
<EuiButtonIcon
onClick={() => {
if (onEditField) {
togglePopover();
onEditField(field.name);
}
}}
iconType="pencil"
data-test-subj={`discoverFieldListPanelEdit-${field.name}`}
aria-label={i18n.translate('discover.fieldChooser.discoverField.editFieldLabel', {
defaultMessage: 'Edit data view field',
})}
/>
<EuiToolTip content={editFieldTooltip}>
<EuiButtonIcon
onClick={() => {
if (onEditField) {
togglePopover();
onEditField(field.name);
}
}}
iconType="pencil"
data-test-subj={`discoverFieldListPanelEdit-${field.name}`}
aria-label={editFieldTooltip}
/>
</EuiToolTip>
</EuiFlexItem>
)}
{canDeleteField && (
<EuiFlexItem grow={false} data-test-subj="discoverFieldListPanelDeleteItem">
<EuiToolTip
content={i18n.translate('discover.fieldChooser.discoverField.deleteFieldLabel', {
defaultMessage: 'Delete data view field',
})}
>
<EuiToolTip content={deleteFieldTooltip}>
<EuiButtonIcon
onClick={() => {
onDeleteField?.(field.name);
@ -362,9 +420,7 @@ function DiscoverFieldComponent({
iconType="trash"
data-test-subj={`discoverFieldListPanelDelete-${field.name}`}
color="danger"
aria-label={i18n.translate('discover.fieldChooser.discoverField.deleteFieldLabel', {
defaultMessage: 'Delete data view field',
})}
aria-label={deleteFieldTooltip}
/>
</EuiToolTip>
</EuiFlexItem>
@ -398,33 +454,57 @@ function DiscoverFieldComponent({
}
const renderPopover = () => {
const details = getDetails(field);
// TODO: integrate <FieldStats .../>
const dateRange = data?.query?.timefilter.timefilter.getAbsoluteTime();
// prioritize an aggregatable multi field if available or take the parent field
const fieldForStats =
(multiFields?.length &&
multiFields.find((multiField) => multiField.field.aggregatable)?.field) ||
field;
const showLegacyFieldStats = services.uiSettings.get(SHOW_LEGACY_FIELD_TOP_VALUES);
return (
<>
{showFieldStats && (
{showLegacyFieldStats ? (
<>
<EuiTitle size="xxxs">
<h5>
{i18n.translate('discover.fieldChooser.discoverField.fieldTopValuesLabel', {
defaultMessage: 'Top 5 values',
})}
</h5>
</EuiTitle>
<DiscoverFieldDetails
dataView={dataView}
field={field}
details={details}
onAddFilter={onAddFilter}
/>
{showFieldStats && (
<>
<EuiTitle size="xxxs">
<h5>
{i18n.translate('discover.fieldChooser.discoverField.fieldTopValuesLabel', {
defaultMessage: 'Top 5 values',
})}
</h5>
</EuiTitle>
<DiscoverFieldDetails
dataView={dataView}
field={field}
details={getDetails(field)}
onAddFilter={onAddFilter}
/>
</>
)}
</>
) : (
<>
{Boolean(dateRange) && (
<FieldStats
services={services}
query={state.query!}
filters={state.filters!}
fromDate={dateRange.from}
toDate={dateRange.to}
dataViewOrDataViewId={dataView}
field={fieldForStats}
data-test-subj="dscFieldStats"
onAddFilter={addFilterAndClosePopover}
/>
)}
</>
)}
{multiFields && (
<>
{showFieldStats && <EuiSpacer size="m" />}
{(showFieldStats || !showLegacyFieldStats) && <EuiSpacer size="m" />}
<MultiFields
multiFields={multiFields}
alwaysShowActionButton={alwaysShowActionButton}
@ -433,13 +513,13 @@ function DiscoverFieldComponent({
/>
</>
)}
{(showFieldStats || multiFields) && <EuiHorizontalRule margin="m" />}
<DiscoverFieldVisualize
field={field}
dataView={dataView}
multiFields={rawMultiFields}
trackUiMetric={trackUiMetric}
details={details}
contextualFields={contextualFields}
persistDataView={persistDataView}
/>
</>

View file

@ -9,33 +9,33 @@
import React, { useEffect, useState } from 'react';
import { METRIC_TYPE, UiCounterMetricType } from '@kbn/analytics';
import type { DataView, DataViewField } from '@kbn/data-views-plugin/public';
import { VISUALIZE_GEO_FIELD_TRIGGER } from '@kbn/ui-actions-plugin/public';
import {
getTriggerConstant,
triggerVisualizeActions,
VisualizeInformation,
} from './lib/visualize_trigger_utils';
import type { FieldDetails } from './types';
import { getVisualizeInformation } from './lib/visualize_trigger_utils';
import { DiscoverFieldVisualizeInner } from './discover_field_visualize_inner';
interface Props {
field: DataViewField;
dataView: DataView;
details: FieldDetails;
multiFields?: DataViewField[];
contextualFields: string[];
trackUiMetric?: (metricType: UiCounterMetricType, eventName: string | string[]) => void;
persistDataView: (dataView: DataView) => Promise<DataView | undefined>;
}
export const DiscoverFieldVisualize: React.FC<Props> = React.memo(
({ field, dataView, details, trackUiMetric, multiFields, persistDataView }) => {
({ field, dataView, contextualFields, trackUiMetric, multiFields, persistDataView }) => {
const [visualizeInfo, setVisualizeInfo] = useState<VisualizeInformation>();
useEffect(() => {
getVisualizeInformation(field, dataView, details.columns, multiFields).then(setVisualizeInfo);
}, [details.columns, field, dataView, multiFields]);
getVisualizeInformation(field, dataView, contextualFields, multiFields).then(
setVisualizeInfo
);
}, [contextualFields, field, dataView, multiFields]);
if (!visualizeInfo) {
return null;
@ -50,7 +50,7 @@ export const DiscoverFieldVisualize: React.FC<Props> = React.memo(
const trigger = getTriggerConstant(field.type);
const triggerVisualization = (updatedDataView: DataView) => {
trackUiMetric?.(METRIC_TYPE.CLICK, 'visualize_link_click');
triggerVisualizeActions(visualizeInfo.field, details.columns, updatedDataView);
triggerVisualizeActions(visualizeInfo.field, contextualFields, updatedDataView);
};
if (trigger === VISUALIZE_GEO_FIELD_TRIGGER) {

View file

@ -7,7 +7,7 @@
*/
import React from 'react';
import { EuiButton } from '@elastic/eui';
import { EuiButton, EuiPopoverFooter } from '@elastic/eui';
import { FormattedMessage } from '@kbn/i18n-react';
import type { DataViewField } from '@kbn/data-views-plugin/public';
import { VisualizeInformation } from './lib/visualize_trigger_utils';
@ -20,19 +20,22 @@ interface DiscoverFieldVisualizeInnerProps {
export const DiscoverFieldVisualizeInner = (props: DiscoverFieldVisualizeInnerProps) => {
const { field, visualizeInfo, handleVisualizeLinkClick } = props;
return (
// eslint-disable-next-line @elastic/eui/href-or-on-click
<EuiButton
fullWidth
size="s"
href={visualizeInfo.href}
onClick={handleVisualizeLinkClick}
data-test-subj={`fieldVisualize-${field.name}`}
>
<FormattedMessage
id="discover.fieldChooser.visualizeButton.label"
defaultMessage="Visualize"
/>
</EuiButton>
<EuiPopoverFooter>
{/* eslint-disable-next-line @elastic/eui/href-or-on-click */}
<EuiButton
fullWidth
size="s"
href={visualizeInfo.href}
onClick={handleVisualizeLinkClick}
data-test-subj={`fieldVisualize-${field.name}`}
>
<FormattedMessage
id="discover.fieldChooser.visualizeButton.label"
defaultMessage="Visualize"
/>
</EuiButton>
</EuiPopoverFooter>
);
};

View file

@ -117,6 +117,7 @@ export function DiscoverSidebarComponent({
viewMode,
createNewDataView,
showDataViewPicker,
state,
persistDataView,
}: DiscoverSidebarProps) {
const { uiSettings, dataViewFieldEditor } = useDiscoverServices();
@ -146,8 +147,8 @@ export function DiscoverSidebarComponent({
);
const getDetailsByField = useCallback(
(ipField: DataViewField) => getDetails(ipField, documents, columns, selectedDataView),
[documents, columns, selectedDataView]
(ipField: DataViewField) => getDetails(ipField, documents, selectedDataView),
[documents, selectedDataView]
);
const popularLimit = useMemo(() => uiSettings.get(FIELDS_LIMIT_SETTING), [uiSettings]);
@ -415,6 +416,8 @@ export function DiscoverSidebarComponent({
onDeleteField={deleteField}
showFieldStats={showFieldStats}
persistDataView={persistDataView}
state={state}
contextualFields={columns}
/>
</li>
);
@ -476,6 +479,8 @@ export function DiscoverSidebarComponent({
onDeleteField={deleteField}
showFieldStats={showFieldStats}
persistDataView={persistDataView}
state={state}
contextualFields={columns}
/>
</li>
);
@ -506,6 +511,8 @@ export function DiscoverSidebarComponent({
onDeleteField={deleteField}
showFieldStats={showFieldStats}
persistDataView={persistDataView}
state={state}
contextualFields={columns}
/>
</li>
);

View file

@ -24,6 +24,48 @@ import { AvailableFields$, DataDocuments$, RecordRawType } from '../../hooks/use
import { stubLogstashDataView } from '@kbn/data-plugin/common/stubs';
import { VIEW_MODE } from '../../../../components/view_mode_toggle';
import { KibanaContextProvider } from '@kbn/kibana-react-plugin/public';
import { chartPluginMock } from '@kbn/charts-plugin/public/mocks';
import { dataViewPluginMocks } from '@kbn/data-views-plugin/public/mocks';
import { fieldFormatsServiceMock } from '@kbn/field-formats-plugin/public/mocks';
import { dataPluginMock } from '@kbn/data-plugin/public/mocks';
jest.mock('@kbn/unified-field-list-plugin/public/services/field_stats', () => ({
loadFieldStats: jest.fn().mockResolvedValue({
totalDocuments: 1624,
sampledDocuments: 1624,
sampledValues: 3248,
topValues: {
buckets: [
{
count: 1349,
key: 'gif',
},
{
count: 1206,
key: 'zip',
},
{
count: 329,
key: 'css',
},
{
count: 164,
key: 'js',
},
{
count: 111,
key: 'png',
},
{
count: 89,
key: 'jpg',
},
],
},
}),
}));
const dataServiceMock = dataPluginMock.createStartContract();
const mockServices = {
history: () => ({
@ -52,6 +94,25 @@ const mockServices = {
editDataView: jest.fn(() => true),
},
},
data: {
...dataServiceMock,
query: {
...dataServiceMock.query,
timefilter: {
...dataServiceMock.query.timefilter,
timefilter: {
...dataServiceMock.query.timefilter.timefilter,
getAbsoluteTime: () => ({
from: '2021-08-31T22:00:00.000Z',
to: '2022-09-01T09:16:29.553Z',
}),
},
},
},
},
dataViews: dataViewPluginMocks.createStartContract(),
fieldFormats: fieldFormatsServiceMock.createStartContract(),
charts: chartPluginMock.createSetupContract(),
} as unknown as DiscoverServices;
const mockfieldCounts: Record<string, number> = {};
@ -105,7 +166,10 @@ function getCompProps(): DiscoverSidebarResponsiveProps {
onAddField: jest.fn(),
onRemoveField: jest.fn(),
selectedDataView: dataView,
state: {},
state: {
query: { query: '', language: 'lucene' },
filters: [],
},
trackUiMetric: jest.fn(),
onFieldEdited: jest.fn(),
viewMode: VIEW_MODE.DOCUMENT_LEVEL,
@ -119,13 +183,16 @@ describe('discover responsive sidebar', function () {
let props: DiscoverSidebarResponsiveProps;
let comp: ReactWrapper<DiscoverSidebarResponsiveProps>;
beforeAll(() => {
beforeAll(async () => {
props = getCompProps();
comp = mountWithIntl(
<KibanaContextProvider services={mockServices}>
<DiscoverSidebarResponsive {...props} />
</KibanaContextProvider>
);
await act(async () => {
comp = await mountWithIntl(
<KibanaContextProvider services={mockServices}>
<DiscoverSidebarResponsive {...props} />
</KibanaContextProvider>
);
comp.update();
});
});
it('should have Selected Fields and Available Fields with Popular Fields sections', function () {
@ -145,11 +212,28 @@ describe('discover responsive sidebar', function () {
findTestSubject(comp, 'fieldToggle-extension').simulate('click');
expect(props.onRemoveField).toHaveBeenCalledWith('extension');
});
it('should allow adding filters', function () {
findTestSubject(comp, 'field-extension-showDetails').simulate('click');
it('should allow adding filters', async function () {
await act(async () => {
const button = findTestSubject(comp, 'field-extension-showDetails');
await button.simulate('click');
await comp.update();
});
await comp.update();
findTestSubject(comp, 'plus-extension-gif').simulate('click');
expect(props.onAddFilter).toHaveBeenCalled();
});
it('should allow adding "exist" filter', async function () {
await act(async () => {
const button = findTestSubject(comp, 'field-extension-showDetails');
await button.simulate('click');
await comp.update();
});
await comp.update();
findTestSubject(comp, 'discoverFieldListPanelAddExistFilter-extension').simulate('click');
expect(props.onAddFilter).toHaveBeenCalledWith('_exists_', 'extension', '+');
});
it('should allow filtering by string, and calcFieldCount should just be executed once', function () {
expect(findTestSubject(comp, 'fieldList-unpopular').children().length).toBe(6);
act(() => {

View file

@ -62,7 +62,7 @@ export interface DiscoverSidebarResponsiveProps {
/**
* Callback function when adding a filter from sidebar
*/
onAddFilter?: (field: DataViewField | string, value: string, type: '+' | '-') => void;
onAddFilter?: (field: DataViewField | string, value: unknown, type: '+' | '-') => void;
/**
* Callback function when changing an data view
*/

View file

@ -14,7 +14,6 @@ import { DataTableRecord } from '../../../../../types';
export function getDetails(
field: DataViewField,
hits: DataTableRecord[] | undefined,
columns: string[],
dataView?: DataView
) {
if (!dataView || !hits) {
@ -27,7 +26,6 @@ export function getDetails(
count: 5,
grouped: false,
}),
columns,
};
if (details.buckets) {
for (const bucket of details.buckets) {

View file

@ -17,7 +17,6 @@ export interface FieldDetails {
exists: number;
total: number;
buckets: Bucket[];
columns: string[];
}
export interface Bucket {

View file

@ -85,6 +85,7 @@ export interface DiscoverServices {
triggersActionsUi: TriggersAndActionsUIPublicPluginStart;
locator: DiscoverAppLocator;
expressions: ExpressionsStart;
charts: ChartsPluginStart;
savedObjectsManagement: SavedObjectsManagementPluginStart;
savedObjectsTagging?: SavedObjectsTaggingApi;
}
@ -132,6 +133,7 @@ export const buildServices = memoize(function (
triggersActionsUi: plugins.triggersActionsUi,
locator,
expressions: plugins.expressions,
charts: plugins.charts,
savedObjectsTagging: plugins.savedObjectsTaggingOss?.getTaggingApi(),
savedObjectsManagement: plugins.savedObjectsManagement,
};

View file

@ -30,6 +30,7 @@ import {
TRUNCATE_MAX_HEIGHT,
SHOW_FIELD_STATISTICS,
ROW_HEIGHT_OPTION,
SHOW_LEGACY_FIELD_TOP_VALUES,
ENABLE_SQL,
} from '../common';
import { DEFAULT_ROWS_PER_PAGE, ROWS_PER_PAGE_OPTIONS } from '../common/constants';
@ -124,6 +125,19 @@ export const getUiSettings: (docLinks: DocLinksServiceSetup) => Record<string, U
category: ['discover'],
schema: schema.boolean(),
},
[SHOW_LEGACY_FIELD_TOP_VALUES]: {
name: i18n.translate('discover.advancedSettings.showLegacyFieldStatsTitle', {
defaultMessage: 'Top values calculation',
}),
value: false,
type: 'boolean',
description: i18n.translate('discover.advancedSettings.showLegacyFieldStatsText', {
defaultMessage:
'To calculate the top values for a field in the sidebar using 500 instead of 5,000 records per shard, turn on this option.',
}),
category: ['discover'],
schema: schema.boolean(),
},
[DOC_HIDE_TIME_COLUMN_SETTING]: {
name: i18n.translate('discover.advancedSettings.docTableHideTimeColumnTitle', {
defaultMessage: "Hide 'Time' column",

View file

@ -29,6 +29,7 @@
{ "path": "../field_formats/tsconfig.json" },
{ "path": "../data_views/tsconfig.json" },
{ "path": "../unified_search/tsconfig.json" },
{ "path": "../unified_field_list/tsconfig.json" },
{ "path": "../../../x-pack/plugins/spaces/tsconfig.json" },
{ "path": "../data_view_editor/tsconfig.json" },
{ "path": "../../../x-pack/plugins/triggers_actions_ui/tsconfig.json" },

View file

@ -167,6 +167,10 @@ export const stackManagementSchema: MakeSchemaFrom<UsageStats> = {
type: 'boolean',
_meta: { description: 'Non-default value of setting.' },
},
'discover:showLegacyFieldTopValues': {
type: 'boolean',
_meta: { description: 'Non-default value of setting.' },
},
'discover:sampleSize': {
type: 'long',
_meta: { description: 'Non-default value of setting.' },

View file

@ -75,6 +75,7 @@ export interface UsageStats {
'doc_table:hideTimeColumn': boolean;
'discover:sampleSize': number;
'discover:sampleRowsPerPage': number;
'discover:showLegacyFieldTopValues': boolean;
defaultColumns: string[];
'context:defaultSize': number;
'context:tieBreakerFields': string[];

View file

@ -7960,6 +7960,12 @@
"description": "Non-default value of setting."
}
},
"discover:showLegacyFieldTopValues": {
"type": "boolean",
"_meta": {
"description": "Non-default value of setting."
}
},
"discover:sampleSize": {
"type": "long",
"_meta": {

View file

@ -0,0 +1,247 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import { keys, clone, uniq, filter, map, flatten } from 'lodash';
import type { DataView } from '@kbn/data-views-plugin/public';
import { stubLogstashDataView as dataView } from '@kbn/data-views-plugin/common/data_view.stub';
import { getFieldExampleBuckets, groupValues, getFieldValues } from './field_examples_calculator';
const hitsAsValues: Array<Record<string, string | number | string[]>> = [
{
extension: 'html',
bytes: 360.20000000000005,
'@tags': ['success', 'info'],
},
{
extension: 'gif',
bytes: 5848.700000000001,
'@tags': ['error'],
},
{
extension: 'png',
bytes: 841.6,
},
{
extension: 'html',
bytes: 1626.4,
},
{
extension: 'php',
bytes: 2070.6,
phpmemory: 276080,
},
{
extension: 'gif',
bytes: 8421.6,
},
{
extension: 'html',
bytes: 994.8000000000001,
},
{
extension: 'html',
bytes: 374,
},
{
extension: 'php',
bytes: 506.09999999999997,
phpmemory: 67480,
},
{
extension: 'php',
bytes: 506.09999999999997,
phpmemory: 67480,
},
{
extension: 'php',
bytes: 2591.1,
phpmemory: 345480,
},
{
extension: 'html',
bytes: 1450,
},
{
extension: 'php',
bytes: 1803.8999999999999,
phpmemory: 240520,
},
{
extension: 'html',
bytes: 1626.4,
},
{
extension: 'gif',
bytes: 10617.2,
},
{
extension: 'gif',
bytes: 10961.5,
},
{
extension: 'html',
bytes: 382.8,
},
{
extension: 'html',
bytes: 374,
},
{
extension: 'png',
bytes: 3059.2000000000003,
},
{
extension: 'gif',
bytes: 10617.2,
},
];
const hits = hitsAsValues.map((value) => ({
_index: 'logstash-2014.09.09',
_id: '1945',
_score: 1,
fields: Object.keys(value).reduce(
(result: Record<string, Array<string | number>>, fieldName: string) => {
const fieldValue = value[fieldName];
result[fieldName] = Array.isArray(fieldValue) ? fieldValue : [fieldValue];
return result;
},
{}
),
}));
describe('fieldExamplesCalculator', function () {
describe('groupValues', function () {
let grouped: { groups: Record<string, any>; sampledValues: number };
let values: any;
beforeEach(function () {
values = [
['foo', 'bar'],
'foo',
'foo',
undefined,
['foo', 'bar', 'bar'],
'bar',
'baz',
null,
null,
null,
'foo',
undefined,
];
grouped = groupValues(values);
});
it('should have a groupValues that counts values', function () {
expect(grouped.groups).toBeInstanceOf(Object);
expect(grouped.sampledValues).toBe(9);
});
it('should throw an error if any value is a plain object', function () {
expect(function () {
groupValues([{}, true, false]);
}).toThrowError();
});
it('should handle values with dots in them', function () {
values = ['0', '0.........', '0.......,.....'];
grouped = groupValues(values);
expect(grouped.groups[values[0]].count).toBe(1);
expect(grouped.groups[values[1]].count).toBe(1);
expect(grouped.groups[values[2]].count).toBe(1);
expect(grouped.sampledValues).toBe(3);
});
it('should have a a key for value in the array when not grouping array terms', function () {
expect(keys(grouped.groups).length).toBe(3);
expect(grouped.groups.foo).toBeInstanceOf(Object);
expect(grouped.groups.bar).toBeInstanceOf(Object);
expect(grouped.groups.baz).toBeInstanceOf(Object);
});
it('should count array terms independently', function () {
expect(grouped.groups['foo,bar']).toBe(undefined);
expect(grouped.groups.foo.count).toBe(5);
expect(grouped.groups.bar.count).toBe(3);
expect(grouped.groups.baz.count).toBe(1);
expect(grouped.sampledValues).toBe(9);
});
});
describe('getFieldValues', function () {
it('Should return an array of values for _source fields', function () {
const values = getFieldValues(hits, dataView.fields.getByName('extension')!, dataView);
expect(values).toBeInstanceOf(Array);
expect(
filter(values, function (v) {
return v.includes('html');
}).length
).toBe(8);
expect(uniq(flatten(clone(values))).sort()).toEqual(['gif', 'html', 'php', 'png']);
});
it('Should return an array of values for core meta fields', function () {
const types = getFieldValues(hits, dataView.fields.getByName('_id')!, dataView);
expect(types).toBeInstanceOf(Array);
expect(types.length).toBe(20);
});
});
describe('getFieldExampleBuckets', function () {
let params: { hits: any; field: any; count: number; dataView: DataView };
beforeEach(function () {
params = {
hits,
field: dataView.fields.getByName('extension'),
count: 3,
dataView,
};
});
it('counts the top 3 values', function () {
const result = getFieldExampleBuckets(params);
expect(result).toBeInstanceOf(Object);
expect(result.buckets).toBeInstanceOf(Array);
expect(result.buckets.length).toBe(3);
expect(map(result.buckets, 'key')).toEqual(['html', 'php', 'gif']);
});
it('fails to analyze geo and attachment types', function () {
params.field = dataView.fields.getByName('point');
expect(() => getFieldExampleBuckets(params)).toThrowError();
params.field = dataView.fields.getByName('area');
expect(() => getFieldExampleBuckets(params)).toThrowError();
params.field = dataView.fields.getByName('request_body');
expect(() => getFieldExampleBuckets(params)).toThrowError();
params.field = dataView.fields.getByName('_score');
expect(() => getFieldExampleBuckets(params)).toThrowError();
});
it('fails to analyze fields that are in the mapping, but not the hits', function () {
params.field = dataView.fields.getByName('machine.os');
expect(getFieldExampleBuckets(params).buckets).toHaveLength(0);
expect(getFieldExampleBuckets(params).sampledValues).toBe(0);
});
it('counts the total hits', function () {
expect(getFieldExampleBuckets(params).sampledDocuments).toBe(params.hits.length);
});
it('counts total number of values', function () {
params.field = dataView.fields.getByName('@tags');
expect(getFieldExampleBuckets(params).sampledValues).toBe(3);
params.field = dataView.fields.getByName('extension');
expect(getFieldExampleBuckets(params).sampledValues).toBe(params.hits.length);
params.field = dataView.fields.getByName('phpmemory');
expect(getFieldExampleBuckets(params).sampledValues).toBe(5);
});
});
});

View file

@ -0,0 +1,113 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
// Adapted from src/plugins/discover/public/application/main/components/sidebar/lib/field_calculator.js
import { map, sortBy, defaults, isObject, pick } from 'lodash';
import * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import type { DataView, DataViewField } from '@kbn/data-plugin/common';
import { flattenHit } from '@kbn/data-plugin/common';
type FieldHitValue = any;
interface FieldValueCountsParams {
hits: estypes.SearchHit[];
dataView: DataView;
field: DataViewField;
count?: number;
}
export const canProvideExamplesForField = (field: DataViewField): boolean => {
if (field.name === '_score') {
return false;
}
return ['string', 'text', 'keyword', 'version', 'ip', 'number'].includes(field.type);
};
export function getFieldExampleBuckets(params: FieldValueCountsParams) {
params = defaults(params, {
count: 5,
});
if (!canProvideExamplesForField(params.field)) {
throw new Error(
`Analysis is not available this field type: "${params.field.type}". Field name: "${params.field.name}"`
);
}
const records = getFieldValues(params.hits, params.field, params.dataView);
const { groups, sampledValues } = groupValues(records);
const buckets = sortBy(groups, ['count', 'order'])
.reverse()
.slice(0, params.count)
.map((bucket) => pick(bucket, ['key', 'count']));
return {
buckets,
sampledValues,
sampledDocuments: params.hits.length,
};
}
export function getFieldValues(
hits: estypes.SearchHit[],
field: DataViewField,
dataView: DataView
): FieldHitValue[] {
return map(hits, function (hit) {
return flattenHit(hit, dataView, { includeIgnoredValues: true })[field.name];
});
}
export function groupValues(records: FieldHitValue[]): {
groups: Record<string, { count: number; key: any; order: number }>;
sampledValues: number;
} {
const groups: Record<string, { count: number; key: any; order: number }> = {};
let sampledValues = 0; // counts in each value's occurrence but only once per a record
records.forEach(function (recordValues) {
if (isObject(recordValues) && !Array.isArray(recordValues)) {
throw new Error('Analysis is not available for object fields.');
}
let order = 0; // will be used for ordering terms with the same 'count'
let values: any[];
const visitedValuesMap: Record<string, boolean> = {};
if (Array.isArray(recordValues)) {
values = recordValues;
} else {
values = recordValues == null ? [] : [recordValues];
}
values.forEach((value) => {
if (visitedValuesMap[value]) {
// already counted in groups
return;
}
if (groups.hasOwnProperty(value)) {
groups[value].count++;
} else {
groups[value] = {
key: value,
count: 1,
order: order++,
};
}
visitedValuesMap[value] = true;
sampledValues++;
});
});
return {
groups,
sampledValues,
};
}

View file

@ -8,16 +8,24 @@
import type * as estypes from '@elastic/elasticsearch/lib/api/typesWithBodyKey';
import DateMath from '@kbn/datemath';
import { ESSearchResponse } from '@kbn/core/types/elasticsearch';
import type { DataViewFieldBase } from '@kbn/es-query';
import type { DataView, DataViewField } from '@kbn/data-views-plugin/common';
import type { ESSearchResponse } from '@kbn/core/types/elasticsearch';
import type { FieldStatsResponse } from '../types';
import { getFieldExampleBuckets, canProvideExamplesForField } from './field_examples_calculator';
export type SearchHandler = (
aggs: Record<string, estypes.AggregationsAggregationContainer>
) => Promise<estypes.SearchResponse<unknown>>;
export type SearchHandler = ({
aggs,
fields,
size,
}: {
aggs?: Record<string, estypes.AggregationsAggregationContainer>;
fields?: object[];
size?: number;
}) => Promise<estypes.SearchResponse<unknown>>;
const SHARD_SIZE = 5000;
const DEFAULT_TOP_VALUES_SIZE = 10;
const SIMPLE_EXAMPLES_SIZE = 100;
export function buildSearchParams({
dataViewPattern,
@ -27,6 +35,8 @@ export function buildSearchParams({
dslQuery,
runtimeMappings,
aggs,
fields,
size,
}: {
dataViewPattern: string;
timeFieldName?: string;
@ -34,7 +44,9 @@ export function buildSearchParams({
toDate: string;
dslQuery: object;
runtimeMappings: estypes.MappingRuntimeFields;
aggs: Record<string, estypes.AggregationsAggregationContainer>;
aggs?: Record<string, estypes.AggregationsAggregationContainer>; // is used for aggregatable fields
fields?: object[]; // is used for non-aggregatable fields
size?: number; // is used for non-aggregatable fields
}) {
const filter = timeFieldName
? [
@ -50,6 +62,12 @@ export function buildSearchParams({
]
: [dslQuery];
if (fields?.length === 1) {
filter.push({
exists: fields[0],
});
}
const query = {
bool: {
filter,
@ -61,27 +79,37 @@ export function buildSearchParams({
body: {
query,
aggs,
fields,
runtime_mappings: runtimeMappings,
_source: fields?.length ? false : undefined,
},
track_total_hits: true,
size: 0,
size: size ?? 0,
};
}
export async function fetchAndCalculateFieldStats({
searchHandler,
dataView,
field,
fromDate,
toDate,
size,
}: {
searchHandler: SearchHandler;
field: DataViewFieldBase;
dataView: DataView;
field: DataViewField;
fromDate: string;
toDate: string;
size?: number;
}) {
if (!canProvideStatsForField(field)) {
if (!field.aggregatable) {
return canProvideExamplesForField(field)
? await getSimpleExamples(searchHandler, field, dataView)
: {};
}
if (!canProvideAggregatedStatsForField(field)) {
return {};
}
@ -100,18 +128,27 @@ export async function fetchAndCalculateFieldStats({
return await getStringSamples(searchHandler, field, size);
}
export function canProvideStatsForField(field: DataViewFieldBase): boolean {
function canProvideAggregatedStatsForField(field: DataViewField): boolean {
return !(
field.type === 'document' ||
field.type.includes('range') ||
field.type === 'geo_point' ||
field.type === 'geo_shape'
field.type === 'geo_shape' ||
field.type === 'murmur3' ||
field.type === 'attachment'
);
}
export function canProvideStatsForField(field: DataViewField): boolean {
return (
(field.aggregatable && canProvideAggregatedStatsForField(field)) ||
(!field.aggregatable && canProvideExamplesForField(field))
);
}
export async function getNumberHistogram(
aggSearchWithBody: SearchHandler,
field: DataViewFieldBase,
field: DataViewField,
useTopHits = true
): Promise<FieldStatsResponse<string | number>> {
const fieldRef = getFieldRef(field);
@ -143,9 +180,9 @@ export async function getNumberHistogram(
},
};
const minMaxResult = (await aggSearchWithBody(
useTopHits ? searchWithHits : searchWithoutHits
)) as
const minMaxResult = (await aggSearchWithBody({
aggs: useTopHits ? searchWithHits : searchWithoutHits,
})) as
| ESSearchResponse<unknown, { body: { aggs: typeof searchWithHits } }>
| ESSearchResponse<unknown, { body: { aggs: typeof searchWithoutHits } }>;
@ -199,7 +236,7 @@ export async function getNumberHistogram(
},
},
};
const histogramResult = (await aggSearchWithBody(histogramBody)) as ESSearchResponse<
const histogramResult = (await aggSearchWithBody({ aggs: histogramBody })) as ESSearchResponse<
unknown,
{ body: { aggs: typeof histogramBody } }
>;
@ -220,7 +257,7 @@ export async function getNumberHistogram(
export async function getStringSamples(
aggSearchWithBody: SearchHandler,
field: DataViewFieldBase,
field: DataViewField,
size = DEFAULT_TOP_VALUES_SIZE
): Promise<FieldStatsResponse<string | number>> {
const fieldRef = getFieldRef(field);
@ -242,7 +279,7 @@ export async function getStringSamples(
},
},
};
const topValuesResult = (await aggSearchWithBody(topValuesBody)) as ESSearchResponse<
const topValuesResult = (await aggSearchWithBody({ aggs: topValuesBody })) as ESSearchResponse<
unknown,
{ body: { aggs: typeof topValuesBody } }
>;
@ -263,7 +300,7 @@ export async function getStringSamples(
// This one is not sampled so that it returns the full date range
export async function getDateHistogram(
aggSearchWithBody: SearchHandler,
field: DataViewFieldBase,
field: DataViewField,
range: { fromDate: string; toDate: string }
): Promise<FieldStatsResponse<string | number>> {
const fromDate = DateMath.parse(range.fromDate);
@ -289,7 +326,7 @@ export async function getDateHistogram(
const histogramBody = {
histo: { date_histogram: { ...getFieldRef(field), fixed_interval: fixedInterval } },
};
const results = (await aggSearchWithBody(histogramBody)) as ESSearchResponse<
const results = (await aggSearchWithBody({ aggs: histogramBody })) as ESSearchResponse<
unknown,
{ body: { aggs: typeof histogramBody } }
>;
@ -305,7 +342,43 @@ export async function getDateHistogram(
};
}
function getFieldRef(field: DataViewFieldBase) {
export async function getSimpleExamples(
search: SearchHandler,
field: DataViewField,
dataView: DataView
): Promise<FieldStatsResponse<string | number>> {
try {
const fieldRef = getFieldRef(field);
const simpleExamplesBody = {
size: SIMPLE_EXAMPLES_SIZE,
fields: [fieldRef],
};
const simpleExamplesResult = await search(simpleExamplesBody);
const fieldExampleBuckets = getFieldExampleBuckets({
hits: simpleExamplesResult.hits.hits,
field,
dataView,
count: DEFAULT_TOP_VALUES_SIZE,
});
return {
totalDocuments: getHitsTotal(simpleExamplesResult),
sampledDocuments: fieldExampleBuckets.sampledDocuments,
sampledValues: fieldExampleBuckets.sampledValues,
topValues: {
buckets: fieldExampleBuckets.buckets,
},
};
} catch (error) {
console.error(error); // eslint-disable-line no-console
return {};
}
}
function getFieldRef(field: DataViewField) {
return field.scripted
? {
script: {

View file

@ -92,6 +92,13 @@ describe('UnifiedFieldList <FieldStats />', () => {
aggregatable: true,
searchable: true,
},
{
name: 'geo_shape',
displayName: 'geo_shape',
type: 'geo_shape',
aggregatable: true,
searchable: true,
},
],
getFormatterForField: jest.fn(() => ({
convert: jest.fn((s: unknown) => JSON.stringify(s)),
@ -101,10 +108,7 @@ describe('UnifiedFieldList <FieldStats />', () => {
defaultProps = {
services: mockedServices,
dataViewOrDataViewId: dataView,
field: {
name: 'bytes',
type: 'number',
} as unknown as DataViewField,
field: dataView.fields.find((f) => f.name === 'bytes')!,
fromDate: 'now-7d',
toDate: 'now',
query: { query: '', language: 'lucene' },
@ -200,50 +204,64 @@ describe('UnifiedFieldList <FieldStats />', () => {
it('should not request field stats for range fields', async () => {
const wrapper = await mountWithIntl(
<FieldStats
{...defaultProps}
field={
{
name: 'ip_range',
displayName: 'ip_range',
type: 'ip_range',
} as DataViewField
}
/>
<FieldStats {...defaultProps} field={dataView.fields.find((f) => f.name === 'ip_range')!} />
);
await wrapper.update();
expect(loadFieldStats).not.toHaveBeenCalled();
});
it('should not request field stats for geo fields', async () => {
const wrapper = await mountWithIntl(
<FieldStats
{...defaultProps}
field={
{
name: 'geo_shape',
displayName: 'geo_shape',
type: 'geo_shape',
} as DataViewField
}
/>
);
await wrapper.update();
expect(loadFieldStats).not.toHaveBeenCalled();
});
it('should render nothing if no data is found', async () => {
const wrapper = mountWithIntl(<FieldStats {...defaultProps} />);
await wrapper.update();
expect(loadFieldStats).toHaveBeenCalled();
expect(wrapper.text()).toBe('');
expect(wrapper.text()).toBe('Analysis is not available for this field.');
});
it('should not request field stats for geo fields', async () => {
const wrapper = await mountWithIntl(
<FieldStats {...defaultProps} field={dataView.fields.find((f) => f.name === 'geo_shape')!} />
);
await wrapper.update();
expect(loadFieldStats).toHaveBeenCalled();
expect(wrapper.text()).toBe('Analysis is not available for this field.');
});
it('should render a message if no data is found', async () => {
const wrapper = await mountWithIntl(<FieldStats {...defaultProps} />);
await wrapper.update();
expect(loadFieldStats).toHaveBeenCalled();
expect(wrapper.text()).toBe('No field data for the current search.');
});
it('should render a message if no data is found in sample', async () => {
let resolveFunction: (arg: unknown) => void;
(loadFieldStats as jest.Mock).mockImplementation(() => {
return new Promise((resolve) => {
resolveFunction = resolve;
});
});
const wrapper = mountWithIntl(<FieldStats {...defaultProps} />);
await wrapper.update();
await act(async () => {
resolveFunction!({
totalDocuments: 10000,
sampledDocuments: 5000,
sampledValues: 0,
});
});
await wrapper.update();
expect(loadFieldStats).toHaveBeenCalledTimes(1);
expect(wrapper.text()).toBe('No field data for 5000 sample records.');
});
it('should render Top Values field stats correctly for a keyword field', async () => {
@ -333,19 +351,93 @@ describe('UnifiedFieldList <FieldStats />', () => {
const firstValue = stats.childAt(0);
expect(stats).toHaveLength(1);
expect(firstValue.find('[data-test-subj="testing-topValues-value"]').first().text()).toBe(
'"success"'
);
expect(firstValue.find('[data-test-subj="testing-topValues-valueCount"]').first().text()).toBe(
'41.5%'
);
expect(
firstValue.find('[data-test-subj="testing-topValues-formattedFieldValue"]').first().text()
).toBe('"success"');
expect(
firstValue.find('[data-test-subj="testing-topValues-formattedPercentage"]').first().text()
).toBe('41.5%');
expect(wrapper.find('[data-test-subj="testing-statsFooter"]').first().text()).toBe(
'100% of 1624 documents'
'Calculated from 1624 records.'
);
expect(wrapper.text()).toBe(
'Top values"success"41.5%"info"37.1%"security"10.1%"warning"5.0%"error"3.4%"login"2.7%100% of 1624 documents'
'Top values"success"41.5%"info"37.1%"security"10.1%"warning"5.0%"error"3.4%"login"2.7%Calculated from 1624 records.'
);
});
it('should render Examples correctly for a non-aggregatable field', async () => {
let resolveFunction: (arg: unknown) => void;
(loadFieldStats as jest.Mock).mockImplementation(() => {
return new Promise((resolve) => {
resolveFunction = resolve;
});
});
const wrapper = mountWithIntl(
<FieldStats
{...defaultProps}
field={
{
name: 'test_text',
type: 'string',
aggregatable: false,
} as unknown as DataViewField
}
/>
);
await wrapper.update();
expect(wrapper.find(EuiLoadingSpinner)).toHaveLength(1);
await act(async () => {
resolveFunction!({
totalDocuments: 1624,
sampledDocuments: 1624,
sampledValues: 3248,
topValues: {
buckets: [
{
count: 1349,
key: 'success',
},
{
count: 1206,
key: 'info',
},
{
count: 329,
key: 'security',
},
{
count: 164,
key: 'warning',
},
{
count: 111,
key: 'error',
},
{
count: 89,
key: 'login',
},
],
},
});
});
await wrapper.update();
expect(wrapper.find(EuiLoadingSpinner)).toHaveLength(0);
expect(wrapper.find(EuiProgress)).toHaveLength(6);
expect(loadFieldStats).toHaveBeenCalledTimes(1);
expect(wrapper.text()).toBe(
'Examples"success"41.5%"info"37.1%"security"10.1%"warning"5.0%"error"3.4%"login"2.7%Calculated from 1624 records.'
);
});
@ -449,10 +541,10 @@ describe('UnifiedFieldList <FieldStats />', () => {
expect(wrapper.find('[data-test-subj="testing-topValues"]')).toHaveLength(0);
expect(wrapper.find('[data-test-subj="testing-histogram"]')).toHaveLength(1);
expect(wrapper.find('[data-test-subj="testing-statsFooter"]').first().text()).toBe(
'13 documents'
'Calculated from 13 records.'
);
expect(wrapper.text()).toBe('Time distribution13 documents');
expect(wrapper.text()).toBe('Time distributionCalculated from 13 records.');
});
it('should render Top Values & Distribution field stats correctly for a number field', async () => {
@ -500,7 +592,7 @@ describe('UnifiedFieldList <FieldStats />', () => {
await act(async () => {
resolveFunction!({
totalDocuments: 23,
totalDocuments: 100,
sampledDocuments: 23,
sampledValues: 23,
histogram: {
@ -537,7 +629,7 @@ describe('UnifiedFieldList <FieldStats />', () => {
expect(loadFieldStats).toHaveBeenCalledTimes(1);
expect(wrapper.text()).toBe(
'Toggle either theTop valuesDistribution1273.9%1326.1%100% of 23 documents'
'Toggle either theTop valuesDistribution1273.9%1326.1%Calculated from 23 sample records.'
);
});
});

View file

@ -20,19 +20,8 @@ import type { DataPublicPluginStart } from '@kbn/data-plugin/public';
import type { FieldFormatsStart } from '@kbn/field-formats-plugin/public';
import type { ChartsPluginSetup } from '@kbn/charts-plugin/public';
import DateMath from '@kbn/datemath';
import {
EuiButtonGroup,
EuiFlexGroup,
EuiFlexItem,
EuiLoadingSpinner,
EuiProgress,
EuiSpacer,
EuiText,
EuiTitle,
EuiToolTip,
useEuiTheme,
} from '@elastic/eui';
import { css } from '@emotion/react';
import { EuiButtonGroup, EuiLoadingSpinner, EuiSpacer, EuiText, EuiTitle } from '@elastic/eui';
import { FormattedMessage } from '@kbn/i18n-react';
import {
Axis,
Chart,
@ -48,6 +37,14 @@ import { buildEsQuery, Query, Filter, AggregateQuery } from '@kbn/es-query';
import type { BucketedAggregation } from '../../../common/types';
import { canProvideStatsForField } from '../../../common/utils/field_stats_utils';
import { loadFieldStats } from '../../services/field_stats';
import type { AddFieldFilterHandler } from '../../types';
import {
FieldTopValues,
getOtherCount,
getBucketsValuesCount,
getDefaultColor,
} from './field_top_values';
import { FieldSummaryMessage } from './field_summary_message';
interface State {
isLoading: boolean;
@ -74,13 +71,18 @@ export interface FieldStatsProps {
toDate: string;
dataViewOrDataViewId: DataView | string;
field: DataViewField;
color?: string;
'data-test-subj'?: string;
overrideMissingContent?: (params?: { noDataFound?: boolean }) => JSX.Element | null;
overrideMissingContent?: (params: {
element: JSX.Element;
noDataFound?: boolean;
}) => JSX.Element | null;
overrideFooter?: (params: {
element: JSX.Element;
totalDocuments?: number;
sampledDocuments?: number;
}) => JSX.Element;
onAddFilter?: AddFieldFilterHandler;
}
const FieldStatsComponent: React.FC<FieldStatsProps> = ({
@ -91,11 +93,12 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
toDate,
dataViewOrDataViewId,
field,
color = getDefaultColor(),
'data-test-subj': dataTestSubject = 'fieldStats',
overrideMissingContent,
overrideFooter,
onAddFilter,
}) => {
const { euiTheme } = useEuiTheme();
const { fieldFormats, uiSettings, charts, dataViews, data } = services;
const [state, changeState] = useState<State>({
isLoading: false,
@ -104,28 +107,6 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
const abortControllerRef = useRef<AbortController | null>(null);
const isCanceledRef = useRef<boolean>(false);
const topValueStyles = useMemo(
() => css`
margin-bottom: ${euiTheme.size.s};
&:last-of-type {
margin-bottom: 0;
}
`,
[euiTheme]
);
const topValueProgressStyles = useMemo(
() => css`
background-color: ${euiTheme.colors.lightestShade};
&::-webkit-progress-bar {
background-color: ${euiTheme.colors.lightestShade};
}
`,
[euiTheme]
);
const setState: typeof changeState = useCallback(
(nextState) => {
if (!isCanceledRef.current) {
@ -157,7 +138,7 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
setDataView(loadedDataView);
if (state.isLoading || !canProvideStatsForField(field)) {
if (state.isLoading) {
return;
}
@ -204,6 +185,20 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
const chartTheme = charts.theme.useChartsTheme();
const chartBaseTheme = charts.theme.useChartsBaseTheme();
const customChartTheme: typeof chartTheme = useMemo(() => {
return color
? {
...chartTheme,
barSeriesStyle: {
...chartTheme.barSeriesStyle,
rect: {
...(chartTheme.barSeriesStyle?.rect || {}),
fill: color,
},
},
}
: chartTheme;
}, [chartTheme, color]);
const { isLoading, histogram, topValues, sampledValues, sampledDocuments, totalDocuments } =
state;
@ -212,19 +207,18 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
const fromDateParsed = DateMath.parse(fromDate);
const toDateParsed = DateMath.parse(toDate);
const totalValuesCount =
topValues && topValues.buckets.reduce((prev, bucket) => bucket.count + prev, 0);
const otherCount = sampledValues && totalValuesCount ? sampledValues - totalValuesCount : 0;
const bucketsValuesCount = getBucketsValuesCount(topValues?.buckets);
const otherCount = getOtherCount(bucketsValuesCount, sampledValues!);
if (
totalValuesCount &&
bucketsValuesCount &&
histogram &&
histogram.buckets.length &&
topValues &&
topValues.buckets.length
) {
// Default to histogram when top values are less than 10% of total
histogramDefault = otherCount / totalValuesCount > 0.9;
histogramDefault = otherCount / bucketsValuesCount > 0.9;
}
const [showingHistogram, setShowingHistogram] = useState(histogramDefault);
@ -240,39 +234,114 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
const formatter = dataView.getFormatterForField(field);
let title = <></>;
if (field.type.includes('range')) {
function combineWithTitleAndFooter(el: React.ReactElement) {
const countsElement = totalDocuments ? (
<EuiText color="subdued" size="xs" data-test-subj={`${dataTestSubject}-statsFooter`}>
{sampledDocuments && sampledDocuments < totalDocuments ? (
<FormattedMessage
id="unifiedFieldList.fieldStats.calculatedFromSampleRecordsLabel"
defaultMessage="Calculated from {sampledDocumentsFormatted} sample {sampledDocuments, plural, one {record} other {records}}."
values={{
sampledDocuments,
sampledDocumentsFormatted: (
<strong>
{fieldFormats
.getDefaultInstance(KBN_FIELD_TYPES.NUMBER, [ES_FIELD_TYPES.INTEGER])
.convert(sampledDocuments)}
</strong>
),
}}
/>
) : (
<FormattedMessage
id="unifiedFieldList.fieldStats.calculatedFromTotalRecordsLabel"
defaultMessage="Calculated from {totalDocumentsFormatted} {totalDocuments, plural, one {record} other {records}}."
values={{
totalDocuments,
totalDocumentsFormatted: (
<strong>
{fieldFormats
.getDefaultInstance(KBN_FIELD_TYPES.NUMBER, [ES_FIELD_TYPES.INTEGER])
.convert(totalDocuments)}
</strong>
),
}}
/>
)}
</EuiText>
) : (
<></>
);
return (
<>
<EuiText size="s">
{i18n.translate('unifiedFieldList.fieldStats.notAvailableForRangeFieldDescription', {
defaultMessage: `Summary information is not available for range type fields.`,
})}
</EuiText>
{title ? <div data-test-subj={`${dataTestSubject}-title`}>{title}</div> : <></>}
<EuiSpacer size="s" />
{el}
{overrideFooter ? (
overrideFooter?.({ element: countsElement, totalDocuments, sampledDocuments })
) : (
<>
<EuiSpacer size="m" />
{countsElement}
</>
)}
</>
);
}
if (field.type === 'murmur3') {
return (
<>
<EuiText size="s">
{i18n.translate('unifiedFieldList.fieldStats.notAvailableForMurmur3FieldDescription', {
defaultMessage: `Summary information is not available for murmur3 fields.`,
})}
</EuiText>
</>
if (!canProvideStatsForField(field)) {
const messageNoAnalysis = (
<FieldSummaryMessage
message={i18n.translate('unifiedFieldList.fieldStats.notAvailableForThisFieldDescription', {
defaultMessage: 'Analysis is not available for this field.',
})}
/>
);
}
if (field.type === 'geo_point' || field.type === 'geo_shape') {
return overrideMissingContent ? overrideMissingContent() : null;
return overrideMissingContent
? overrideMissingContent({
noDataFound: false,
element: messageNoAnalysis,
})
: messageNoAnalysis;
}
if (
(!histogram || histogram.buckets.length === 0) &&
(!topValues || topValues.buckets.length === 0)
) {
return overrideMissingContent ? overrideMissingContent({ noDataFound: true }) : null;
const messageNoData =
sampledDocuments && totalDocuments && sampledDocuments < totalDocuments ? (
<FieldSummaryMessage
message={i18n.translate('unifiedFieldList.fieldStats.noFieldDataInSampleDescription', {
defaultMessage:
'No field data for {sampledDocumentsFormatted} sample {sampledDocuments, plural, one {record} other {records}}.',
values: {
sampledDocuments,
sampledDocumentsFormatted: fieldFormats
.getDefaultInstance(KBN_FIELD_TYPES.NUMBER, [ES_FIELD_TYPES.INTEGER])
.convert(sampledDocuments),
},
})}
/>
) : (
<FieldSummaryMessage
message={i18n.translate('unifiedFieldList.fieldStats.noFieldDataDescription', {
defaultMessage: 'No field data for the current search.',
})}
/>
);
return overrideMissingContent
? overrideMissingContent({
noDataFound: true,
element: messageNoData,
})
: messageNoData;
}
if (histogram && histogram.buckets.length && topValues && topValues.buckets.length) {
@ -317,60 +386,18 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
title = (
<EuiTitle size="xxxs">
<h6>
{i18n.translate('unifiedFieldList.fieldStats.topValuesLabel', {
defaultMessage: 'Top values',
})}
{field.aggregatable
? i18n.translate('unifiedFieldList.fieldStats.topValuesLabel', {
defaultMessage: 'Top values',
})
: i18n.translate('unifiedFieldList.fieldStats.examplesLabel', {
defaultMessage: 'Examples',
})}
</h6>
</EuiTitle>
);
}
function combineWithTitleAndFooter(el: React.ReactElement) {
const countsElement = totalDocuments ? (
<EuiText color="subdued" size="xs" data-test-subj={`${dataTestSubject}-statsFooter`}>
{sampledDocuments && (
<>
{i18n.translate('unifiedFieldList.fieldStats.percentageOfLabel', {
defaultMessage: '{percentage}% of',
values: {
percentage: Math.round((sampledDocuments / totalDocuments) * 100),
},
})}{' '}
</>
)}
<strong>
{fieldFormats
.getDefaultInstance(KBN_FIELD_TYPES.NUMBER, [ES_FIELD_TYPES.INTEGER])
.convert(totalDocuments)}
</strong>{' '}
{i18n.translate('unifiedFieldList.fieldStats.ofDocumentsLabel', {
defaultMessage: 'documents',
})}
</EuiText>
) : (
<></>
);
return (
<>
{title ? title : <></>}
<EuiSpacer size="s" />
{el}
{overrideFooter ? (
overrideFooter?.({ element: countsElement, totalDocuments, sampledDocuments })
) : (
<>
<EuiSpacer />
{countsElement}
</>
)}
</>
);
}
if (histogram && histogram.buckets.length) {
const specId = i18n.translate('unifiedFieldList.fieldStats.countLabel', {
defaultMessage: 'Count',
@ -378,48 +405,47 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
if (field.type === 'date') {
return combineWithTitleAndFooter(
<Chart
data-test-subj={`${dataTestSubject}-histogram`}
size={{ height: 200, width: 300 - 32 }}
>
<Settings
tooltip={{ type: TooltipType.None }}
theme={chartTheme}
baseTheme={chartBaseTheme}
xDomain={
fromDateParsed && toDateParsed
? {
min: fromDateParsed.valueOf(),
max: toDateParsed.valueOf(),
minInterval: Math.round(
(toDateParsed.valueOf() - fromDateParsed.valueOf()) / 10
),
}
: undefined
}
/>
<div data-test-subj={`${dataTestSubject}-histogram`}>
<Chart size={{ height: 200, width: 300 - 32 }}>
<Settings
tooltip={{ type: TooltipType.None }}
theme={customChartTheme}
baseTheme={chartBaseTheme}
xDomain={
fromDateParsed && toDateParsed
? {
min: fromDateParsed.valueOf(),
max: toDateParsed.valueOf(),
minInterval: Math.round(
(toDateParsed.valueOf() - fromDateParsed.valueOf()) / 10
),
}
: undefined
}
/>
<Axis
id="key"
position={Position.Bottom}
tickFormat={
fromDateParsed && toDateParsed
? niceTimeFormatter([fromDateParsed.valueOf(), toDateParsed.valueOf()])
: undefined
}
showOverlappingTicks={true}
/>
<Axis
id="key"
position={Position.Bottom}
tickFormat={
fromDateParsed && toDateParsed
? niceTimeFormatter([fromDateParsed.valueOf(), toDateParsed.valueOf()])
: undefined
}
showOverlappingTicks={true}
/>
<HistogramBarSeries
data={histogram.buckets}
id={specId}
xAccessor={'key'}
yAccessors={['count']}
xScaleType={ScaleType.Time}
yScaleType={ScaleType.Linear}
timeZone="local"
/>
</Chart>
<HistogramBarSeries
data={histogram.buckets}
id={specId}
xAccessor={'key'}
yAccessors={['count']}
xScaleType={ScaleType.Time}
yScaleType={ScaleType.Linear}
timeZone="local"
/>
</Chart>
</div>
);
}
@ -432,7 +458,7 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
<Settings
rotation={90}
tooltip={{ type: TooltipType.None }}
theme={chartTheme}
theme={customChartTheme}
baseTheme={chartBaseTheme}
/>
@ -457,97 +483,16 @@ const FieldStatsComponent: React.FC<FieldStatsProps> = ({
}
if (topValues && topValues.buckets.length) {
const digitsRequired = topValues.buckets.some(
(topValue) => !Number.isInteger(topValue.count / sampledValues!)
);
return combineWithTitleAndFooter(
<div data-test-subj={`${dataTestSubject}-topValues`}>
{topValues.buckets.map((topValue) => {
const formatted = formatter.convert(topValue.key);
return (
<div css={topValueStyles} key={topValue.key}>
<EuiFlexGroup
alignItems="stretch"
key={topValue.key}
gutterSize="xs"
responsive={false}
>
<EuiFlexItem
grow={true}
className="eui-textTruncate"
data-test-subj={`${dataTestSubject}-topValues-value`}
>
{formatted === '' ? (
<EuiText size="xs" color="subdued">
<em>
{i18n.translate('unifiedFieldList.fieldStats.emptyStringValueLabel', {
defaultMessage: 'Empty string',
})}
</em>
</EuiText>
) : (
<EuiToolTip content={formatted} delay="long">
<EuiText size="xs" color="subdued" className="eui-textTruncate">
{formatted}
</EuiText>
</EuiToolTip>
)}
</EuiFlexItem>
<EuiFlexItem
grow={false}
data-test-subj={`${dataTestSubject}-topValues-valueCount`}
>
<EuiText size="xs" textAlign="left" color="accent">
{(Math.round((topValue.count / sampledValues!) * 1000) / 10).toFixed(
digitsRequired ? 1 : 0
)}
%
</EuiText>
</EuiFlexItem>
</EuiFlexGroup>
<EuiProgress
css={topValueProgressStyles}
value={topValue.count / sampledValues!}
max={1}
size="s"
color="accent"
/>
</div>
);
})}
{otherCount ? (
<>
<EuiFlexGroup alignItems="stretch" gutterSize="xs" responsive={false}>
<EuiFlexItem grow={true} className="eui-textTruncate">
<EuiText size="xs" className="eui-textTruncate" color="subdued">
{i18n.translate('unifiedFieldList.fieldStats.otherDocsLabel', {
defaultMessage: 'Other',
})}
</EuiText>
</EuiFlexItem>
<EuiFlexItem grow={false} className="eui-textTruncate">
<EuiText size="xs" color="subdued">
{(Math.round((otherCount / sampledValues!) * 1000) / 10).toFixed(
digitsRequired ? 1 : 0
)}
%
</EuiText>
</EuiFlexItem>
</EuiFlexGroup>
<EuiProgress
css={topValueProgressStyles}
value={otherCount / sampledValues!}
max={1}
size="s"
color="subdued"
/>
</>
) : (
<></>
)}
</div>
<FieldTopValues
buckets={topValues.buckets}
dataView={dataView}
field={field}
sampledValuesCount={sampledValues!}
color={color}
data-test-subj={dataTestSubject}
onAddFilter={onAddFilter}
/>
);
}

View file

@ -0,0 +1,18 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import React from 'react';
import { EuiText } from '@elastic/eui';
export interface FieldSummaryMessageProps {
message: string;
}
export const FieldSummaryMessage: React.FC<FieldSummaryMessageProps> = ({ message }) => {
return <EuiText size="s">{message}</EuiText>;
};

View file

@ -0,0 +1,148 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import React from 'react';
import { EuiProgress, EuiButtonIcon } from '@elastic/eui';
import { mountWithIntl } from '@kbn/test-jest-helpers';
import type { DataView } from '@kbn/data-views-plugin/common';
import { FieldTopValues, FieldTopValuesProps } from './field_top_values';
import { fieldFormatsServiceMock } from '@kbn/field-formats-plugin/public/mocks';
import { ES_FIELD_TYPES, KBN_FIELD_TYPES } from '@kbn/data-plugin/common';
describe('UnifiedFieldList <FieldTopValues />', () => {
let defaultProps: FieldTopValuesProps;
let dataView: DataView;
beforeEach(() => {
dataView = {
id: '1',
title: 'my-fake-index-pattern',
timeFieldName: 'timestamp',
fields: [
{
name: 'source',
displayName: 'source',
type: 'string',
aggregatable: true,
searchable: true,
filterable: true,
},
],
getFormatterForField: jest.fn(() => ({
convert: jest.fn((s: unknown) =>
fieldFormatsServiceMock
.createStartContract()
.getDefaultInstance(KBN_FIELD_TYPES.STRING, [ES_FIELD_TYPES.STRING])
.convert(s)
),
})),
} as unknown as DataView;
defaultProps = {
dataView,
field: dataView.fields.find((f) => f.name === 'source')!,
sampledValuesCount: 5000,
buckets: [
{
count: 500,
key: 'sourceA',
},
{
count: 1,
key: 'sourceB',
},
],
'data-test-subj': 'testing',
};
});
it('should render correctly without filter actions', async () => {
const wrapper = mountWithIntl(<FieldTopValues {...defaultProps} />);
expect(wrapper.text()).toBe('sourceA10.0%sourceB0.0%Other90.0%');
expect(wrapper.find(EuiProgress)).toHaveLength(3);
expect(wrapper.find(EuiButtonIcon)).toHaveLength(0);
});
it('should render correctly with filter actions', async () => {
const mockAddFilter = jest.fn();
const wrapper = mountWithIntl(<FieldTopValues {...defaultProps} onAddFilter={mockAddFilter} />);
expect(wrapper.text()).toBe('sourceA10.0%sourceB0.0%Other90.0%');
expect(wrapper.find(EuiProgress)).toHaveLength(3);
expect(wrapper.find(EuiButtonIcon)).toHaveLength(4);
wrapper.find(EuiButtonIcon).first().simulate('click');
expect(mockAddFilter).toHaveBeenCalledWith(defaultProps.field, 'sourceA', '+');
});
it('should render correctly without Other section', async () => {
const wrapper = mountWithIntl(
<FieldTopValues
{...defaultProps}
buckets={[
{
count: 3000,
key: 'sourceA',
},
{
count: 1500,
key: 'sourceB',
},
{
count: 500,
key: 'sourceC',
},
]}
/>
);
expect(wrapper.text()).toBe('sourceA60.0%sourceB30.0%sourceC10.0%');
});
it('should render correctly with empty strings', async () => {
const wrapper = mountWithIntl(
<FieldTopValues
{...defaultProps}
buckets={[
{
count: 3000,
key: '',
},
{
count: 1500,
key: 'sourceA',
},
{
count: 20,
key: 'sourceB',
},
]}
/>
);
expect(wrapper.text()).toBe('(empty)60.0%sourceA30.0%sourceB0.4%Other9.6%');
});
it('should render correctly without floating point', async () => {
const wrapper = mountWithIntl(
<FieldTopValues
{...defaultProps}
buckets={[
{
count: 5000,
key: 'sourceA',
},
]}
/>
);
expect(wrapper.text()).toBe('sourceA100%');
});
});

View file

@ -0,0 +1,120 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import React, { Fragment } from 'react';
import { euiPaletteColorBlind, EuiSpacer } from '@elastic/eui';
import { DataView, DataViewField } from '@kbn/data-plugin/common';
import type { BucketedAggregation } from '../../../common/types';
import type { AddFieldFilterHandler } from '../../types';
import { FieldTopValuesBucket } from './field_top_values_bucket';
export interface FieldTopValuesProps {
buckets: BucketedAggregation<number | string>['buckets'];
dataView: DataView;
field: DataViewField;
sampledValuesCount: number;
color?: string;
'data-test-subj': string;
onAddFilter?: AddFieldFilterHandler;
}
export const FieldTopValues: React.FC<FieldTopValuesProps> = ({
buckets,
dataView,
field,
sampledValuesCount,
color = getDefaultColor(),
'data-test-subj': dataTestSubject,
onAddFilter,
}) => {
if (!buckets?.length) {
return null;
}
const formatter = dataView.getFormatterForField(field);
const otherCount = getOtherCount(getBucketsValuesCount(buckets), sampledValuesCount);
const digitsRequired = buckets.some(
(bucket) => !Number.isInteger(bucket.count / sampledValuesCount)
);
return (
<div data-test-subj={`${dataTestSubject}-topValues`}>
{buckets.map((bucket, index) => {
const fieldValue = bucket.key;
const formatted = formatter.convert(fieldValue);
return (
<Fragment key={fieldValue}>
{index > 0 && <EuiSpacer size="s" />}
<FieldTopValuesBucket
field={field}
fieldValue={fieldValue}
formattedFieldValue={formatted}
formattedPercentage={getFormattedPercentageValue(
bucket.count,
sampledValuesCount,
digitsRequired
)}
progressValue={getProgressValue(bucket.count, sampledValuesCount)}
count={bucket.count}
color={color}
data-test-subj={dataTestSubject}
onAddFilter={onAddFilter}
/>
</Fragment>
);
})}
{otherCount > 0 && (
<>
<EuiSpacer size="s" />
<FieldTopValuesBucket
type="other"
field={field}
fieldValue={undefined}
formattedPercentage={getFormattedPercentageValue(
otherCount,
sampledValuesCount,
digitsRequired
)}
progressValue={getProgressValue(otherCount, sampledValuesCount)}
count={otherCount}
color={color}
data-test-subj={dataTestSubject}
onAddFilter={onAddFilter}
/>
</>
)}
</div>
);
};
export const getDefaultColor = () => euiPaletteColorBlind()[1];
export const getFormattedPercentageValue = (
currentValue: number,
totalCount: number,
digitsRequired: boolean
): string => {
return totalCount > 0
? `${(Math.round((currentValue / totalCount) * 1000) / 10).toFixed(digitsRequired ? 1 : 0)}%`
: '';
};
export const getProgressValue = (currentValue: number, totalCount: number): number => {
return totalCount > 0 ? currentValue / totalCount : 0;
};
export const getBucketsValuesCount = (
buckets?: BucketedAggregation<number | string>['buckets']
): number => {
return buckets?.reduce((prev, bucket) => bucket.count + prev, 0) || 0;
};
export const getOtherCount = (bucketsValuesCount: number, sampledValuesCount: number): number => {
return sampledValuesCount && bucketsValuesCount ? sampledValuesCount - bucketsValuesCount : 0;
};

View file

@ -0,0 +1,177 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
import React from 'react';
import {
EuiButtonIcon,
EuiFlexGroup,
EuiFlexItem,
EuiProgress,
EuiText,
EuiToolTip,
} from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import { css } from '@emotion/react';
import type { IFieldSubTypeMulti } from '@kbn/es-query';
import type { DataViewField } from '@kbn/data-views-plugin/common';
import type { AddFieldFilterHandler } from '../../types';
export interface FieldTopValuesBucketProps {
type?: 'normal' | 'other';
field: DataViewField;
fieldValue: unknown;
formattedFieldValue?: string;
formattedPercentage: string;
progressValue: number;
count: number;
color: string;
'data-test-subj': string;
onAddFilter?: AddFieldFilterHandler;
}
export const FieldTopValuesBucket: React.FC<FieldTopValuesBucketProps> = ({
type = 'normal',
field,
fieldValue,
formattedFieldValue,
formattedPercentage,
progressValue,
count,
color,
'data-test-subj': dataTestSubject,
onAddFilter,
}) => {
const fieldLabel = (field?.subType as IFieldSubTypeMulti)?.multi?.parent ?? field.name;
return (
<EuiFlexGroup
alignItems="stretch"
gutterSize="s"
responsive={false}
data-test-subj={`${dataTestSubject}-topValues-bucket`}
>
<EuiFlexItem
grow={1}
css={css`
min-width: 0;
`}
>
<EuiFlexGroup alignItems="stretch" gutterSize="s" responsive={false}>
<EuiFlexItem
grow={true}
className="eui-textTruncate"
data-test-subj={`${dataTestSubject}-topValues-formattedFieldValue`}
>
{(formattedFieldValue?.length ?? 0) > 0 ? (
<EuiToolTip content={formattedFieldValue} delay="long">
<EuiText size="xs" className="eui-textTruncate" color="subdued">
{formattedFieldValue}
</EuiText>
</EuiToolTip>
) : (
<EuiText size="xs">
{type === 'other'
? i18n.translate('unifiedFieldList.fieldStats.otherDocsLabel', {
defaultMessage: 'Other',
})
: formattedFieldValue === ''
? i18n.translate('unifiedFieldList.fieldStats.emptyStringValueLabel', {
defaultMessage: '(empty)',
})
: '-'}
</EuiText>
)}
</EuiFlexItem>
<EuiFlexItem
grow={false}
data-test-subj={`${dataTestSubject}-topValues-formattedPercentage`}
>
<EuiToolTip
content={i18n.translate('unifiedFieldList.fieldStats.bucketPercentageTooltip', {
defaultMessage:
'{formattedPercentage} ({count, plural, one {# record} other {# records}})',
values: {
formattedPercentage,
count,
},
})}
delay="long"
>
<EuiText size="xs" textAlign="left" color={color}>
{formattedPercentage}
</EuiText>
</EuiToolTip>
</EuiFlexItem>
</EuiFlexGroup>
<EuiProgress
value={progressValue}
max={1}
size="s"
color={type === 'other' ? 'subdued' : color}
aria-label={`${formattedFieldValue} (${formattedPercentage})`}
/>
</EuiFlexItem>
{onAddFilter && field.filterable && (
<EuiFlexItem grow={false}>
{type === 'other' ? (
<div
css={css`
width: 48px;
`}
/>
) : (
<div>
<EuiButtonIcon
iconSize="s"
iconType="plusInCircle"
onClick={() => onAddFilter(field, fieldValue, '+')}
aria-label={i18n.translate(
'unifiedFieldList.fieldStats.filterValueButtonAriaLabel',
{
defaultMessage: 'Filter for {field}: "{value}"',
values: { value: formattedFieldValue, field: fieldLabel },
}
)}
data-test-subj={`plus-${fieldLabel}-${fieldValue}`}
style={{
minHeight: 'auto',
minWidth: 'auto',
paddingRight: 2,
paddingLeft: 2,
paddingTop: 0,
paddingBottom: 0,
}}
/>
<EuiButtonIcon
iconSize="s"
iconType="minusInCircle"
onClick={() => onAddFilter(field, fieldValue, '-')}
aria-label={i18n.translate(
'unifiedFieldList.fieldStats.filterOutValueButtonAriaLabel',
{
defaultMessage: 'Filter out {field}: "{value}"',
values: { value: formattedFieldValue, field: fieldLabel },
}
)}
data-test-subj={`minus-${fieldLabel}-${fieldValue}`}
style={{
minHeight: 'auto',
minWidth: 'auto',
paddingTop: 0,
paddingBottom: 0,
paddingRight: 2,
paddingLeft: 2,
}}
/>
</div>
)}
</EuiFlexItem>
)}
</EuiFlexGroup>
);
};

View file

@ -24,4 +24,8 @@ export { loadFieldExisting } from './services/field_existing';
export function plugin() {
return new UnifiedFieldListPlugin();
}
export type { UnifiedFieldListPluginSetup, UnifiedFieldListPluginStart } from './types';
export type {
UnifiedFieldListPluginSetup,
UnifiedFieldListPluginStart,
AddFieldFilterHandler,
} from './types';

View file

@ -7,9 +7,8 @@
*/
import { lastValueFrom } from 'rxjs';
import type { DataView } from '@kbn/data-views-plugin/common';
import type { DataView, DataViewField } from '@kbn/data-views-plugin/common';
import type { DataPublicPluginStart } from '@kbn/data-plugin/public';
import type { DataViewFieldBase } from '@kbn/es-query';
import type { FieldStatsResponse } from '../../../common/types';
import {
fetchAndCalculateFieldStats,
@ -22,7 +21,7 @@ interface FetchFieldStatsParams {
data: DataPublicPluginStart;
};
dataView: DataView;
field: DataViewFieldBase;
field: DataViewField;
fromDate: string;
toDate: string;
dslQuery: object;
@ -62,7 +61,7 @@ export const loadFieldStats: LoadFieldStatsHandler = async ({
return {};
}
const searchHandler: SearchHandler = async (aggs) => {
const searchHandler: SearchHandler = async (body) => {
const result = await lastValueFrom(
data.search.search(
{
@ -73,7 +72,7 @@ export const loadFieldStats: LoadFieldStatsHandler = async ({
toDate,
dslQuery,
runtimeMappings: dataView.getRuntimeMappings(),
aggs,
...body,
}),
},
{
@ -86,6 +85,7 @@ export const loadFieldStats: LoadFieldStatsHandler = async ({
return await fetchAndCalculateFieldStats({
searchHandler,
dataView,
field,
fromDate,
toDate,

View file

@ -0,0 +1,9 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
export { loadFieldStats } from './field_stats';

View file

@ -6,8 +6,12 @@
* Side Public License, v 1.
*/
import type { DataViewField } from '@kbn/data-views-plugin/common';
// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface UnifiedFieldListPluginSetup {}
// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface UnifiedFieldListPluginStart {}
export type AddFieldFilterHandler = (field: DataViewField, value: unknown, type: '+' | '-') => void;

View file

@ -57,7 +57,7 @@ export async function initFieldStatsRoute(setup: CoreSetup<PluginStart>) {
throw new Error(`Field {fieldName} not found in data view ${dataView.title}`);
}
const searchHandler: SearchHandler = async (aggs) => {
const searchHandler: SearchHandler = async (body) => {
const result = await requestClient.search(
buildSearchParams({
dataViewPattern: dataView.title,
@ -66,7 +66,7 @@ export async function initFieldStatsRoute(setup: CoreSetup<PluginStart>) {
toDate,
dslQuery,
runtimeMappings: dataView.getRuntimeMappings(),
aggs,
...body,
})
);
return result;
@ -74,6 +74,7 @@ export async function initFieldStatsRoute(setup: CoreSetup<PluginStart>) {
const stats = await fetchAndCalculateFieldStats({
searchHandler,
dataView,
field,
fromDate,
toDate,

View file

@ -395,6 +395,25 @@ export default ({ getService }: FtrProviderContext) => {
});
});
it('should return examples for non-aggregatable fields', async () => {
const { body } = await supertest
.post(API_PATH)
.set(COMMON_HEADERS)
.send({
dataViewId: 'logstash-2015.09.22',
dslQuery: { match_all: {} },
fromDate: TEST_START_TIME,
toDate: TEST_END_TIME,
fieldName: 'extension', // `extension.keyword` is an aggregatable field but `extension` is not
})
.expect(200);
expect(body.totalDocuments).to.eql(4634);
expect(body.sampledDocuments).to.eql(100);
expect(body.sampledValues).to.eql(100);
expect(body.topValues.buckets.length).to.eql(5);
});
it('should return top values for index pattern runtime string fields', async () => {
const { body } = await supertest
.post(API_PATH)

View file

@ -11,7 +11,7 @@ import { FtrProviderContext } from '../../ftr_provider_context';
const TEST_COLUMN_NAMES = ['@message'];
const TEST_FILTER_COLUMN_NAMES = [
['extension', 'jpg'],
['extension.raw', 'jpg'],
['geo.src', 'IN'],
];

View file

@ -11,8 +11,8 @@ import { FtrProviderContext } from '../../../ftr_provider_context';
const TEST_COLUMN_NAMES = ['@message'];
const TEST_FILTER_COLUMN_NAMES = [
['extension', 'jpg'],
['geo.src', 'IN'],
['extension', 'jpg', 'extension.raw'],
['geo.src', 'IN', 'geo.src'],
];
export default function ({ getService, getPageObjects }: FtrProviderContext) {
@ -98,8 +98,8 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
it('should open the context view with the filters disabled', async () => {
let disabledFilterCounter = 0;
for (const [columnName, value] of TEST_FILTER_COLUMN_NAMES) {
if (await filterBar.hasFilter(columnName, value, false)) {
for (const [_, value, columnId] of TEST_FILTER_COLUMN_NAMES) {
if (await filterBar.hasFilter(columnId, value, false)) {
disabledFilterCounter++;
}
}

View file

@ -11,8 +11,8 @@ import { FtrProviderContext } from '../ftr_provider_context';
const TEST_COLUMN_NAMES = ['@message'];
const TEST_FILTER_COLUMN_NAMES = [
['extension', 'jpg'],
['geo.src', 'IN'],
['extension', 'jpg', 'extension.raw'],
['geo.src', 'IN', 'geo.src'],
];
export default function ({ getService, getPageObjects }: FtrProviderContext) {
@ -84,8 +84,8 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
it('should open the context view with the filters disabled', async () => {
let disabledFilterCounter = 0;
for (const [columnName, value] of TEST_FILTER_COLUMN_NAMES) {
if (await filterBar.hasFilter(columnName, value, false)) {
for (const [_, value, columnId] of TEST_FILTER_COLUMN_NAMES) {
if (await filterBar.hasFilter(columnId, value, false)) {
disabledFilterCounter++;
}
}

View file

@ -487,12 +487,10 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
});
it('should filter by scripted field value in Discover', async function () {
await PageObjects.discover.clickFieldListItem(scriptedPainlessFieldName2);
await log.debug('filter by "Sep 17, 2015 @ 23:00" in the expanded scripted field list');
await PageObjects.discover.clickFieldListPlusFilter(
scriptedPainlessFieldName2,
'1442531297065'
);
await PageObjects.header.waitUntilLoadingHasFinished();
const documentCell = await dataGrid.getCellElement(0, 3);
await documentCell.click();
await testSubjects.click('filterForButton');
await PageObjects.header.waitUntilLoadingHasFinished();
await retry.try(async function () {

View file

@ -48,7 +48,10 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) {
await browser.setWindowSize(1200, 800);
await kibanaServer.importExport.load('test/functional/fixtures/kbn_archiver/discover');
await kibanaServer.uiSettings.replace({});
await kibanaServer.uiSettings.update({ 'doc_table:legacy': true });
await kibanaServer.uiSettings.update({
'doc_table:legacy': true,
'discover:showLegacyFieldTopValues': true,
});
});
after(async function afterAll() {

View file

@ -22,7 +22,8 @@ import { KibanaContextProvider } from '@kbn/kibana-react-plugin/public';
import { dataPluginMock } from '@kbn/data-plugin/public/mocks';
import { dataViewPluginMocks } from '@kbn/data-views-plugin/public/mocks';
import type { DataView } from '@kbn/data-views-plugin/common';
import { loadFieldStats } from '@kbn/unified-field-list-plugin/public';
import { loadFieldStats } from '@kbn/unified-field-list-plugin/public/services/field_stats';
import { FieldStats } from '@kbn/unified-field-list-plugin/public';
import { DOCUMENT_FIELD_NAME } from '../../common';
import { LensFieldIcon } from '../shared_components';
@ -34,7 +35,9 @@ const chartsThemeService = chartPluginMock.createSetupContract().theme;
const clickField = async (wrapper: ReactWrapper, field: string) => {
await act(async () => {
wrapper.find(`[data-test-subj="lnsFieldListPanelField-${field}"] button`).simulate('click');
await wrapper
.find(`[data-test-subj="lnsFieldListPanelField-${field}"] button`)
.simulate('click');
});
};
@ -153,6 +156,11 @@ describe('IndexPattern Field Item', () => {
});
});
beforeEach(() => {
(loadFieldStats as jest.Mock).mockReset();
(loadFieldStats as jest.Mock).mockImplementation(() => Promise.resolve({}));
});
it('should display displayName of a field', () => {
const wrapper = mountWithIntl(<InnerFieldItemWrapper {...defaultProps} />);
@ -182,7 +190,7 @@ describe('IndexPattern Field Item', () => {
<InnerFieldItemWrapper {...defaultProps} editField={editFieldSpy} hideDetails />
);
await clickField(wrapper, 'bytes');
wrapper.update();
await wrapper.update();
const popoverContent = wrapper.find(EuiPopover).prop('children');
act(() => {
mountWithIntl(popoverContent as ReactElement)
@ -204,7 +212,7 @@ describe('IndexPattern Field Item', () => {
/>
);
await clickField(wrapper, documentField.name);
wrapper.update();
await wrapper.update();
const popoverContent = wrapper.find(EuiPopover).prop('children');
expect(
mountWithIntl(popoverContent as ReactElement)
@ -329,13 +337,10 @@ describe('IndexPattern Field Item', () => {
toDate: 'now-7d',
field: defaultProps.field,
});
(loadFieldStats as jest.Mock).mockReset();
(loadFieldStats as jest.Mock).mockImplementation(() => Promise.resolve({}));
});
it('should not request field stats for document field', async () => {
const wrapper = mountWithIntl(
const wrapper = await mountWithIntl(
<InnerFieldItemWrapper {...defaultProps} field={documentField} />
);
@ -343,13 +348,14 @@ describe('IndexPattern Field Item', () => {
await wrapper.update();
expect(loadFieldStats).not.toHaveBeenCalled();
expect(loadFieldStats).toHaveBeenCalled();
expect(wrapper.find(EuiPopover).prop('isOpen')).toEqual(true);
expect(wrapper.find(EuiLoadingSpinner)).toHaveLength(0);
expect(wrapper.find(FieldStats).text()).toBe('Analysis is not available for this field.');
});
it('should not request field stats for range fields', async () => {
const wrapper = mountWithIntl(
const wrapper = await mountWithIntl(
<InnerFieldItemWrapper
{...defaultProps}
field={{
@ -364,6 +370,11 @@ describe('IndexPattern Field Item', () => {
await clickField(wrapper, 'ip_range');
expect(loadFieldStats).not.toHaveBeenCalled();
await wrapper.update();
expect(loadFieldStats).toHaveBeenCalled();
expect(wrapper.find(EuiPopover).prop('isOpen')).toEqual(true);
expect(wrapper.find(EuiLoadingSpinner)).toHaveLength(0);
expect(wrapper.find(FieldStats).text()).toBe('Analysis is not available for this field.');
});
});

View file

@ -16,7 +16,6 @@ import {
EuiPopover,
EuiPopoverTitle,
EuiPopoverFooter,
EuiSpacer,
EuiText,
EuiTitle,
EuiToolTip,
@ -38,7 +37,6 @@ import type { IndexPattern, IndexPatternField } from '../types';
import type { DraggedField } from './types';
import { LensFieldIcon } from '../shared_components/field_picker/lens_field_icon';
import { VisualizeGeoFieldButton } from './visualize_geo_field_button';
import { getVisualizeGeoFieldMessage } from '../utils';
import type { LensAppServices } from '../app_plugin/types';
import { debouncedComponent } from '../debounced_component';
import { getFieldType } from './pure_utils';
@ -349,24 +347,25 @@ function FieldItemPopoverContents(props: FieldItemProps) {
dataViewOrDataViewId={indexPattern.id} // TODO: Refactor to pass a variable with DataView type instead of IndexPattern
field={field as DataViewField}
data-test-subj="lnsFieldListPanel"
overrideFooter={({ element }) => <EuiPopoverFooter>{element}</EuiPopoverFooter>}
overrideMissingContent={(params) => {
if (field.type === 'geo_point' || field.type === 'geo_shape') {
return (
<>
<EuiText size="s">{getVisualizeGeoFieldMessage(field.type)}</EuiText>
{params.element}
<EuiSpacer size="m" />
<VisualizeGeoFieldButton
uiActions={uiActions}
indexPattern={indexPattern}
fieldName={field.name}
/>
<EuiPopoverFooter>
<VisualizeGeoFieldButton
uiActions={uiActions}
indexPattern={indexPattern}
fieldName={field.name}
/>
</EuiPopoverFooter>
</>
);
}
if (params?.noDataFound) {
// TODO: should we replace this with a default message "Analysis is not available for this field?"
const isUsingSampling = core.uiSettings.get('lens:useFieldExistenceSampling');
return (
<>
@ -385,7 +384,7 @@ function FieldItemPopoverContents(props: FieldItemProps) {
);
}
return null;
return params.element;
}}
/>
</>

View file

@ -10,6 +10,7 @@ import { uniq } from 'lodash';
import type { CoreStart } from '@kbn/core/public';
import { buildEsQuery } from '@kbn/es-query';
import { getEsQueryConfig, DataPublicPluginStart } from '@kbn/data-plugin/public';
import type { DataViewField } from '@kbn/data-views-plugin/common';
import { FieldStatsResponse, loadFieldStats } from '@kbn/unified-field-list-plugin/public';
import { GenericIndexPatternColumn, operationDefinitionMap } from '..';
import { defaultLabel } from '../filters';
@ -142,7 +143,7 @@ export function getDisallowedTermsMessage(
const response: FieldStatsResponse<string | number> = await loadFieldStats({
services: { data },
dataView: currentDataView,
field: indexPattern.getFieldByName(fieldNames[0])!,
field: indexPattern.getFieldByName(fieldNames[0])! as DataViewField,
dslQuery: buildEsQuery(
indexPattern,
frame.query,

View file

@ -60,14 +60,15 @@ export function VisualizeGeoFieldButton(props: Props) {
<>
{/* eslint-disable-next-line @elastic/eui/href-or-on-click */}
<EuiButton
fullWidth
onClick={onClick}
href={href}
size="s"
data-test-subj={`lensVisualize-GeoField-${props.fieldName}`}
>
<FormattedMessage
id="xpack.lens.indexPattern.fieldItem.visualizeGeoFieldLinkText"
defaultMessage="Visualize in Maps"
id="xpack.lens.indexPattern.fieldItem.visualizeGeoFieldButtonText"
defaultMessage="Visualize"
/>
</EuiButton>
</>

View file

@ -17430,7 +17430,6 @@
"xpack.lens.indexPattern.existenceErrorLabel": "Impossible de charger les informations de champ",
"xpack.lens.indexPattern.existenceTimeoutAriaLabel": "La récupération de l'existence a expiré",
"xpack.lens.indexPattern.existenceTimeoutLabel": "Les informations de champ ont pris trop de temps",
"xpack.lens.indexPattern.fieldItem.visualizeGeoFieldLinkText": "Visualiser dans Maps",
"xpack.lens.indexPattern.fieldItemTooltip": "Effectuez un glisser-déposer pour visualiser.",
"xpack.lens.indexPattern.fieldPlaceholder": "Champ",
"xpack.lens.indexPattern.fieldStatsButtonEmptyLabel": "Ce champ ne comporte aucune donnée mais vous pouvez toujours effectuer un glisser-déposer pour visualiser.",

View file

@ -17415,7 +17415,6 @@
"xpack.lens.indexPattern.existenceErrorLabel": "フィールド情報を読み込めません",
"xpack.lens.indexPattern.existenceTimeoutAriaLabel": "存在の取り込みがタイムアウトしました",
"xpack.lens.indexPattern.existenceTimeoutLabel": "フィールド情報に時間がかかりすぎました",
"xpack.lens.indexPattern.fieldItem.visualizeGeoFieldLinkText": "Mapsで可視化",
"xpack.lens.indexPattern.fieldItemTooltip": "可視化するには、ドラッグアンドドロップします。",
"xpack.lens.indexPattern.fieldPlaceholder": "フィールド",
"xpack.lens.indexPattern.fieldStatsButtonEmptyLabel": "このフィールドにはデータがありませんが、ドラッグアンドドロップで可視化できます。",

View file

@ -17437,7 +17437,6 @@
"xpack.lens.indexPattern.existenceErrorLabel": "无法加载字段信息",
"xpack.lens.indexPattern.existenceTimeoutAriaLabel": "现有内容提取超时",
"xpack.lens.indexPattern.existenceTimeoutLabel": "字段信息花费时间过久",
"xpack.lens.indexPattern.fieldItem.visualizeGeoFieldLinkText": "在 Maps 中可视化",
"xpack.lens.indexPattern.fieldItemTooltip": "拖放以可视化。",
"xpack.lens.indexPattern.fieldPlaceholder": "字段",
"xpack.lens.indexPattern.fieldStatsButtonEmptyLabel": "此字段不包含任何数据,但您仍然可以拖放以进行可视化。",