github-mirrors/kibana
1
0
Fork 0
mirror of https://github.com/elastic/kibana.git synced 2025-04-23 09:19:04 -04:00
Code Issues Projects Releases Packages Wiki Activity

Add nested field support to KQL (#47070)

Browse source 
This PR adds a new syntax to KQL for querying nested fields.

Nested fields can be queried in two different ways:

Parts of the query may only match a single nested doc (bool inside nested). This is what most people want when querying on a nested field.
Parts of the query may match different nested docs (nested inside bool). This is how a regular object field works but nested fields can be queried in the same way. Although generally less useful, there are occasions where one might want to query a nested field in this way.
The new KQL syntax supports both.
This commit is contained in:
Matt Bargar 2019-10-30 13:07:43 -04:00 • committed by GitHub
parent 636fa27107
commit 1dcec9d210
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
66 changed files with 1953 additions and 475 deletions
Download patch file Download diff file Expand all files Collapse all files

84
docs/discover/kuery.asciidoc
View file

@ -73,3 +73,87 @@ set these terms will be matched against all fields. For example, a query for `re
in the response field, but a query for just `200` will search for 200 across all fields in your index.
============
===== Nested Field Support
KQL supports querying on {ref}/nested.html[nested fields] through a special syntax. You can query nested fields in subtly different
ways, depending on the results you want, so crafting nested queries requires extra thought.
One main consideration is how to match parts of the nested query to the individual nested documents.
There are two main approaches to take:
* *Parts of the query may only match a single nested document.* This is what most users want when querying on a nested field.
* *Parts of the query can match different nested documents.* This is how a regular object field works.
Although generally less useful, there might be occasions where you want to query a nested field in this way.
Let's take a look at the first approach. In the following document, `items` is a nested field:
[source,json]
----------------------------------
{
"grocery_name": "Elastic Eats",
"items": [
{
"name": "banana",
"stock": "12",
"category": "fruit"
},
{
"name": "peach",
"stock": "10",
"category": "fruit"
},
{
"name": "carrot",
"stock": "9",
"category": "vegetable"
},
{
"name": "broccoli",
"stock": "5",
"category": "vegetable"
}
]
}
----------------------------------
To find stores that have more than 10 bananas in stock, you would write a query like this:
`items:{ name:banana and stock > 10 }`
`items` is the "nested path". Everything inside the curly braces (the "nested group") must match a single document.
For example, `items:{ name:banana and stock:9 }` does not match because there isn't a single nested document that
matches the entire query in the nested group.
What if you want to find a store with more than 10 bananas that *also* stocks vegetables? This is the second way of querying a nested field, and you can do it like this:
`items:{ name:banana and stock > 10 } and items:{ category:vegetable }`
The first nested group (`name:banana and stock > 10`) must still match a single document, but the `category:vegetables`
subquery can match a different nested document because it is in a separate group.
KQL's syntax also supports nested fields inside of other nested fields—you simply have to specify the full path. Suppose you
have a document where `level1` and `level2` are both nested fields:
[source,json]
----------------------------------
{
"level1": [
{
"level2": [
{
"prop1": "foo",
"prop2": "bar"
},
{
"prop1": "baz",
"prop2": "qux"
}
]
}
]
}
----------------------------------
You can match on a single nested document by specifying the full path:
`level1.level2:{ prop1:foo and prop2:bar }`

25
packages/kbn-es-query/src/__fixtures__/index_pattern_response.json
View file

@ -161,8 +161,7 @@
"searchable": true,
"aggregatable": true,
"readFromDocValues": true,
"parent": "machine.os",
"subType": "multi"
"subType": { "multi": { "parent": "machine.os" } }
},
{
"name": "geo.src",
@ -277,6 +276,28 @@
"searchable": true,
"aggregatable": true,
"readFromDocValues": false
},
{
"name": "nestedField.child",
"type": "string",
"esTypes": ["text"],
"count": 0,
"scripted": false,
"searchable": true,
"aggregatable": false,
"readFromDocValues": false,
"subType": { "nested": { "path": "nestedField" } }
},
{
"name": "nestedField.nestedChild.doublyNestedChild",
"type": "string",
"esTypes": ["text"],
"count": 0,
"scripted": false,
"searchable": true,
"aggregatable": false,
"readFromDocValues": false,
"subType": { "nested": { "path": "nestedField.nestedChild" } }
}
]
}

62
packages/kbn-es-query/src/kuery/ast/__tests__/ast.js
View file

@ -198,6 +198,68 @@ describe('kuery AST API', function () {
expect(actual).to.eql(expected);
});
it('should support nested queries indicated by curly braces', () => {
const expected = nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('is', 'childOfNested', 'foo')
);
const actual = ast.fromKueryExpression('nestedField:{ childOfNested: foo }');
expect(actual).to.eql(expected);
});
it('should support nested subqueries and subqueries inside nested queries', () => {
const expected = nodeTypes.function.buildNode(
'and',
[
nodeTypes.function.buildNode('is', 'response', '200'),
nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('or', [
nodeTypes.function.buildNode('is', 'childOfNested', 'foo'),
nodeTypes.function.buildNode('is', 'childOfNested', 'bar'),
])
)]);
const actual = ast.fromKueryExpression('response:200 and nestedField:{ childOfNested:foo or childOfNested:bar }');
expect(actual).to.eql(expected);
});
it('should support nested sub-queries inside paren groups', () => {
const expected = nodeTypes.function.buildNode(
'and',
[
nodeTypes.function.buildNode('is', 'response', '200'),
nodeTypes.function.buildNode('or', [
nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('is', 'childOfNested', 'foo')
),
nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode('is', 'childOfNested', 'bar')
),
])
]);
const actual = ast.fromKueryExpression('response:200 and ( nestedField:{ childOfNested:foo } or nestedField:{ childOfNested:bar } )');
expect(actual).to.eql(expected);
});
it('should support nested groups inside other nested groups', () => {
const expected = nodeTypes.function.buildNode(
'nested',
'nestedField',
nodeTypes.function.buildNode(
'nested',
'nestedChild',
nodeTypes.function.buildNode('is', 'doublyNestedChild', 'foo')
)
);
const actual = ast.fromKueryExpression('nestedField:{ nestedChild:{ doublyNestedChild:foo } }');
expect(actual).to.eql(expected);
});
});
describe('fromLiteralExpression', function () {

4
packages/kbn-es-query/src/kuery/ast/ast.js
View file

@ -63,12 +63,12 @@ function fromExpression(expression, parseOptions = {}, parse = parseKuery) {
* IndexPattern isn't required, but if you pass one in, we can be more intelligent
* about how we craft the queries (e.g. scripted fields)
*/
export function toElasticsearchQuery(node, indexPattern, config = {}) {
export function toElasticsearchQuery(node, indexPattern, config = {}, context = {}) {
if (!node || !node.type || !nodeTypes[node.type]) {
return toElasticsearchQuery(nodeTypes.function.buildNode('and', []));
}
return nodeTypes[node.type].toElasticsearchQuery(node, indexPattern, config);
return nodeTypes[node.type].toElasticsearchQuery(node, indexPattern, config, context);
}
export function doesKueryExpressionHaveLuceneSyntaxError(expression) {

775
packages/kbn-es-query/src/kuery/ast/kuery.js
View file

File diff suppressed because it is too large Load diff

23
packages/kbn-es-query/src/kuery/ast/kuery.peg
View file

@ -59,7 +59,26 @@ SubQuery
}
return query;
}
/ Expression
/ NestedQuery
NestedQuery
= field:Field Space* ':' Space* '{' Space* query:OrQuery trailing:OptionalSpace '}' {
if (query.type === 'cursor') {
return {
...query,
nestedPath: query.nestedPath ? `${field.value}.${query.nestedPath}` : field.value,
}
};
if (trailing.type === 'cursor') {
return {
...trailing,
suggestionTypes: ['conjunction']
};
}
return buildFunctionNode('nested', [field, query]);
}
/ Expression
Expression
= FieldRangeExpression
@ -272,7 +291,7 @@ Keyword
= Or / And / Not
SpecialCharacter
= [\\():<>"*]
= [\\():<>"*{}]
RangeOperator
= '<=' { return 'lte'; }

4
packages/kbn-es-query/src/kuery/errors/index.test.js
View file

@ -25,7 +25,7 @@ describe('kql syntax errors', () => {
it('should throw an error for a field query missing a value', () => {
expect(() => {
fromKueryExpression('response:');
}).toThrow('Expected "(", value, whitespace but end of input found.\n' +
}).toThrow('Expected "(", "{", value, whitespace but end of input found.\n' +
'response:\n' +
'---------^');
});
@ -65,7 +65,7 @@ describe('kql syntax errors', () => {
it('should throw an error for unbalanced quotes', () => {
expect(() => {
fromKueryExpression('foo:"ba ');
}).toThrow('Expected "(", value, whitespace but "\"" found.\n' +
}).toThrow('Expected "(", "{", value, whitespace but """ found.\n' +
'foo:"ba \n' +
'----^');
});

15
packages/kbn-es-query/src/kuery/functions/__tests__/exists.js
View file

@ -72,6 +72,21 @@ describe('kuery functions', function () {
expect(exists.toElasticsearchQuery)
.withArgs(existsNode, indexPattern).to.throwException(/Exists query does not support scripted fields/);
});
it('should use a provided nested context to create a full field name', function () {
const expected = {
exists: { field: 'nestedField.response' }
};
const existsNode = nodeTypes.function.buildNode('exists', 'response');
const result = exists.toElasticsearchQuery(
existsNode,
indexPattern,
{},
{ nested: { path: 'nestedField' } }
);
expect(_.isEqual(expected, result)).to.be(true);
});
});
});
});

13
packages/kbn-es-query/src/kuery/functions/__tests__/geo_bounding_box.js
View file

@ -102,6 +102,19 @@ describe('kuery functions', function () {
expect(geoBoundingBox.toElasticsearchQuery)
.withArgs(node, indexPattern).to.throwException(/Geo bounding box query does not support scripted fields/);
});
it('should use a provided nested context to create a full field name', function () {
const node = nodeTypes.function.buildNode('geoBoundingBox', 'geo', params);
const result = geoBoundingBox.toElasticsearchQuery(
node,
indexPattern,
{},
{ nested: { path: 'nestedField' } }
);
expect(result).to.have.property('geo_bounding_box');
expect(result.geo_bounding_box).to.have.property('nestedField.geo');
});
});
});
});

12
packages/kbn-es-query/src/kuery/functions/__tests__/geo_polygon.js
View file

@ -114,6 +114,18 @@ describe('kuery functions', function () {
expect(geoPolygon.toElasticsearchQuery)
.withArgs(node, indexPattern).to.throwException(/Geo polygon query does not support scripted fields/);
});
it('should use a provided nested context to create a full field name', function () {
const node = nodeTypes.function.buildNode('geoPolygon', 'geo', points);
const result = geoPolygon.toElasticsearchQuery(
node,
indexPattern,
{},
{ nested: { path: 'nestedField' } }
);
expect(result).to.have.property('geo_polygon');
expect(result.geo_polygon).to.have.property('nestedField.geo');
});
});
});
});

60
packages/kbn-es-query/src/kuery/functions/__tests__/is.js
View file

@ -245,6 +245,66 @@ describe('kuery functions', function () {
expect(result).to.eql(expected);
});
it('should use a provided nested context to create a full field name', function () {
const expected = {
bool: {
should: [
{ match: { 'nestedField.extension': 'jpg' } },
],
minimum_should_match: 1
}
};
const node = nodeTypes.function.buildNode('is', 'extension', 'jpg');
const result = is.toElasticsearchQuery(
node,
indexPattern,
{},
{ nested: { path: 'nestedField' } }
);
expect(result).to.eql(expected);
});
it('should support wildcard field names', function () {
const expected = {
bool: {
should: [
{ match: { extension: 'jpg' } },
],
minimum_should_match: 1
}
};
const node = nodeTypes.function.buildNode('is', 'ext*', 'jpg');
const result = is.toElasticsearchQuery(node, indexPattern);
expect(result).to.eql(expected);
});
it('should automatically add a nested query when a wildcard field name covers a nested field', () => {
const expected = {
bool: {
should: [
{
nested: {
path: 'nestedField.nestedChild',
query: {
match: {
'nestedField.nestedChild.doublyNestedChild': 'foo'
}
},
score_mode: 'none'
}
}
],
minimum_should_match: 1
}
};
const node = nodeTypes.function.buildNode('is', '*doublyNested*', 'foo');
const result = is.toElasticsearchQuery(node, indexPattern);
expect(result).to.eql(expected);
});
});
});
});

68
packages/kbn-es-query/src/kuery/functions/__tests__/nested.js Normal file
View file

@ -0,0 +1,68 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import expect from '@kbn/expect';
import * as nested from '../nested';
import { nodeTypes } from '../../node_types';
import * as ast from '../../ast';
import indexPatternResponse from '../../../__fixtures__/index_pattern_response.json';
let indexPattern;
const childNode = nodeTypes.function.buildNode('is', 'child', 'foo');
describe('kuery functions', function () {
describe('nested', function () {
beforeEach(() => {
indexPattern = indexPatternResponse;
});
describe('buildNodeParams', function () {
it('arguments should contain the unmodified child nodes', function () {
const result = nested.buildNodeParams('nestedField', childNode);
const { arguments: [ resultPath, resultChildNode ] } = result;
expect(ast.toElasticsearchQuery(resultPath)).to.be('nestedField');
expect(resultChildNode).to.be(childNode);
});
});
describe('toElasticsearchQuery', function () {
it('should wrap subqueries in an ES nested query', function () {
const node = nodeTypes.function.buildNode('nested', 'nestedField', childNode);
const result = nested.toElasticsearchQuery(node, indexPattern);
expect(result).to.only.have.keys('nested');
expect(result.nested.path).to.be('nestedField');
expect(result.nested.score_mode).to.be('none');
});
it('should pass the nested path to subqueries so the full field name can be used', function () {
const node = nodeTypes.function.buildNode('nested', 'nestedField', childNode);
const result = nested.toElasticsearchQuery(node, indexPattern);
const expectedSubQuery = ast.toElasticsearchQuery(
nodeTypes.function.buildNode('is', 'nestedField.child', 'foo')
);
expect(result.nested.query).to.eql(expectedSubQuery);
});
});
});
});

54
packages/kbn-es-query/src/kuery/functions/__tests__/range.js
View file

@ -181,6 +181,60 @@ describe('kuery functions', function () {
expect(result).to.eql(expected);
});
it('should use a provided nested context to create a full field name', function () {
const expected = {
bool: {
should: [
{
range: {
'nestedField.bytes': {
gt: 1000,
lt: 8000
}
}
}
],
minimum_should_match: 1
}
};
const node = nodeTypes.function.buildNode('range', 'bytes', { gt: 1000, lt: 8000 });
const result = range.toElasticsearchQuery(
node,
indexPattern,
{},
{ nested: { path: 'nestedField' } }
);
expect(result).to.eql(expected);
});
it('should automatically add a nested query when a wildcard field name covers a nested field', function () {
const expected = {
bool: {
should: [
{
nested: {
path: 'nestedField.nestedChild',
query: {
range: {
'nestedField.nestedChild.doublyNestedChild': {
gt: 1000,
lt: 8000
}
}
},
score_mode: 'none'
}
}
],
minimum_should_match: 1
}
};
const node = nodeTypes.function.buildNode('range', '*doublyNested*', { gt: 1000, lt: 8000 });
const result = range.toElasticsearchQuery(node, indexPattern);
expect(result).to.eql(expected);
});
});
});
});

88
packages/kbn-es-query/src/kuery/functions/__tests__/utils/get_full_field_name_node.js Normal file
View file

@ -0,0 +1,88 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import expect from '@kbn/expect';
import { nodeTypes } from '../../../node_types';
import indexPatternResponse from '../../../../__fixtures__/index_pattern_response.json';
import { getFullFieldNameNode } from '../../utils/get_full_field_name_node';
let indexPattern;
describe('getFullFieldNameNode', function () {
beforeEach(() => {
indexPattern = indexPatternResponse;
});
it('should return unchanged name node if no nested path is passed in', () => {
const nameNode = nodeTypes.literal.buildNode('notNested');
const result = getFullFieldNameNode(nameNode, indexPattern);
expect(result).to.eql(nameNode);
});
it('should add the nested path if it is valid according to the index pattern', () => {
const nameNode = nodeTypes.literal.buildNode('child');
const result = getFullFieldNameNode(nameNode, indexPattern, 'nestedField');
expect(result).to.eql(nodeTypes.literal.buildNode('nestedField.child'));
});
it('should throw an error if a path is provided for a non-nested field', () => {
const nameNode = nodeTypes.literal.buildNode('os');
expect(getFullFieldNameNode)
.withArgs(nameNode, indexPattern, 'machine')
.to
.throwException(/machine.os is not a nested field but is in nested group "machine" in the KQL expression/);
});
it('should throw an error if a nested field is not passed with a path', () => {
const nameNode = nodeTypes.literal.buildNode('nestedField.child');
expect(getFullFieldNameNode)
.withArgs(nameNode, indexPattern)
.to
.throwException(/nestedField.child is a nested field, but is not in a nested group in the KQL expression./);
});
it('should throw an error if a nested field is passed with the wrong path', () => {
const nameNode = nodeTypes.literal.buildNode('nestedChild.doublyNestedChild');
expect(getFullFieldNameNode)
.withArgs(nameNode, indexPattern, 'nestedField')
.to
// eslint-disable-next-line max-len
.throwException(/Nested field nestedField.nestedChild.doublyNestedChild is being queried with the incorrect nested path. The correct path is nestedField.nestedChild/);
});
it('should skip error checking for wildcard names', () => {
const nameNode = nodeTypes.wildcard.buildNode('nested*');
const result = getFullFieldNameNode(nameNode, indexPattern);
expect(result).to.eql(nameNode);
});
it('should skip error checking if no index pattern is passed in', () => {
const nameNode = nodeTypes.literal.buildNode('os');
expect(getFullFieldNameNode)
.withArgs(nameNode, null, 'machine')
.to
.not
.throwException();
const result = getFullFieldNameNode(nameNode, null, 'machine');
expect(result).to.eql(nodeTypes.literal.buildNode('machine.os'));
});
});

4
packages/kbn-es-query/src/kuery/functions/and.js
View file

@ -25,13 +25,13 @@ export function buildNodeParams(children) {
};
}
export function toElasticsearchQuery(node, indexPattern, config) {
export function toElasticsearchQuery(node, indexPattern, config, context) {
const children = node.arguments || [];
return {
bool: {
filter: children.map((child) => {
return ast.toElasticsearchQuery(child, indexPattern, config);
return ast.toElasticsearchQuery(child, indexPattern, config, context);
})
}
};

5
packages/kbn-es-query/src/kuery/functions/exists.js
View file

@ -26,9 +26,10 @@ export function buildNodeParams(fieldName) {
};
}
export function toElasticsearchQuery(node, indexPattern) {
export function toElasticsearchQuery(node, indexPattern = null, config, context = {}) {
const { arguments: [ fieldNameArg ] } = node;
const fieldName = literal.toElasticsearchQuery(fieldNameArg);
const fullFieldNameArg = { ...fieldNameArg, value: context.nested ? `${context.nested.path}.${fieldNameArg.value}` : fieldNameArg.value };
const fieldName = literal.toElasticsearchQuery(fullFieldNameArg);
const field = get(indexPattern, 'fields', []).find(field => field.name === fieldName);
if (field && field.scripted) {

6
packages/kbn-es-query/src/kuery/functions/geo_bounding_box.js
View file

@ -34,9 +34,10 @@ export function buildNodeParams(fieldName, params) {
};
}
export function toElasticsearchQuery(node, indexPattern) {
export function toElasticsearchQuery(node, indexPattern, config, context = {}) {
const [ fieldNameArg, ...args ] = node.arguments;
const fieldName = nodeTypes.literal.toElasticsearchQuery(fieldNameArg);
const fullFieldNameArg = { ...fieldNameArg, value: context.nested ? `${context.nested.path}.${fieldNameArg.value}` : fieldNameArg.value };
const fieldName = nodeTypes.literal.toElasticsearchQuery(fullFieldNameArg);
const field = _.get(indexPattern, 'fields', []).find(field => field.name === fieldName);
const queryParams = args.reduce((acc, arg) => {
const snakeArgName = _.snakeCase(arg.name);
@ -57,4 +58,3 @@ export function toElasticsearchQuery(node, indexPattern) {
},
};
}

7
packages/kbn-es-query/src/kuery/functions/geo_polygon.js
View file

@ -33,12 +33,13 @@ export function buildNodeParams(fieldName, points) {
};
}
export function toElasticsearchQuery(node, indexPattern) {
export function toElasticsearchQuery(node, indexPattern, config = {}, context = {}) {
const [ fieldNameArg, ...points ] = node.arguments;
const fieldName = nodeTypes.literal.toElasticsearchQuery(fieldNameArg);
const fullFieldNameArg = { ...fieldNameArg, value: context.nested ? `${context.nested.path}.${fieldNameArg.value}` : fieldNameArg.value };
const fieldName = nodeTypes.literal.toElasticsearchQuery(fullFieldNameArg);
const field = get(indexPattern, 'fields', []).find(field => field.name === fieldName);
const queryParams = {
points: points.map(ast.toElasticsearchQuery)
points: points.map((point) => { return ast.toElasticsearchQuery(point, indexPattern, config, context); })
};
if (field && field.scripted) {

2
packages/kbn-es-query/src/kuery/functions/index.js
View file

@ -25,6 +25,7 @@ import * as range from './range';
import * as exists from './exists';
import * as geoBoundingBox from './geo_bounding_box';
import * as geoPolygon from './geo_polygon';
import * as nested from './nested';
export const functions = {
is,
@ -35,4 +36,5 @@ export const functions = {
exists,
geoBoundingBox,
geoPolygon,
nested,
};

52
packages/kbn-es-query/src/kuery/functions/is.js
View file

@ -24,6 +24,7 @@ import * as wildcard from '../node_types/wildcard';
import { getPhraseScript } from '../../filters';
import { getFields } from './utils/get_fields';
import { getTimeZoneFromSettings } from '../../utils/get_time_zone_from_settings';
import { getFullFieldNameNode } from './utils/get_full_field_name_node';
export function buildNodeParams(fieldName, value, isPhrase = false) {
if (_.isUndefined(fieldName)) {
@ -40,12 +41,13 @@ export function buildNodeParams(fieldName, value, isPhrase = false) {
};
}
export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
export function toElasticsearchQuery(node, indexPattern = null, config = {}, context = {}) {
const { arguments: [fieldNameArg, valueArg, isPhraseArg] } = node;
const fieldName = ast.toElasticsearchQuery(fieldNameArg);
const fullFieldNameArg = getFullFieldNameNode(fieldNameArg, indexPattern, context.nested ? context.nested.path : undefined);
const fieldName = ast.toElasticsearchQuery(fullFieldNameArg);
const value = !_.isUndefined(valueArg) ? ast.toElasticsearchQuery(valueArg) : valueArg;
const type = isPhraseArg.value ? 'phrase' : 'best_fields';
if (fieldNameArg.value === null) {
if (fullFieldNameArg.value === null) {
if (valueArg.type === 'wildcard') {
return {
query_string: {
@ -63,7 +65,7 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
};
}
const fields = indexPattern ? getFields(fieldNameArg, indexPattern) : [];
const fields = indexPattern ? getFields(fullFieldNameArg, indexPattern) : [];
// If no fields are found in the index pattern we send through the given field name as-is. We do this to preserve
// the behaviour of lucene on dashboards where there are panels based on different index patterns that have different
// fields. If a user queries on a field that exists in one pattern but not the other, the index pattern without the
@ -71,14 +73,14 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
// keep things familiar for now.
if (fields && fields.length === 0) {
fields.push({
name: ast.toElasticsearchQuery(fieldNameArg),
name: ast.toElasticsearchQuery(fullFieldNameArg),
scripted: false,
});
}
const isExistsQuery = valueArg.type === 'wildcard' && value === '*';
const isAllFieldsQuery =
(fieldNameArg.type === 'wildcard' && fieldName === '*')
(fullFieldNameArg.type === 'wildcard' && fieldName === '*')
|| (fields && indexPattern && fields.length === indexPattern.fields.length);
const isMatchAllQuery = isExistsQuery && isAllFieldsQuery;
@ -87,6 +89,27 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
}
const queries = fields.reduce((accumulator, field) => {
const wrapWithNestedQuery = (query) => {
// Wildcards can easily include nested and non-nested fields. There isn't a good way to let
// users handle this themselves so we automatically add nested queries in this scenario.
if (
!(fullFieldNameArg.type === 'wildcard')
|| !_.get(field, 'subType.nested')
|| context.nested
) {
return query;
}
else {
return {
nested: {
path: field.subType.nested.path,
query,
score_mode: 'none'
}
};
}
};
if (field.scripted) {
// Exists queries don't make sense for scripted fields
if (!isExistsQuery) {
@ -98,19 +121,19 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
}
}
else if (isExistsQuery) {
return [...accumulator, {
return [...accumulator, wrapWithNestedQuery({
exists: {
field: field.name
}
}];
})];
}
else if (valueArg.type === 'wildcard') {
return [...accumulator, {
return [...accumulator, wrapWithNestedQuery({
query_string: {
fields: [field.name],
query: wildcard.toQueryStringQuery(valueArg),
}
}];
})];
}
/*
If we detect that it's a date field and the user wants an exact date, we need to convert the query to both >= and <= the value provided to force a range query. This is because match and match_phrase queries do not accept a timezone parameter.
@ -118,7 +141,7 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
*/
else if (field.type === 'date') {
const timeZoneParam = config.dateFormatTZ ? { time_zone: getTimeZoneFromSettings(config.dateFormatTZ) } : {};
return [...accumulator, {
return [...accumulator, wrapWithNestedQuery({
range: {
[field.name]: {
gte: value,
@ -126,15 +149,15 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
...timeZoneParam,
},
}
}];
})];
}
else {
const queryType = type === 'phrase' ? 'match_phrase' : 'match';
return [...accumulator, {
return [...accumulator, wrapWithNestedQuery({
[queryType]: {
[field.name]: value
}
}];
})];
}
}, []);
@ -146,4 +169,3 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
};
}

46
packages/kbn-es-query/src/kuery/functions/nested.js Normal file
View file

@ -0,0 +1,46 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import * as ast from '../ast';
import * as literal from '../node_types/literal';
export function buildNodeParams(path, child) {
const pathNode = typeof path === 'string' ? ast.fromLiteralExpression(path) : literal.buildNode(path);
return {
arguments: [pathNode, child],
};
}
export function toElasticsearchQuery(node, indexPattern, config, context = {}) {
if (!indexPattern) {
throw new Error('Cannot use nested queries without an index pattern');
}
const [path, child] = node.arguments;
const stringPath = ast.toElasticsearchQuery(path);
const fullPath = context.nested && context.nested.path ? `${context.nested.path}.${stringPath}` : stringPath;
return {
nested: {
path: fullPath,
query: ast.toElasticsearchQuery(child, indexPattern, config, { ...context, nested: { path: fullPath } }),
score_mode: 'none',
},
};
}

5
packages/kbn-es-query/src/kuery/functions/not.js
View file

@ -25,13 +25,12 @@ export function buildNodeParams(child) {
};
}
export function toElasticsearchQuery(node, indexPattern, config) {
export function toElasticsearchQuery(node, indexPattern, config, context) {
const [ argument ] = node.arguments;
return {
bool: {
must_not: ast.toElasticsearchQuery(argument, indexPattern, config)
must_not: ast.toElasticsearchQuery(argument, indexPattern, config, context)
}
};
}

4
packages/kbn-es-query/src/kuery/functions/or.js
View file

@ -25,13 +25,13 @@ export function buildNodeParams(children) {
};
}
export function toElasticsearchQuery(node, indexPattern, config) {
export function toElasticsearchQuery(node, indexPattern, config, context) {
const children = node.arguments || [];
return {
bool: {
should: children.map((child) => {
return ast.toElasticsearchQuery(child, indexPattern, config);
return ast.toElasticsearchQuery(child, indexPattern, config, context);
}),
minimum_should_match: 1,
},

37
packages/kbn-es-query/src/kuery/functions/range.js
View file

@ -23,6 +23,7 @@ import * as ast from '../ast';
import { getRangeScript } from '../../filters';
import { getFields } from './utils/get_fields';
import { getTimeZoneFromSettings } from '../../utils/get_time_zone_from_settings';
import { getFullFieldNameNode } from './utils/get_full_field_name_node';
export function buildNodeParams(fieldName, params) {
params = _.pick(params, 'gt', 'lt', 'gte', 'lte', 'format');
@ -36,9 +37,10 @@ export function buildNodeParams(fieldName, params) {
};
}
export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
export function toElasticsearchQuery(node, indexPattern = null, config = {}, context = {}) {
const [ fieldNameArg, ...args ] = node.arguments;
const fields = indexPattern ? getFields(fieldNameArg, indexPattern) : [];
const fullFieldNameArg = getFullFieldNameNode(fieldNameArg, indexPattern, context.nested ? context.nested.path : undefined);
const fields = indexPattern ? getFields(fullFieldNameArg, indexPattern) : [];
const namedArgs = extractArguments(args);
const queryParams = _.mapValues(namedArgs, ast.toElasticsearchQuery);
@ -49,13 +51,34 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
// keep things familiar for now.
if (fields && fields.length === 0) {
fields.push({
name: ast.toElasticsearchQuery(fieldNameArg),
name: ast.toElasticsearchQuery(fullFieldNameArg),
scripted: false,
});
}
const queries = fields.map((field) => {
const wrapWithNestedQuery = (query) => {
// Wildcards can easily include nested and non-nested fields. There isn't a good way to let
// users handle this themselves so we automatically add nested queries in this scenario.
if (
!fullFieldNameArg.type === 'wildcard'
|| !_.get(field, 'subType.nested')
|| context.nested
) {
return query;
}
else {
return {
nested: {
path: field.subType.nested.path,
query,
score_mode: 'none'
}
};
}
};
if (field.scripted) {
return {
script: getRangeScript(field, queryParams),
@ -63,20 +86,20 @@ export function toElasticsearchQuery(node, indexPattern = null, config = {}) {
}
else if (field.type === 'date') {
const timeZoneParam = config.dateFormatTZ ? { time_zone: getTimeZoneFromSettings(config.dateFormatTZ) } : {};
return {
return wrapWithNestedQuery({
range: {
[field.name]: {
...queryParams,
...timeZoneParam,
}
}
};
});
}
return {
return wrapWithNestedQuery({
range: {
[field.name]: queryParams
}
};
});
});
return {

63
packages/kbn-es-query/src/kuery/functions/utils/get_full_field_name_node.js Normal file
View file

@ -0,0 +1,63 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import { getFields } from './get_fields';
export function getFullFieldNameNode(rootNameNode, indexPattern, nestedPath) {
const fullFieldNameNode = {
...rootNameNode,
value: nestedPath ? `${nestedPath}.${rootNameNode.value}` : rootNameNode.value
};
// Wildcards can easily include nested and non-nested fields. There isn't a good way to let
// users handle this themselves so we automatically add nested queries in this scenario and skip the
// error checking below.
if (!indexPattern || (fullFieldNameNode.type === 'wildcard' && !nestedPath)) {
return fullFieldNameNode;
}
const fields = getFields(fullFieldNameNode, indexPattern);
const errors = fields.reduce((acc, field) => {
const nestedPathFromField = field.subType && field.subType.nested ? field.subType.nested.path : undefined;
if (nestedPath && !nestedPathFromField) {
return [...acc, `${field.name} is not a nested field but is in nested group "${nestedPath}" in the KQL expression.`];
}
if (nestedPathFromField && !nestedPath) {
return [...acc, `${field.name} is a nested field, but is not in a nested group in the KQL expression.`];
}
if (nestedPathFromField !== nestedPath) {
return [
...acc,
`Nested field ${field.name} is being queried with the incorrect nested path. The correct path is ${field.subType.nested.path}.`
];
}
return acc;
}, []);
if (errors.length > 0) {
throw new Error(errors.join('\n'));
}
return fullFieldNameNode;
}

5
packages/kbn-es-query/src/kuery/node_types/function.js
View file

@ -47,8 +47,7 @@ export function buildNodeWithArgumentNodes(functionName, argumentNodes) {
};
}
export function toElasticsearchQuery(node, indexPattern, config = {}) {
export function toElasticsearchQuery(node, indexPattern, config = {}, context = {}) {
const kueryFunction = functions[node.function];
return kueryFunction.toElasticsearchQuery(node, indexPattern, config);
return kueryFunction.toElasticsearchQuery(node, indexPattern, config, context);
}

3
packages/kbn-es-query/src/kuery/node_types/index.d.ts vendored
View file

@ -31,7 +31,8 @@ type FunctionName =
| 'range'
| 'exists'
| 'geoBoundingBox'
| 'geoPolygon';
| 'geoPolygon'
| 'nested';
interface FunctionTypeBuildNode {
type: 'function';

2
src/core/server/saved_objects/service/lib/repository.test.js
View file

@ -1377,7 +1377,7 @@ describe('SavedObjectsRepository', () => {
};
await expect(savedObjectsRepository.find(findOpts)).rejects.toMatchInlineSnapshot(`
[Error: KQLSyntaxError: Expected "(", value, whitespace but "<" found.
[Error: KQLSyntaxError: Expected "(", "{", value, whitespace but "<" found.
dashboard.attributes.otherField:<
--------------------------------^: Bad Request]
`);

8
src/fixtures/logstash_fields.js
View file

@ -25,7 +25,7 @@ function stubbedLogstashFields() {
return [
// |aggregatable
// | |searchable
// name esType | | |metadata | parent | subType
// name esType | | |metadata | subType
['bytes', 'long', true, true, { count: 10 } ],
['ssl', 'boolean', true, true, { count: 20 } ],
['@timestamp', 'date', true, true, { count: 30 } ],
@ -40,9 +40,9 @@ function stubbedLogstashFields() {
['hashed', 'murmur3', false, true ],
['geo.coordinates', 'geo_point', true, true ],
['extension', 'text', true, true],
['extension.keyword', 'keyword', true, true, {}, 'extension', 'multi' ],
['extension.keyword', 'keyword', true, true, {}, { multi: { parent: 'extension' } } ],
['machine.os', 'text', true, true ],
['machine.os.raw', 'keyword', true, true, {}, 'machine.os', 'multi' ],
['machine.os.raw', 'keyword', true, true, {}, { multi: { parent: 'machine.os' } } ],
['geo.src', 'keyword', true, true ],
['_id', '_id', true, true ],
['_type', '_type', true, true ],
@ -61,7 +61,6 @@ function stubbedLogstashFields() {
aggregatable,
searchable,
metadata = {},
parent = undefined,
subType = undefined,
] = row;
@ -87,7 +86,6 @@ function stubbedLogstashFields() {
script,
lang,
scripted,
parent,
subType,
};
});

8
src/legacy/core_plugins/dashboard_embeddable_container/public/np_ready/public/lib/types.ts
View file

@ -61,6 +61,11 @@ export interface SavedObjectMetaData<T extends SavedObjectAttributes> {
showSavedObject?(savedObject: SimpleSavedObject<T>): boolean;
}
interface FieldSubType {
multi?: { parent: string };
nested?: { path: string };
}
export interface Field {
name: string;
type: string;
@ -70,6 +75,5 @@ export interface Field {
aggregatable: boolean;
filterable: boolean;
searchable: boolean;
parent?: string;
subType?: string;
subType?: FieldSubType;
}

12
src/legacy/core_plugins/data/public/index_patterns/fields/field.ts
View file

@ -30,6 +30,11 @@ import { getNotifications } from '../services';
import { getKbnFieldType } from '../../../../../../plugins/data/public';
interface FieldSubType {
multi?: { parent: string };
nested?: { path: string };
}
export type FieldSpec = Record<string, any>;
export interface FieldType {
name: string;
@ -47,8 +52,7 @@ export interface FieldType {
visualizable?: boolean;
readFromDocValues?: boolean;
scripted?: boolean;
parent?: string;
subType?: string;
subType?: FieldSubType;
displayName?: string;
format?: any;
}
@ -68,8 +72,7 @@ export class Field implements FieldType {
sortable?: boolean;
visualizable?: boolean;
scripted?: boolean;
parent?: string;
subType?: string;
subType?: FieldSubType;
displayName?: string;
format: any;
routes: Record<string, string> = {
@ -165,7 +168,6 @@ export class Field implements FieldType {
obj.writ('conflictDescriptions');
// multi info
obj.fact('parent');
obj.fact('subType');
return obj.create();

93
src/legacy/core_plugins/data/public/query/query_bar/components/query_bar_input.tsx
View file

@ -21,11 +21,20 @@ import { Component } from 'react';
import React from 'react';
import { i18n } from '@kbn/i18n';
import { EuiFieldText, EuiOutsideClickDetector, PopoverAnchorPosition } from '@elastic/eui';
import {
EuiFieldText,
EuiOutsideClickDetector,
PopoverAnchorPosition,
EuiFlexGroup,
EuiFlexItem,
EuiButton,
EuiLink,
} from '@elastic/eui';
import { InjectedIntl, injectI18n } from '@kbn/i18n/react';
import { InjectedIntl, injectI18n, FormattedMessage } from '@kbn/i18n/react';
import { debounce, compact, isEqual } from 'lodash';
import { documentationLinks } from 'ui/documentation_links';
import { Toast } from 'src/core/public';
import {
AutocompleteSuggestion,
AutocompleteSuggestionType,
@ -302,20 +311,13 @@ export class QueryBarInputUI extends Component<Props, State> {
}
};
private selectSuggestion = ({
type,
text,
start,
end,
}: {
type: AutocompleteSuggestionType;
text: string;
start: number;
end: number;
}) => {
private selectSuggestion = (suggestion: AutocompleteSuggestion) => {
if (!this.inputRef) {
return;
}
const { type, text, start, end, cursorIndex } = suggestion;
this.handleNestedFieldSyntaxNotification(suggestion);
const query = this.getQueryString();
const { selectionStart, selectionEnd } = this.inputRef;
@ -328,12 +330,75 @@ export class QueryBarInputUI extends Component<Props, State> {
this.onQueryStringChange(newQueryString);
this.setState({
selectionStart: start + (cursorIndex ? cursorIndex : text.length),
selectionEnd: start + (cursorIndex ? cursorIndex : text.length),
});
if (type === recentSearchType) {
this.setState({ isSuggestionsVisible: false, index: null });
this.onSubmit({ query: newQueryString, language: this.props.query.language });
}
};
private handleNestedFieldSyntaxNotification = (suggestion: AutocompleteSuggestion) => {
if (
'field' in suggestion &&
suggestion.field.subType &&
suggestion.field.subType.nested &&
!this.services.storage.get('kibana.KQLNestedQuerySyntaxInfoOptOut')
) {
const notifications = this.services.notifications;
const onKQLNestedQuerySyntaxInfoOptOut = (toast: Toast) => {
if (!this.services.storage) return;
this.services.storage.set('kibana.KQLNestedQuerySyntaxInfoOptOut', true);
notifications!.toasts.remove(toast);
};
if (notifications) {
const toast = notifications.toasts.add({
title: this.props.intl.formatMessage({
id: 'data.query.queryBar.KQLNestedQuerySyntaxInfoTitle',
defaultMessage: 'KQL nested query syntax',
}),
text: (
<div>
<p>
<FormattedMessage
id="data.query.queryBar.KQLNestedQuerySyntaxInfoText"
defaultMessage="It looks like you're querying on a nested field.
You can construct KQL syntax for nested queries in different ways, depending on the results you want.
Learn more in our {link}."
values={{
link: (
<EuiLink href={documentationLinks.query.kueryQuerySyntax} target="_blank">
<FormattedMessage
id="data.query.queryBar.KQLNestedQuerySyntaxInfoDocLinkText"
defaultMessage="docs"
/>
</EuiLink>
),
}}
/>
</p>
<EuiFlexGroup justifyContent="flexEnd" gutterSize="s">
<EuiFlexItem grow={false}>
<EuiButton size="s" onClick={() => onKQLNestedQuerySyntaxInfoOptOut(toast)}>
<FormattedMessage
id="data.query.queryBar.KQLNestedQuerySyntaxInfoOptOutText"
defaultMessage="Don't show again"
/>
</EuiButton>
</EuiFlexItem>
</EuiFlexGroup>
</div>
),
});
}
}
};
private increaseLimit = () => {
this.setState({
suggestionLimit: this.state.suggestionLimit + 50,

28
src/legacy/core_plugins/kibana/migrations/migrations.js
View file

@ -460,6 +460,31 @@ function migrateFiltersAggQueryStringQueries(doc) {
}
function migrateSubTypeAndParentFieldProperties(doc) {
if (!doc.attributes.fields) return doc;
const fieldsString = doc.attributes.fields;
const fields = JSON.parse(fieldsString);
const migratedFields = fields.map(field => {
if (field.subType === 'multi') {
return {
...omit(field, 'parent'),
subType: { multi: { parent: field.parent } }
};
}
return field;
});
return {
...doc,
attributes: {
...doc.attributes,
fields: JSON.stringify(migratedFields),
}
};
}
const executeMigrations720 = flow(
migratePercentileRankAggregation,
migrateDateHistogramAggregation
@ -490,6 +515,9 @@ export const migrations = {
doc.attributes.typeMeta = doc.attributes.typeMeta || undefined;
return doc;
},
'7.6.0': flow(
migrateSubTypeAndParentFieldProperties
)
},
visualization: {
/**

23
src/legacy/core_plugins/kibana/migrations/migrations.test.js
View file

@ -56,6 +56,29 @@ Object {
`);
});
});
describe('7.6.0', function () {
const migrate = doc => migrations['index-pattern']['7.6.0'](doc);
it('should remove the parent property and update the subType prop on every field that has them', () => {
const input = {
attributes: {
title: 'test',
// eslint-disable-next-line max-len
fields: '[{"name":"customer_name","type":"string","esTypes":["text"],"count":0,"scripted":false,"searchable":true,"aggregatable":false,"readFromDocValues":false},{"name":"customer_name.keyword","type":"string","esTypes":["keyword"],"count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true,"subType":"multi","parent":"customer_name"}]',
},
};
const expected = {
attributes: {
title: 'test',
// eslint-disable-next-line max-len
fields: '[{"name":"customer_name","type":"string","esTypes":["text"],"count":0,"scripted":false,"searchable":true,"aggregatable":false,"readFromDocValues":false},{"name":"customer_name.keyword","type":"string","esTypes":["keyword"],"count":0,"scripted":false,"searchable":true,"aggregatable":true,"readFromDocValues":true,"subType":{"multi":{"parent":"customer_name"}}}]',
},
};
expect(migrate(input)).toEqual(expected);
});
});
});
describe('visualization', () => {

36
src/legacy/core_plugins/kibana/server/routes/api/suggestions/register_value_suggestions.js
View file

@ -28,16 +28,28 @@ export function registerValueSuggestions(server) {
method: ['POST'],
handler: async function (req) {
const { index } = req.params;
const { field, query, boolFilter } = req.payload;
const { field: fieldName, query, boolFilter } = req.payload;
const { callWithRequest } = server.plugins.elasticsearch.getCluster('data');
const savedObjectsClient = req.getSavedObjectsClient();
const savedObjectsResponse = await savedObjectsClient.find(
{ type: 'index-pattern', fields: ['fields'], search: `"${index}"`, searchFields: ['title'] }
);
const indexPattern = savedObjectsResponse.total > 0 ? savedObjectsResponse.saved_objects[0] : null;
const fields = indexPattern ? JSON.parse(indexPattern.attributes.fields) : null;
const field = fields ? fields.find((field) => field.name === fieldName) : fieldName;
const body = getBody(
{ field, query, boolFilter },
autocompleteTerminateAfter,
autocompleteTimeout
);
try {
const response = await callWithRequest(req, 'search', { index, body });
const buckets = get(response, 'aggregations.suggestions.buckets') || [];
const buckets = get(response, 'aggregations.suggestions.buckets')
|| get(response, 'aggregations.nestedSuggestions.suggestions.buckets')
|| [];
const suggestions = map(buckets, 'key');
return suggestions;
} catch (error) {
@ -55,7 +67,7 @@ function getBody({ field, query, boolFilter = [] }, terminateAfter, timeout) {
// the amount of information that needs to be transmitted to the coordinating node
const shardSize = 10;
return {
const body = {
size: 0,
timeout: `${timeout}ms`,
terminate_after: terminateAfter,
@ -67,7 +79,7 @@ function getBody({ field, query, boolFilter = [] }, terminateAfter, timeout) {
aggs: {
suggestions: {
terms: {
field,
field: field.name || field,
include: `${getEscapedQuery(query)}.*`,
execution_hint: executionHint,
shard_size: shardSize,
@ -75,6 +87,22 @@ function getBody({ field, query, boolFilter = [] }, terminateAfter, timeout) {
},
},
};
if (field.subType && field.subType.nested) {
return {
...body,
aggs: {
nestedSuggestions: {
nested: {
path: field.subType.nested.path,
},
aggs: body.aggs,
},
}
};
}
return body;
}
function getEscapedQuery(query = '') {

8
src/legacy/server/index_patterns/service/index_patterns_service.ts
View file

@ -28,8 +28,12 @@ export interface FieldDescriptor {
searchable: boolean;
type: string;
esTypes: string[];
parent?: string;
subType?: string;
subType?: FieldSubType;
}
interface FieldSubType {
multi?: { parent: string };
nested?: { path: string };
}
export class IndexPatternsService {

21
src/legacy/server/index_patterns/service/lib/field_capabilities/__fixtures__/es_field_caps_response.json
View file

@ -221,6 +221,27 @@
"searchable": true,
"aggregatable": true
}
},
"nested_object_parent": {
"nested": {
"type": "nested",
"searchable": false,
"aggregatable": false
}
},
"nested_object_parent.child": {
"text": {
"type": "text",
"searchable": true,
"aggregatable": false
}
},
"nested_object_parent.child.keyword": {
"keyword": {
"type": "keyword",
"searchable": true,
"aggregatable": true
}
}
}
}

45
src/legacy/server/index_patterns/service/lib/field_capabilities/field_caps_response.test.js
View file

@ -37,10 +37,10 @@ describe('index_patterns/field_capabilities/field_caps_response', () => {
describe('conflicts', () => {
it('returns a field for each in response, no filtering', () => {
const fields = readFieldCapsResponse(esResponse);
expect(fields).toHaveLength(22);
expect(fields).toHaveLength(24);
});
it('includes only name, type, esTypes, searchable, aggregatable, readFromDocValues, and maybe conflictDescriptions, parent, ' +
it('includes only name, type, esTypes, searchable, aggregatable, readFromDocValues, and maybe conflictDescriptions, ' +
'and subType of each field', () => {
const responseClone = cloneDeep(esResponse);
// try to trick it into including an extra field
@ -48,7 +48,7 @@ describe('index_patterns/field_capabilities/field_caps_response', () => {
const fields = readFieldCapsResponse(responseClone);
fields.forEach(field => {
const fieldWithoutOptionalKeys = omit(field, 'conflictDescriptions', 'parent', 'subType');
const fieldWithoutOptionalKeys = omit(field, 'conflictDescriptions', 'subType');
expect(Object.keys(fieldWithoutOptionalKeys)).toEqual([
'name',
@ -65,8 +65,8 @@ describe('index_patterns/field_capabilities/field_caps_response', () => {
sandbox.spy(shouldReadFieldFromDocValuesNS, 'shouldReadFieldFromDocValues');
const fields = readFieldCapsResponse(esResponse);
const conflictCount = fields.filter(f => f.type === 'conflict').length;
// +1 is for the object field which gets filtered out of the final return value from readFieldCapsResponse
sinon.assert.callCount(shouldReadFieldFromDocValues, fields.length - conflictCount + 1);
// +2 is for the object and nested fields which get filtered out of the final return value from readFieldCapsResponse
sinon.assert.callCount(shouldReadFieldFromDocValues, fields.length - conflictCount + 2);
});
it('converts es types to kibana types', () => {
@ -132,20 +132,41 @@ describe('index_patterns/field_capabilities/field_caps_response', () => {
expect(mixSearchableOther.searchable).toBe(true);
});
it('returns multi fields with parent and subType keys describing the relationship', () => {
it('returns multi fields with a subType key describing the relationship', () => {
const fields = readFieldCapsResponse(esResponse);
const child = fields.find(f => f.name === 'multi_parent.child');
expect(child).toHaveProperty('parent', 'multi_parent');
expect(child).toHaveProperty('subType', 'multi');
expect(child).toHaveProperty('subType', { multi: { parent: 'multi_parent' } });
});
it('should not confuse object children for multi field children', () => {
it('returns nested sub-fields with a subType key describing the relationship', () => {
const fields = readFieldCapsResponse(esResponse);
const child = fields.find(f => f.name === 'nested_object_parent.child');
expect(child).toHaveProperty('subType', { nested: { path: 'nested_object_parent' } });
});
it('handles fields that are both nested and multi', () => {
const fields = readFieldCapsResponse(esResponse);
const child = fields.find(f => f.name === 'nested_object_parent.child.keyword');
expect(child).toHaveProperty(
'subType',
{
nested: { path: 'nested_object_parent' },
multi: { parent: 'nested_object_parent.child' }
});
});
it('does not include the field actually mapped as nested itself', () => {
const fields = readFieldCapsResponse(esResponse);
const child = fields.find(f => f.name === 'nested_object_parent');
expect(child).toBeUndefined();
});
it('should not confuse object children for multi or nested field children', () => {
// We detect multi fields by finding fields that have a dot in their name and then looking
// to see if their parents are *not* object or nested fields. In the future we may want to
// add parent and subType info for object and nested fields but for now we don't need it.
// to see if their parents are *not* object fields. In the future we may want to
// add subType info for object fields but for now we don't need it.
const fields = readFieldCapsResponse(esResponse);
const child = fields.find(f => f.name === 'object_parent.child');
expect(child).not.toHaveProperty('parent');
expect(child).not.toHaveProperty('subType');
});
});

36
src/legacy/server/index_patterns/service/lib/field_capabilities/field_caps_response.ts
View file

@ -151,18 +151,38 @@ export function readFieldCapsResponse(fieldCapsResponse: FieldCapsResponse): Fie
return field.name.includes('.');
});
// Discern which sub fields are multi fields. If the parent field is not an object or nested field
// the child must be a multi field.
// Determine the type of each sub field.
subFields.forEach(field => {
const parentFieldName = field.name
const parentFieldNames = field.name
.split('.')
.slice(0, -1)
.join('.');
const parentFieldCaps = kibanaFormattedCaps.find(caps => caps.name === parentFieldName);
.map((_, index, parentFieldNameParts) => {
return parentFieldNameParts.slice(0, index + 1).join('.');
});
const parentFieldCaps = parentFieldNames.map(parentFieldName => {
return kibanaFormattedCaps.find(caps => caps.name === parentFieldName);
});
const parentFieldCapsAscending = parentFieldCaps.reverse();
if (parentFieldCaps && !['object', 'nested'].includes(parentFieldCaps.type)) {
field.parent = parentFieldName;
field.subType = 'multi';
if (parentFieldCaps && parentFieldCaps.length > 0) {
let subType = {};
// If the parent field is not an object or nested field the child must be a multi field.
const firstParent = parentFieldCapsAscending[0];
if (firstParent && !['object', 'nested'].includes(firstParent.type)) {
subType = { ...subType, multi: { parent: firstParent.name } };
}
// We need to know if any parent field is nested
const nestedParentCaps = parentFieldCapsAscending.find(
parentCaps => parentCaps && parentCaps.type === 'nested'
);
if (nestedParentCaps) {
subType = { ...subType, nested: { path: nestedParentCaps.name } };
}
if (Object.keys(subType).length > 0) {
field.subType = subType;
}
}
});

2
src/legacy/server/sample_data/data_sets/ecommerce/saved_objects.js
View file

File diff suppressed because one or more lines are too long

2
src/legacy/server/sample_data/data_sets/logs/saved_objects.js
View file

File diff suppressed because one or more lines are too long

15
src/plugins/data/public/autocomplete_provider/types.ts
View file

@ -17,7 +17,7 @@
* under the License.
*/
import { StaticIndexPattern } from 'ui/index_patterns';
import { StaticIndexPattern, Field } from 'ui/index_patterns';
import { AutocompleteProviderRegister } from '.';
export type AutocompletePublicPluginSetup = Pick<
@ -50,11 +50,22 @@ export type AutocompleteSuggestionType =
| 'conjunction'
| 'recentSearch';
// A union type allows us to do easy type guards in the code. For example, if I want to ensure I'm
// working with a FieldAutocompleteSuggestion, I can just do `if ('field' in suggestion)` and the
// TypeScript compiler will narrow the type to the parts of the union that have a field prop.
/** @public **/
export interface AutocompleteSuggestion {
export type AutocompleteSuggestion = BasicAutocompleteSuggestion | FieldAutocompleteSuggestion;
interface BasicAutocompleteSuggestion {
description?: string;
end: number;
start: number;
text: string;
type: AutocompleteSuggestionType;
cursorIndex?: number;
}
export type FieldAutocompleteSuggestion = BasicAutocompleteSuggestion & {
type: 'field';
field: Field;
};

2
src/plugins/kibana_react/public/context/index.ts
View file

@ -25,4 +25,4 @@ export {
withKibana,
UseKibana,
} from './context';
export { KibanaReactContext, KibanaReactContextValue } from './types';
export { KibanaReactContext, KibanaReactContextValue, KibanaServices } from './types';

36
test/api_integration/apis/index_patterns/fields_for_wildcard_route/response.js
View file

@ -61,8 +61,7 @@ export default function ({ getService }) {
aggregatable: true,
name: 'baz.keyword',
readFromDocValues: true,
parent: 'baz',
subType: 'multi',
subType: { multi: { parent: 'baz' } }
},
{
type: 'number',
@ -72,6 +71,21 @@ export default function ({ getService }) {
name: 'foo',
readFromDocValues: true,
},
{
aggregatable: true,
esTypes: [
'keyword'
],
name: 'nestedField.child',
readFromDocValues: true,
searchable: true,
subType: {
nested: {
path: 'nestedField'
}
},
type: 'string',
},
],
})
.then(ensureFieldsAreSorted));
@ -124,8 +138,7 @@ export default function ({ getService }) {
aggregatable: true,
name: 'baz.keyword',
readFromDocValues: true,
parent: 'baz',
subType: 'multi',
subType: { multi: { parent: 'baz' } }
},
{
aggregatable: false,
@ -142,6 +155,21 @@ export default function ({ getService }) {
name: 'foo',
readFromDocValues: true,
},
{
aggregatable: true,
esTypes: [
'keyword'
],
name: 'nestedField.child',
readFromDocValues: true,
searchable: true,
subType: {
nested: {
path: 'nestedField'
}
},
type: 'string',
},
],
})
.then(ensureFieldsAreSorted));

20
test/api_integration/apis/suggestions/suggestions.js
View file

@ -22,8 +22,14 @@ export default function ({ getService }) {
const supertest = getService('supertest');
describe('Suggestions API', function () {
before(() => esArchiver.load('index_patterns/basic_index'));
after(() => esArchiver.unload('index_patterns/basic_index'));
before(async () => {
await esArchiver.load('index_patterns/basic_index');
await esArchiver.load('index_patterns/basic_kibana');
});
after(async () => {
await esArchiver.unload('index_patterns/basic_index');
await esArchiver.unload('index_patterns/basic_kibana');
});
it('should return 200 with special characters', () => (
supertest
@ -34,5 +40,15 @@ export default function ({ getService }) {
})
.expect(200)
));
it('should support nested fields', () => (
supertest
.post('/api/kibana/suggestions/values/basic_index')
.send({
field: 'nestedField.child',
query: 'nes'
})
.expect(200, ['nestedValue'])
));
});
}

BIN
test/api_integration/fixtures/es_archiver/index_patterns/basic_index/data.json.gz
View file

Binary file not shown.

12
test/api_integration/fixtures/es_archiver/index_patterns/basic_index/mappings.json
View file

@ -24,8 +24,18 @@
},
"foo": {
"type": "long"
},
"nestedField": {
"type": "nested",
"properties": {
"child": {
"type": "keyword"
}
}
}
}
}
}
}
}

15
test/api_integration/fixtures/es_archiver/index_patterns/basic_kibana/data.json Normal file
View file

@ -0,0 +1,15 @@
{
"type": "doc",
"value": {
"index": ".kibana",
"id": "index-pattern:91200a00-9efd-11e7-acb3-3dab96693fab",
"source": {
"type": "index-pattern",
"updated_at": "2017-09-21T18:49:16.270Z",
"index-pattern": {
"title": "basic_index",
"fields": "[{\"name\":\"bar\",\"type\":\"boolean\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"name\":\"baz\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"baz.keyword\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"name\":\"foo\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"name\":\"nestedField.child\",\"type\":\"string\",\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"nested\":{\"path\":\"nestedField\"}}}]"
}
}
}
}

253
test/api_integration/fixtures/es_archiver/index_patterns/basic_kibana/mappings.json Normal file
View file

@ -0,0 +1,253 @@
{
"type": "index",
"value": {
"index": ".kibana",
"settings": {
"index": {
"number_of_shards": "1",
"number_of_replicas": "1"
}
},
"mappings": {
"dynamic": "strict",
"properties": {
"config": {
"dynamic": "true",
"properties": {
"buildNum": {
"type": "keyword"
},
"defaultIndex": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
},
"dashboard": {
"properties": {
"description": {
"type": "text"
},
"hits": {
"type": "integer"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"optionsJSON": {
"type": "text"
},
"panelsJSON": {
"type": "text"
},
"refreshInterval": {
"properties": {
"display": {
"type": "keyword"
},
"pause": {
"type": "boolean"
},
"section": {
"type": "integer"
},
"value": {
"type": "integer"
}
}
},
"timeFrom": {
"type": "keyword"
},
"timeRestore": {
"type": "boolean"
},
"timeTo": {
"type": "keyword"
},
"title": {
"type": "text"
},
"uiStateJSON": {
"type": "text"
},
"version": {
"type": "integer"
}
}
},
"index-pattern": {
"properties": {
"fieldFormatMap": {
"type": "text"
},
"fields": {
"type": "text"
},
"intervalName": {
"type": "keyword"
},
"notExpandable": {
"type": "boolean"
},
"sourceFilters": {
"type": "text"
},
"timeFieldName": {
"type": "keyword"
},
"title": {
"type": "text"
}
}
},
"search": {
"properties": {
"columns": {
"type": "keyword"
},
"description": {
"type": "text"
},
"hits": {
"type": "integer"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"sort": {
"type": "keyword"
},
"title": {
"type": "text"
},
"version": {
"type": "integer"
}
}
},
"server": {
"properties": {
"uuid": {
"type": "keyword"
}
}
},
"timelion-sheet": {
"properties": {
"description": {
"type": "text"
},
"hits": {
"type": "integer"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"timelion_chart_height": {
"type": "integer"
},
"timelion_columns": {
"type": "integer"
},
"timelion_interval": {
"type": "keyword"
},
"timelion_other_interval": {
"type": "keyword"
},
"timelion_rows": {
"type": "integer"
},
"timelion_sheet": {
"type": "text"
},
"title": {
"type": "text"
},
"version": {
"type": "integer"
}
}
},
"namespace": {
"type": "keyword"
},
"type": {
"type": "keyword"
},
"updated_at": {
"type": "date"
},
"url": {
"properties": {
"accessCount": {
"type": "long"
},
"accessDate": {
"type": "date"
},
"createDate": {
"type": "date"
},
"url": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 2048
}
}
}
}
},
"visualization": {
"properties": {
"description": {
"type": "text"
},
"kibanaSavedObjectMeta": {
"properties": {
"searchSourceJSON": {
"type": "text"
}
}
},
"savedSearchId": {
"type": "keyword"
},
"title": {
"type": "text"
},
"uiStateJSON": {
"type": "text"
},
"version": {
"type": "integer"
},
"visState": {
"type": "text"
}
}
}
}
}
}
}

21
test/functional/apps/discover/_discover.js
View file

@ -26,6 +26,7 @@ export default function ({ getService, getPageObjects }) {
const browser = getService('browser');
const kibanaServer = getService('kibanaServer');
const filterBar = getService('filterBar');
const queryBar = getService('queryBar');
const PageObjects = getPageObjects(['common', 'discover', 'header', 'timePicker']);
const defaultSettings = {
defaultIndex: 'logstash-*',
@ -154,6 +155,26 @@ export default function ({ getService, getPageObjects }) {
});
});
describe('nested query', () => {
before(async () => {
log.debug('setAbsoluteRangeForAnotherQuery');
await PageObjects.timePicker.setAbsoluteRange(fromTime, toTime);
await PageObjects.discover.waitUntilSearchingHasFinished();
});
it('should support querying on nested fields', async function () {
await queryBar.setQuery('nestedField:{ child: nestedValue }');
await queryBar.submitQuery();
await retry.try(async function () {
expect(await PageObjects.discover.getHitCount()).to.be(
'1'
);
});
});
});
describe('filter editor', function () {
it('should add a phrases filter', async function () {
await filterBar.addFilter('extension.raw', 'is one of', 'jpg');

BIN
test/functional/fixtures/es_archiver/discover/data.json.gz
View file

Binary file not shown.

BIN
test/functional/fixtures/es_archiver/logstash_functional/data.json.gz
View file

Binary file not shown.

26
test/functional/fixtures/es_archiver/logstash_functional/mappings.json
View file

@ -153,6 +153,14 @@
}
}
},
"nestedField": {
"type": "nested",
"properties": {
"child": {
"type": "keyword"
}
}
},
"phpmemory": {
"type": "long"
},
@ -518,6 +526,14 @@
}
}
},
"nestedField": {
"type": "nested",
"properties": {
"child": {
"type": "keyword"
}
}
},
"phpmemory": {
"type": "long"
},
@ -883,6 +899,14 @@
}
}
},
"nestedField": {
"type": "nested",
"properties": {
"child": {
"type": "keyword"
}
}
},
"phpmemory": {
"type": "long"
},
@ -1091,4 +1115,4 @@
}
}
}
}
}

25
x-pack/legacy/plugins/kuery_autocomplete/public/autocomplete_providers/__fixtures__/index_pattern_response.json
View file

@ -161,8 +161,7 @@
"searchable": true,
"aggregatable": true,
"readFromDocValues": true,
"parent": "machine.os",
"subType": "multi"
"subType":{ "multi":{ "parent": "machine.os" } }
},
{
"name": "geo.src",
@ -277,6 +276,28 @@
"searchable": true,
"aggregatable": true,
"readFromDocValues": false
},
{
"name": "nestedField.child",
"type": "string",
"esTypes": ["text"],
"count": 0,
"scripted": false,
"searchable": true,
"aggregatable": false,
"readFromDocValues": false,
"subType": { "nested": { "path": "nestedField" } }
},
{
"name": "nestedField.nestedChild.doublyNestedChild",
"type": "string",
"esTypes": ["text"],
"count": 0,
"scripted": false,
"searchable": true,
"aggregatable": false,
"readFromDocValues": false,
"subType": { "nested": { "path": "nestedField.nestedChild" } }
}
]
}

56
x-pack/legacy/plugins/kuery_autocomplete/public/autocomplete_providers/__tests__/field.js
View file

@ -80,4 +80,60 @@ describe('Kuery field suggestions', function () {
expect(suggestion).to.have.property('description');
});
});
describe('nested fields', function () {
it('should automatically wrap nested fields in KQL\'s nested syntax', () => {
const prefix = 'ch';
const suffix = '';
const suggestions = getSuggestions({ prefix, suffix });
const suggestion = suggestions.find(({ field }) => field.name === 'nestedField.child');
expect(suggestion.text).to.be('nestedField:{ child }');
// For most suggestions the cursor can be placed at the end of the suggestion text, but
// for the nested field syntax we want to place the cursor inside the curly braces
expect(suggestion.cursorIndex).to.be(20);
});
it('should narrow suggestions to children of a nested path if provided', () => {
const prefix = 'ch';
const suffix = '';
const allSuggestions = getSuggestions({ prefix, suffix });
expect(allSuggestions.length).to.be.greaterThan(2);
const nestedSuggestions = getSuggestions({ prefix, suffix, nestedPath: 'nestedField' });
expect(nestedSuggestions).to.have.length(2);
});
it('should not wrap the suggestion in KQL\'s nested syntax if the correct nested path is already provided', () => {
const prefix = 'ch';
const suffix = '';
const suggestions = getSuggestions({ prefix, suffix, nestedPath: 'nestedField' });
const suggestion = suggestions.find(({ field }) => field.name === 'nestedField.child');
expect(suggestion.text).to.be('child ');
});
it('should handle fields nested multiple levels deep', () => {
const prefix = 'doubly';
const suffix = '';
const suggestionsWithNoPath = getSuggestions({ prefix, suffix });
expect(suggestionsWithNoPath).to.have.length(1);
const [ noPathSuggestion ] = suggestionsWithNoPath;
expect(noPathSuggestion.text).to.be('nestedField.nestedChild:{ doublyNestedChild }');
const suggestionsWithPartialPath = getSuggestions({ prefix, suffix, nestedPath: 'nestedField' });
expect(suggestionsWithPartialPath).to.have.length(1);
const [ partialPathSuggestion ] = suggestionsWithPartialPath;
expect(partialPathSuggestion.text).to.be('nestedChild:{ doublyNestedChild }');
const suggestionsWithFullPath = getSuggestions({ prefix, suffix, nestedPath: 'nestedField.nestedChild' });
expect(suggestionsWithFullPath).to.have.length(1);
const [ fullPathSuggestion ] = suggestionsWithFullPath;
expect(fullPathSuggestion.text).to.be('doublyNestedChild ');
});
});
});

5
x-pack/legacy/plugins/kuery_autocomplete/public/autocomplete_providers/__tests__/operator.js
View file

@ -56,4 +56,9 @@ describe('Kuery operator suggestions', function () {
expect(suggestion).to.have.property('description');
});
});
it('should handle nested paths', () => {
const suggestions = getSuggestions({ fieldName: 'child', nestedPath: 'nestedField' });
expect(suggestions.length).to.be.greaterThan(0);
});
});

37
x-pack/legacy/plugins/kuery_autocomplete/public/autocomplete_providers/field.js
View file

@ -29,15 +29,28 @@ export function getSuggestionsProvider({ indexPatterns }) {
const allFields = flatten(indexPatterns.map(indexPattern => {
return indexPattern.fields.filter(isFilterable);
}));
return function getFieldSuggestions({ start, end, prefix, suffix }) {
const search = `${prefix}${suffix}`.toLowerCase();
const fieldNames = allFields.map(field => field.name);
const matchingFieldNames = fieldNames.filter(name => name.toLowerCase().includes(search));
const sortedFieldNames = sortPrefixFirst(matchingFieldNames.sort(keywordComparator), search);
const suggestions = sortedFieldNames.map(fieldName => {
const text = `${escapeKuery(fieldName)} `;
const description = getDescription(fieldName);
return { type, text, description, start, end };
return function getFieldSuggestions({ start, end, prefix, suffix, nestedPath = '' }) {
const search = `${prefix}${suffix}`.trim().toLowerCase();
const matchingFields = allFields.filter(field => {
return (
!nestedPath
|| (nestedPath && (field.subType && field.subType.nested && field.subType.nested.path.includes(nestedPath)))
)
&& field.name.toLowerCase().includes(search) && field.name !== search;
});
const sortedFields = sortPrefixFirst(matchingFields.sort(keywordComparator), search, 'name');
const suggestions = sortedFields.map(field => {
const remainingPath = field.subType && field.subType.nested
? field.subType.nested.path.slice(nestedPath ? nestedPath.length + 1 : 0)
: '';
const text = field.subType && field.subType.nested && remainingPath.length > 0
? `${escapeKuery(remainingPath)}:{ ${escapeKuery(field.name.slice(field.subType.nested.path.length + 1))} }`
: `${escapeKuery(field.name.slice(nestedPath ? nestedPath.length + 1 : 0))} `;
const description = getDescription(field.name);
const cursorIndex = field.subType && field.subType.nested && remainingPath.length > 0
? text.length - 2
: text.length;
return { type, text, description, start, end, cursorIndex, field };
});
return suggestions;
};
@ -45,10 +58,10 @@ export function getSuggestionsProvider({ indexPatterns }) {
function keywordComparator(first, second) {
const extensions = ['raw', 'keyword'];
if (extensions.map(ext => `${first}.${ext}`).includes(second)) {
if (extensions.map(ext => `${first.name}.${ext}`).includes(second.name)) {
return 1;
} else if (extensions.map(ext => `${second}.${ext}`).includes(first)) {
} else if (extensions.map(ext => `${second.name}.${ext}`).includes(first.name)) {
return -1;
}
return first.localeCompare(second);
return first.name.localeCompare(second.name);
}

5
x-pack/legacy/plugins/kuery_autocomplete/public/autocomplete_providers/operator.js
View file

@ -123,8 +123,9 @@ export function getSuggestionsProvider({ indexPatterns }) {
const allFields = flatten(indexPatterns.map(indexPattern => {
return indexPattern.fields.slice();
}));
return function getOperatorSuggestions({ end, fieldName }) {
const fields = allFields.filter(field => field.name === fieldName);
return function getOperatorSuggestions({ end, fieldName, nestedPath }) {
const fullFieldName = nestedPath ? `${nestedPath}.${fieldName}` : fieldName;
const fields = allFields.filter(field => field.name === fullFieldName);
return flatten(fields.map(field => {
const matchingOperators = Object.keys(operators).filter(operator => {
const { fieldTypes } = operators[operator];

6
x-pack/legacy/plugins/kuery_autocomplete/public/autocomplete_providers/value.js
View file

@ -26,9 +26,11 @@ export function getSuggestionsProvider({ indexPatterns, boolFilter }) {
prefix,
suffix,
fieldName,
nestedPath,
}) {
const fields = allFields.filter(field => field.name === fieldName);
const query = `${prefix}${suffix}`;
const fullFieldName = nestedPath ? `${nestedPath}.${fieldName}` : fieldName;
const fields = allFields.filter(field => field.name === fullFieldName);
const query = `${prefix}${suffix}`.trim();
const { getSuggestions } = npStart.plugins.data;
const suggestionsByField = fields.map(field => {

17
x-pack/legacy/plugins/kuery_autocomplete/public/autocomplete_providers/value.test.js
View file

@ -19,7 +19,11 @@ jest.mock('ui/new_platform', () => ({
res = [true, false];
} else if (field.name === 'machine.os') {
res = ['Windo"ws', 'Mac\'', 'Linux'];
} else {
}
else if (field.name === 'nestedField.child') {
res = ['foo'];
}
else {
res = [];
}
return Promise.resolve(res);
@ -67,6 +71,17 @@ describe('Kuery value suggestions', function () {
expect(suggestions[0].end).toEqual(end);
});
test('should handle nested paths', async () => {
const suggestions = await getSuggestions({
fieldName: 'child',
nestedPath: 'nestedField',
prefix: '',
suffix: '',
});
expect(suggestions.length).toEqual(1);
expect(suggestions[0].text).toEqual('"foo" ');
});
describe('Boolean suggestions', function () {
test('should stringify boolean fields', async () => {
const fieldName = 'ssl';

2
x-pack/legacy/plugins/lens/server/routes/existing_fields.test.ts
View file

@ -10,7 +10,7 @@ describe('existingFields', () => {
function field(name: string, parent?: string) {
return {
name,
parent,
subType: parent ? { multi: { parent } } : undefined,
aggregatable: true,
esTypes: [],
readFromDocValues: true,

13
x-pack/legacy/plugins/lens/server/routes/existing_fields.ts
View file

@ -112,11 +112,14 @@ export function existingFields(
docs: Array<{ _source: Document }>,
fields: FieldDescriptor[]
): string[] {
const allFields = fields.map(field => ({
name: field.name,
parent: field.parent,
path: (field.parent || field.name).split('.'),
}));
const allFields = fields.map(field => {
const parent = field.subType && field.subType.multi && field.subType.multi.parent;
return {
name: field.name,
parent,
path: (parent || field.name).split('.'),
};
});
const missingFields = new Set(allFields);
for (const doc of docs) {

2
x-pack/legacy/plugins/maps/public/index_pattern_util.js
View file

@ -28,6 +28,6 @@ export function getTermsFields(fields) {
export function getSourceFields(fields) {
return fields.filter(field => {
// Multi fields are not stored in _source and only exist in index.
return field.subType !== 'multi';
return field.subType && field.subType.multi;
});
}

2
x-pack/test/functional/es_archives/maps/kibana/data.json
View file

File diff suppressed because one or more lines are too long

2
x-pack/test/functional/es_archives/security/flstest/kibana/data.json
View file

@ -6,7 +6,7 @@
"source": {
"index-pattern" : {
"title" : "flstest",
"fields" : "[{\"name\":\"_id\",\"type\":\"string\",\"esTypes\":[\"_id\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_index\",\"type\":\"string\",\"esTypes\":[\"_index\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_score\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_source\",\"type\":\"_source\",\"esTypes\":[\"_source\"],\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_type\",\"type\":\"string\",\"esTypes\":[\"_type\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"customer_name\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"customer_name.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"parent\":\"customer_name\",\"subType\":\"multi\"},{\"name\":\"customer_region\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"customer_region.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"parent\":\"customer_region\",\"subType\":\"multi\"},{\"name\":\"customer_ssn\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"customer_ssn.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"parent\":\"customer_ssn\",\"subType\":\"multi\"}]"
"fields" : "[{\"name\":\"_id\",\"type\":\"string\",\"esTypes\":[\"_id\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_index\",\"type\":\"string\",\"esTypes\":[\"_index\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_score\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_source\",\"type\":\"_source\",\"esTypes\":[\"_source\"],\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_type\",\"type\":\"string\",\"esTypes\":[\"_type\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"customer_name\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"customer_name.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\": \"customer_name\"}}},{\"name\":\"customer_region\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"customer_region.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\": \"customer_region\"}}},{\"name\":\"customer_ssn\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"customer_ssn.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\": \"customer_ssn\"}}}]"
},
"type" : "index-pattern",
"references" : [ ],

6
x-pack/test/saved_object_api_integration/common/fixtures/es_archiver/saved_objects/spaces/data.json
View file

File diff suppressed because one or more lines are too long