kibana/docs/concepts/kuery.asciidoc
Lukas Olson be35641e17
[docs] [kql] Update KQL docs to include info about errors with multiple field types (#158035)
## Summary

We received feedback that the given example would result in errors,
since it targets both
[string](https://www.elastic.co/guide/en/ecs/current/ecs-http.html#field-http-response-body-content)
and
[numeric](https://www.elastic.co/guide/en/ecs/current/ecs-http.html#field-http-response-bytes)
fields, so we've updated the example with something that won't result in
an error, and updated the docs with a note that explains when/why these
errors may occur.

### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
2023-05-23 12:29:47 -04:00

252 lines
7.8 KiB
Text
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

[[kuery-query]]
=== {kib} Query Language
The {kib} Query Language (KQL) is a simple text-based query language for filtering data.
* KQL only filters data, and has no role in aggregating, transforming, or sorting data.
* KQL is not to be confused with the <<lucene-query,Lucene query language>>, which has a different feature set.
Use KQL to filter documents where a value for a field exists, matches a given value, or is within a given range.
[discrete]
=== Filter for documents where a field exists
To filter documents for which an indexed value exists for a given field, use the `*` operator.
For example, to filter for documents where the `http.request.method` field exists, use the following syntax:
[source,yaml]
-------------------
http.request.method: *
-------------------
This checks for any indexed value, including an empty string.
[discrete]
=== Filter for documents that match a value
Use KQL to filter for documents that match a specific number, text, date, or boolean value.
For example, to filter for documents where the `http.request.method` is GET, use the following query:
[source,yaml]
-------------------
http.request.method: GET
-------------------
The field parameter is optional. If not provided, all fields are searched for the given value.
For example, to search all fields for “Hello”, use the following:
[source,yaml]
-------------------
Hello
-------------------
When querying keyword, numeric, date, or boolean fields, the value must be an exact match,
including punctuation and case. However, when querying text fields, {es} analyzes the
value provided according to the {ref}/analysis.html[fields mapping settings].
For example, to search for documents where `http.request.body.content` (a `text` field)
contains the text “null pointer”:
[source,yaml]
-------------------
http.request.body.content: null pointer
-------------------
Because this is a `text` field, the order of these search terms does not matter, and
even documents containing “pointer null” are returned. To search `text` fields where the
terms are in the order provided, surround the value in quotation marks, as follows:
[source,yaml]
-------------------
http.request.body.content: "null pointer"
-------------------
Certain characters must be escaped by a backslash (unless surrounded by quotes).
For example, to search for documents where `http.request.referrer` is https://example.com,
use either of the following queries:
[source,yaml]
-------------------
http.request.referrer: "https://example.com"
http.request.referrer: https\://example.com
-------------------
You must escape following characters:
[source,yaml]
-------------------
\():<>"*
-------------------
[discrete]
=== Filter for documents within a range
To search documents that contain terms within a provided range, use KQLs range syntax.
For example, to search for all documents for which `http.response.bytes` is less than 10000,
use the following syntax:
[source,yaml]
-------------------
http.response.bytes < 10000
-------------------
To search for an inclusive range, combine multiple range queries.
For example, to search for documents where `http.response.bytes` is greater than 10000
but less than or equal to 20000, use the following syntax:
[source,yaml]
-------------------
http.response.bytes > 10000 and http.response.bytes <= 20000
-------------------
You can also use range syntax for string values, IP addresses, and timestamps.
For example, to search for documents earlier than two weeks ago, use the following syntax:
[source,yaml]
-------------------
@timestamp < now-2w
-------------------
For more examples on acceptable date formats, refer to {ref}/common-options.html#date-math[Date Math].
[discrete]
=== Filter for documents using wildcards
To search for documents matching a pattern, use the wildcard syntax.
For example, to find documents where `http.response.status_code` begins with a 4, use the following syntax:
[source,yaml]
-------------------
http.response.status_code: 4*
-------------------
By default, leading wildcards are not allowed for performance reasons.
You can modify this with the <<query-allowleadingwildcards,`query:allowLeadingWildcards`>> advanced setting.
NOTE: Only `*` is currently supported. This matches zero or more characters.
[discrete]
=== Negating a query
To negate or exclude a set of documents, use the `not` keyword (not case-sensitive).
For example, to filter documents where the `http.request.method` is *not* GET, use the following query:
[source,yaml]
-------------------
NOT http.request.method: GET
-------------------
[discrete]
=== Combining multiple queries
To combine multiple queries, use the `and`/`or` keywords (not case-sensitive).
For example, to find documents where the `http.request.method` is GET *or* the `http.response.status_code` is 400,
use the following query:
[source,yaml]
-------------------
http.request.method: GET OR http.response.status_code: 400
-------------------
Similarly, to find documents where the `http.request.method` is GET *and* the
`http.response.status_code` is 400, use this query:
[source,yaml]
-------------------
http.request.method: GET AND http.response.status_code: 400
-------------------
To specify precedence when combining multiple queries, use parentheses.
For example, to find documents where the `http.request.method` is GET *and*
the `http.response.status_code` is 200, *or* the `http.request.method` is POST *and*
`http.response.status_code` is 400, use the following:
[source,yaml]
-------------------
(http.request.method: GET AND http.response.status_code: 200) OR
(http.request.method: POST AND http.response.status_code: 400)
-------------------
You can also use parentheses for shorthand syntax when querying multiple values for the same field.
For example, to find documents where the `http.request.method` is GET, POST, *or* DELETE, use the following:
[source,yaml]
-------------------
http.request.method: (GET OR POST OR DELETE)
-------------------
[discrete]
=== Matching multiple fields
Wildcards can also be used to query multiple fields. For example, to search for
documents where any sub-field of `datastream` contains “logs”, use the following:
[source,yaml]
-------------------
datastream.*: logs
-------------------
NOTE: When using wildcards to query multiple fields, errors might occur if the fields are of
different types. For example, if `datastream.*` matches both numeric and string fields, the
above query will result in an error because numeric fields cannot be queried for string values.
[discrete]
=== Querying nested fields
Querying {ref}/nested.html[nested fields] requires a special syntax. Consider the
following document, where `user` is a nested field:
[source,yaml]
-------------------
{
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}
-------------------
To find documents where a single value inside the `user` array contains a first name of
“Alice” and last name of “White”, use the following:
[source,yaml]
-------------------
user:{ first: "Alice" and last: "White" }
-------------------
Because nested fields can be inside other nested fields,
you must specify the full path of the nested field you want to query.
For example, consider the following document where `user` and `names` are both nested fields:
[source,yaml]
-------------------
{
"user": [
{
"names": [
{
"first": "John",
"last": "Smith"
},
{
"first": "Alice",
"last": "White"
}
]
}
]
}
-------------------
To find documents where a single value inside the `user.names` array contains a first name of “Alice” *and*
last name of “White”, use the following:
[source,yaml]
-------------------
user.names:{ first: "Alice" and last: "White" }
-------------------