Commit graph

14 commits

Author SHA1 Message Date
Drew Tate
15ef37f2fd
[ES|QL] Auto open suggestions for field lists (#190466)
## Summary

Part of https://github.com/elastic/kibana/issues/189662

## Suggests comma and pipe


https://github.com/user-attachments/assets/c09bc6fd-20a6-4f42-a871-c70e68e6d81a

## Doesn't suggest comma when there are no more fields


https://github.com/user-attachments/assets/29fce13e-e58b-4d93-bce5-6b1f913b4d92

## Doesn't work for escaped columns :(


https://github.com/user-attachments/assets/3d65f3b9-923d-4c0e-9c50-51dd83115c8b

As part of this effort I discovered
https://github.com/elastic/kibana/issues/191100 and
https://github.com/elastic/kibana/issues/191105, as well as a problem
with column name validation (see
https://github.com/elastic/kibana/issues/177699)

I think we can revisit column escaping and probably resolve all of these
issues (issue [here](https://github.com/elastic/kibana/issues/191111)).


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2024-08-23 12:17:03 -05:00
Drew Tate
bb2cc70a3b
[ES|QL] improve test case generator script (#190441)
## Summary

Fixes a bug I introduced in
https://github.com/elastic/kibana/pull/189941

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-08-15 09:07:16 -05:00
Kibana Machine
8b2e05b09e
[ES|QL] Update function metadata (#190290)
This PR updates the function definitions and inline docs based on the
latest metadata from Elasticsearch.

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
2024-08-13 07:47:18 +02:00
Drew Tate
edf5c762d8
[ES|QL] Stronger typing for ESQL field interface (#189941)
## Summary

The heart of this PR is [this
change](https://github.com/elastic/kibana/pull/189941/files#diff-88513481c44d7b2de70ca2f7826c2b1fb1d8bda5b308aab0f8917a42ac2c24b7R11-R94)
where I clean up and clarify the various data-type-related types
floating around in the engines + [this
change](https://github.com/elastic/kibana/pull/189941/files#diff-f48b526b82119bd591cf781262173d7a0233d236ab26496a4c06f5ea9a441561R21)
where I add strong typing to the ES|QL field interface.

Pretty much everything else is a result of that. For example, strongly
typing the fields and test helpers highlighted a bunch of tests that
were still using Kibana types instead of Elasticsearch types. So, then
those had to be updated.

There's more work to do to extend the strong field typing to the rest of
the engines, but this got big and I decided to do it piece-meal.

Next plans
- Extend typing to subroutines in autocomplete and validation engines
(e.g. the stuff in `factories.ts`)
- Add typing to the [variable
interface](https://github.com/elastic/kibana/pull/189941/files#diff-f48b526b82119bd591cf781262173d7a0233d236ab26496a4c06f5ea9a441561R13)
- Consider merging `time_literal` and `time_duration` types
- "It looks like timespan literals are the way to write a constant
time_duration. and time_durations can only be constants at the moment.
so they aren't the same, but sure are about the same" - Nik
- Consider merging
`packages/kbn-esql-validation-autocomplete/src/shared/esql_types.ts`
with
`packages/kbn-esql-validation-autocomplete/src/definitions/types.ts` in
some common place


### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
2024-08-08 08:11:43 -05:00
Quynh Nguyen (Quinn)
7dca2aa712
[ES|QL] Improved support for Elasticsearch sub-types in AST for both validation and autocomplete (#189689)
## Summary

Fixed version of https://github.com/elastic/kibana/pull/188600 that
updates the failed tests [caused by clash with the visitor API
tests](https://github.com/elastic/kibana/pull/189516).

### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces&mdash;unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes&mdash;Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-08-01 11:04:21 -05:00
Jonathan Budzenski
8a60cd4e68 Revert "[ES|QL] Add improved support for Elasticsearch sub-types in AST for both validation and autocomplete (#188600)"
This reverts commit d9282a6e6e.
2024-07-31 18:37:05 -05:00
Quynh Nguyen (Quinn)
d9282a6e6e
[ES|QL] Add improved support for Elasticsearch sub-types in AST for both validation and autocomplete (#188600)
## Summary

This PR addresses https://github.com/elastic/kibana/issues/174710 and
leverages sub-types in AST for both validation and autocomplete. Changes
include:
- Expanded the automatically generated function signatures to not longer
cast to `string` or `number`. Instead, it expands out to `keyword`,
`text`, `double`, `long`, `float`, etc.
- Expanded math function signatures to reflect the true return types
(e.g double + interger => double)
- Expanded function signatures for `bucket`
- Expanded builtin functions to better support types
-  Literal support: 
- Added a `decimal` literal context → Which will be accepted by function
signatures for 'double', 'long', 'float', etc.
- Added and `integer` literal type → Which will be accepted by function
signatures for 'interger'
- Updated tests

After:
<img width="1181" alt="Screenshot 2024-07-24 at 13 06 23"
src="https://github.com/user-attachments/assets/42f32f3b-2e6e-4dff-8b20-dd60f8c45750">
<img width="1726" alt="Screenshot 2024-07-24 at 13 06 47"
src="https://github.com/user-attachments/assets/3c6f1eac-600e-4b2c-9e57-f71f79ad9ce9">
<img width="1726" alt="Screenshot 2024-07-24 at 13 07 09"
src="https://github.com/user-attachments/assets/e8b5817e-d4de-446b-8b41-c6c7b8612420">

### Checklist

Delete any items that are not applicable to this PR.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)


### Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.

When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:

| Risk | Probability | Severity | Mitigation/Notes |

|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces&mdash;unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes&mdash;Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |


### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2024-07-31 17:08:11 -05:00
Drew Tate
a0b018b12c
[ES|QL] remove chrono literals (#186536)
## Summary

The "chrono-literals" can be replaced with the simpler system of
`literalOptions`. This is better because it moves us toward having _one_
way to do things and removing Kibana-specific types which don't actually
exist in ES|QL and, therefore, can be confusing.
2024-07-10 10:22:59 -05:00
Drew Tate
197fde2c17
[ES|QL] add time unit abbreviations (#186424)
## Summary

Close https://github.com/elastic/kibana/issues/186413

<img width="303" alt="Screenshot 2024-06-18 at 1 35 52 PM"
src="92d10866-dc46-4c0f-b6d9-e4aa09e0c9c9">

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
2024-06-20 14:12:37 -05:00
Kibana Machine
67940a2425
[ES|QL] Update function definitions (#184626)
This PR updates the function definitions based on the latest metadata
from Elasticsearch.

---------

Co-authored-by: Drew Tate <andrew.tate@elastic.co>
2024-06-05 11:56:03 -05:00
Drew Tate
9abe56f1c6
[ES|QL] add tests for implicit date casting (#184612)
## Summary

Relates to https://github.com/elastic/elasticsearch/issues/109265

These test reflect the reality of how date casting is working in
Elasticsearch today. When the above issue is addressed in Elasticsearch,
these tests will fail the error sync check and prompt us to update the
validator.

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
2024-06-03 09:10:44 -05:00
Drew Tate
574fe0a436
[ES|QL] accept null values for function arguments (#184254)
## Summary

Accepts null values for all function and operator arguments.

<img width="510" alt="Screenshot 2024-05-24 at 11 04 43 AM"
src="75ce23ed-ec91-404b-8c65-6f023fb71a57">



### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
2024-05-28 08:55:32 -05:00
Drew Tate
72d2a56019
[ES|QL] Generate function definitions from Elasticsearch metadata (#179584)
## Summary

Resolve https://github.com/elastic/kibana/issues/179634

This PR introduces a script to gather metadata from Elasticsearch and
build the function definitions for Kibana's client-side validation.

To run the script: `cd packages/kbn-esql-validation-autocomplete && yarn
makedefs path/to/elasticsearch/repo`

## Current limitations
### Things we can't (yet) gather from ES

#### Things that are currently being filled in on the Kibana side but
would be missing if we had to rely completely on ES
- function aliases (e.g. `to_str`)
- operators and aggs (issues for Elasticsearch
https://github.com/elastic/elasticsearch/issues/107219 and
https://github.com/elastic/elasticsearch/issues/107220)
- Kibana AST parameter settings
- `constantOnly` — whether a parameter can be a non-literal or not
(`percentile`, `auto_bucket`)
- `constantOptions`/`constantSuggestions` — specific information about
which constants are accepted or suggested for particular parameters.
- `supportsWildcard` — whether a parameter can contain `*` (only `count`
at the moment)
- `noNestingFunctions` — whether a parameter can contain nested
functions (used for all agg functions)
- Kibana date constant types
  - `chrono_literal`
  - `time_literal`

### Other considerations
- Asciidoc links need to be scrubbed (e.g. `Function
{wikipedia}/foo/bar[arccosine] returns blah blah`)
- As of https://github.com/elastic/elasticsearch/pull/106454 some
descriptions reference outdated parameter names and some parameter names
are questionable:
<img width="600" alt="Screenshot 2024-03-28 at 1 07 16 PM"
src="3b9ab457-e9fd-4c5f-afb2-23ae769bb44a">

_parameter `n` is now called `number`, but the description still calls
it `n`... also, is `number` really the best name for the variable?_

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
2024-05-24 08:03:56 -05:00
Drew Tate
c4ef9bdccf
[ES|QL] generate function validation tests (#183343)
## Summary

Close https://github.com/elastic/kibana/issues/182390


### To test

You can test all aspects of the script by first making some changes to
`packages/kbn-esql-validation-autocomplete/src/validation/validation.test.ts`
- delete several tests from an existing function describe block ("block
1"—except don't choose "date_diff" since it has a bunch of custom tests)
- delete another function describe block completely ("block 2")
- change the expected result of several of the tests in a third function
describe block ("block 3")

Then, run `yarn maketests` from within
`packages/kbn-esql-validation-autocomplete`

**Expected result**
- Block 1 should have the deleted tests restored
- Block 2 should be restored entirely (though it may be moved in the
tests file)
- Block 3 should be untouched

### Checklist

- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2024-05-16 16:06:16 +02:00