mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-06-29 01:44:36 -04:00
3d307e0d78
Always return `KEYWORD` for functions that previously returned `TEXT`, because any change to the value, no matter how small, is enough to render meaningless the original analyzer associated with the `TEXT` field value. In principle, if the attribute is no longer the original `FieldAttribute`, it can no longer claim to have the type `TEXT`. This has been done for all functions: conversion functions, aggregating functions, multi-value functions. There were several that already produced `KEYWORD` for `TEXT` input (eg. ToString, FromBase64 and ToBase64, MvZip, ToLower, ToUpper, DateFormat, Concat, Left, Repeat, Replace, Right, Split, Substring), but many others that incorrectly claimed to produce `TEXT`, while this was really a false claim. This PR makes that now strict, and includes changes to the functions' units tests to disallow the tests to expect any functions output to be `TEXT`. One side effect of this change is that methods that take multiple parameters that require all of them to have the same type, will now treat TEXT and KEYWORD the same. This was already the case for functions like `Concat`, but is now also the case for `Greatest`, `Least`, `Case`, `Coalesce` and `MvAppend`. An associated change is that the type casting operator `::text` has been entirely removed. It used to map onto the `ToString` function which returned type KEYWORD, and so `::text` really produced a `KEYWORD`, which is a lie, or at least a `bug`, which is now fixed. Should we ever wish to actually produce real `TEXT`, we might love the fact that this operator has been freed up for future use (although it seems likely that function will require parameters to specify the analyzer, so might never be an operator again). ### Backwards compatibility issues: This is a change that will fail BWC tests, since we have many tests that assert on TEXT output to functions. For this reason we needed to block two scenarios: * We used the capability `functions_never_emit_text` to prevent 7 csv-spec tests and 2 yaml tests from being run against older versions that still emit text. * We used `skipTest` to also block those two yaml tests from being run against the latest build, but using older yaml files downloaded (as far back as 8.14). In all cases the change observed in these tests was simply the results columns no longer having `text` type, and instead being `keyword`. --------- Co-authored-by: Luigi Dell'Aquila <luigi.dellaquila@gmail.com> |
||
|---|---|---|
| .. | ||
| appendix | ||
| description | ||
| examples | ||
| kibana | ||
| layout | ||
| parameters | ||
| signature | ||
| types | ||
| aggregation-functions.asciidoc | ||
| binary.asciidoc | ||
| cast.asciidoc | ||
| conditional-functions-and-expressions.asciidoc | ||
| date-time-functions.asciidoc | ||
| grouping-functions.asciidoc | ||
| in.asciidoc | ||
| ip-functions.asciidoc | ||
| like.asciidoc | ||
| logical.asciidoc | ||
| math-functions.asciidoc | ||
| mv-functions.asciidoc | ||
| operators.asciidoc | ||
| predicates.asciidoc | ||
| README.md | ||
| rlike.asciidoc | ||
| spatial-functions.asciidoc | ||
| string-functions.asciidoc | ||
| type-conversion-functions.asciidoc | ||
| unary.asciidoc | ||
The files in these subdirectories are generated by ESQL's test suite:
description- description of each function scraped from@FunctionInfo#descriptionexamples- examples of each function scraped from@FunctionInfo#examplesparameters- description of each function's parameters scraped from@Paramsignature- railroad diagram of the syntax to invoke each functiontypes- a table of each combination of support type for each parameter. These are generated from tests.layout- a fully generated description for each functionkibana/definition- function definitions for kibana's ESQL editorkibana/docs- the inline docs for kibana
Most functions can use the generated docs generated in the layout directory.
If we need something more custom for the function we can make a file in this
directory that can include:: any parts of the files above.
To regenerate the files for a function run its tests using gradle:
./gradlew :x-pack:plugin:esql:test -Dtests.class='*SinTests'
To regenerate the files for all functions run all of ESQL's tests using gradle:
./gradlew :x-pack:plugin:esql:test