mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-06-28 17:34:17 -04:00
ed86750fa4
Currently the `token_chars` setting in both `edgeNGram` and `ngram` tokenizers only allows for a list of predefined character classes, which might not fit every use case. For example, including underscore "_" in a token would currently require the `punctuation` class which comes with a lot of other characters. This change adds an additional "custom" option to the `token_chars` setting, which requires an additional `custom_token_chars` setting to be present and which will be interpreted as a set of characters to inlcude into a token. Closes #25894 |
||
|---|---|---|
| .. | ||
| chargroup-tokenizer.asciidoc | ||
| classic-tokenizer.asciidoc | ||
| edgengram-tokenizer.asciidoc | ||
| keyword-tokenizer.asciidoc | ||
| letter-tokenizer.asciidoc | ||
| lowercase-tokenizer.asciidoc | ||
| ngram-tokenizer.asciidoc | ||
| pathhierarchy-tokenizer-examples.asciidoc | ||
| pathhierarchy-tokenizer.asciidoc | ||
| pattern-tokenizer.asciidoc | ||
| simplepattern-tokenizer.asciidoc | ||
| simplepatternsplit-tokenizer.asciidoc | ||
| standard-tokenizer.asciidoc | ||
| thai-tokenizer.asciidoc | ||
| uaxurlemail-tokenizer.asciidoc | ||
| whitespace-tokenizer.asciidoc | ||