mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-25 07:37:19 -04:00
281ec58b8d
The char filter replaces the previous default of `first_non_blank_line`. `first_non_blank_line` worked well to figure out what line had characters at all, but log lines like the following were handled poorly: ``` -------------------------------------------------------------------------------- Alias 'foo' already exists and this prevents setting up ILM for logs -------------------------------------------------------------------------------- ``` When combined with the `ml_standard` tokenizer, the first line was used: ``` -------------------------------------------------------------------------------- ``` This has no valid tokens for our standard tokenizer. Consequently, no tokens were found by `ml_standard` tokenizer. The new filter, `first_line_with_letters`, returns the first line with any letter character (e.g. `Character#isLetter` returns true). Given the previously poorly handled log, when combining with our `ml_standard` tokenizer, we get the following, more appropriate, tokens: ``` "tokens" : ["Alias", "foo", "already", "exists", "and", "this", "prevents", "setting", "up", "ILM", "for", "logs"] ``` |
||
|---|---|---|
| .. | ||
| apis | ||
| functions | ||
| ml-configuring-aggregations.asciidoc | ||
| ml-configuring-alerts.asciidoc | ||
| ml-configuring-categories.asciidoc | ||
| ml-configuring-detector-custom-rules.asciidoc | ||
| ml-configuring-populations.asciidoc | ||
| ml-configuring-transform.asciidoc | ||
| ml-configuring-url.asciidoc | ||
| ml-delayed-data-detection.asciidoc | ||