Backports #110222 to 7.17.
JDK 23 removes the COMPAT locale provider, leaving CLDR as the only option. This commit configures Elasticsearch
to use the CLDR provider when on JDK 23, but still use the existing COMPAT provider when on JDK 22 and below.
This causes some differences in locale behaviour; this also adapts various tests to still work whether run on COMPAT or CLDR.
Documentation incorrectly states that all aggregations are supported by
the `aggregate_metric_double` field.
This PR rectifies this error.
Closes#92236
There has been some confusion over the definition of a field type family. This
PR clarifies the definition in the docs: the two types should have the exact
same search behavior (including supporting the same queries/ aggs, and producing
the same response). It's not sufficient for them to just support the samme
search operations.
This change also fixes an inaccurate statement that there is only one field type
family so far.
Clarifies that the `orientation` mapping parameter only applies to WKT polygons. GeoJSON polygons use a default orientation of `RIGHT`, regardless of the mapping parameter.
Also notes that the document-level `orientation` parameter overrides the default orientation for both WKT and GeoJSON polygons.
Closes https://github.com/elastic/elasticsearch/issues/84009.
(cherry picked from commit 6ad3f8bfdd)
* [DOCS] Update dynamic mapping docs to clarify supported match_mapping_type
* Add ES data type column header
* Remove sentence about always choosing the larger data type
* Clarify that JSON doesn't distinguish types
* Add frame to table
Updates and reuses a warning against creating multi-level `join` fields to make it more prominent.
The current warning is low on the page, where some users may not seeing until they've already begun mapping fields.
Closes https://github.com/elastic/elasticsearch/issues/82818.
(cherry picked from commit d3fb014914)
Adds an `experimental` annotation to the following:
* `time_series_metric` mapping parameter
* `time_series_dimension` mapping parameter
* `index.mapping.dimension_fields.limit` index setting
* `time_series_dimension` and `time_series_metric` properties in the field caps API response
# Conflicts:
# docs/reference/search/field-caps.asciidoc
Backports the following PRs:
* Add dimension mapping parameter (#74450)
Added the dimension parameter to the following field types:
keyword
ip
Numeric field types (integer, long, byte, short)
The dimension parameter is of type boolean (default: false) and is used
to mark that a field is a time series dimension field.
Relates to #74014
* Add constraints to dimension fields (#74939)
This PR adds the following constraints to dimension fields:
It must be an indexed field and must has doc values
It cannot be multi-valued
The number of dimension fields in the index mapping must not be more than 16. This should be configurable through an index property (index.mapping.dimension_fields.limit)
keyword fields cannot be more than 1024 bytes long
keyword fields must not use a normalizer
Based on the code added in PR #74450
Relates to #74660
* Expand DocumentMapperTests (#76368)
Adds a test for setting the maximum number of dimensions setting and
tests the names and types of the metadata fields in the index.
Previously we just asserted the count of metadata fields. That made it
hard to read failures.
* Fix broken test for dimension keywords (#75408)
Test was failing because it was testing 1024 bytes long keyword and assertion was failing.
Closes#75225
* Checkstyle
* Add time_series_metric parameter (#76766)
This PR adds the time_series_metric parameter to the following field types:
Numeric field types
histogram
aggregate_metric_double
* Rename `dimension` mapping parameter to `time_series_dimension` (#78012)
This PR renames dimension mapping parameter to time_series_dimension to make it consistent with time_series_metric parameter (#76766)
Relates to #74450 and #74014
* Add time series params to `unsigned_long` and `scaled_float` (#78204)
Added the time_series_metric mapping parameter to the unsigned_long and scaled_float field types
Added the time_series_dimension mapping parameter to the unsigned_long field type
Fixes#78100
Relates to #76766, #74450 and #74014
Co-authored-by: Nik Everett <nik9000@gmail.com>
Changes:
* Documents the `time_series_metric` mapping parameter for PR #76766.
* Renames the `dimension` parameter to `time_series_dimension` for PR #78012.
* Adds support for `unsigned_long` to `time_series_dimension` for PR #78204.
Adds additional information about how Elasticsearch uses polygon orientation. Elasticsearch only uses a polygon's orientation to determine if it crosses the international dateline. If so, Elasticsearch splits the polygon at the dateline.
Closes#74891
# Conflicts:
# docs/reference/mapping/types/geo-shape.asciidoc
If you specify a deprecated mapping parameter related to prefix trees, a
`geo_shape` field will use prefix tree encoding.
This adds a related admon to the top of the geoshape field docs page. This admon will
help users avoid accidentally switching from the default BKD tree encoding.
Closes#40266.
* Mention match_only_text in disk usage docs
Previously we explained how to manually disable norms, freqs, and positions. We
now have a ready-made solution in the new `match_only_text` field type.
* Fixing typo and minor grammar changes
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Adam Locke <adam.locke@elastic.co>
The `_routing` metadata field docs currently include formulas for how
Elasticsearch routes documents to shards. However, these formulas were not
updated for #18699. This updates the routing formulas and adds xrefs for
related settings.
Closes#76072
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Changes:
* Use "geopoint" when not referring to the literal field type
* Use "geoshape" when not referring to the literal field type or query type
* Use "GeoJSON" consistently
# Conflicts:
# docs/reference/ingest/processors/enrich.asciidoc
* [DOCS] Add performance info for runtime fields
* Add script-based sorting and clarify performance
* Changing title to Incentives and reworking the intro
* Add support for range aggregations on histogram mapped fields (#74146)
This adds support for the range aggregation over `histogram` mapped fields.
Decisions made for implementation:
- Sub-aggregations are not allowed. This is to simplify implementation and follows the prior art set by the `histogram` aggregation
- Nothing fancy is done with the ranges. No filter translations as we cannot easily do a `range` filter query against histogram fields. This may be an optimization in the future.
- Ranges check the histogram value ONLY. No interpolation of values is done. If we have better statistics around the histogram this MAY be possible.
When we introduced dynamic:runtime (#65489) we decided to have it create objects dynamically under properties, as the runtime section did not (and still does not) support object fields. That proved to be a poor choice, because the runtime section is flat, supports dots in field names, and does not really need objects. Also, these end up causing unnecessary mapping conflicts.
With this commit we adapt dynamic:runtime to not dynamically create objects.
Closes#70268
* [DOCS] Add retriving from flattened fields
* Clarify sub-field syntax
* Moving sub-field retrieval to flattened field docs
* Remove full example and de-emphasize runtime fields
* Remove extraneous sample tag
* [DOCS] [7.x] Create a new page for grok content in scripting docs (#73118)
* [DOCS] Moving grok to its own scripting page
* Adding examples
* Updating cross link for grok page
* Adds same runtime field in a search request for #73262
* Clarify titles and shift navigation
* Incorporating review feedback
* Updating cross-link to Painless
* Adding doc type to response
* [DOCS] Expand information on when to use a runtime field without a script
* Reworking information based on review feedback
* Clarify case where doc_values are disabled
* A few minor changes from review feedback
We currently don't support `copy_to` for fields that take the form of objects
(e.g. `date_range` or certain kinds of `geo_point` variants). The current
problem with objects is that when DocumentParser parses anything other than
single values, it potentially advances the underlying parser past the value that
we would need to stay on for parsing the value again. While we might want to
support this in the future, for now this PR enhances the otherwise confusing
MapperParsingException with something more helpful and adds a short note in the
documentation about this restriction.
Closes#49344
