There has been some confusion over the definition of a field type family. This
PR clarifies the definition in the docs: the two types should have the exact
same search behavior (including supporting the same queries/ aggs, and producing
the same response). It's not sufficient for them to just support the samme
search operations.
This change also fixes an inaccurate statement that there is only one field type
family so far.
This PR introduces the lookup runtime fields which are used to retrieve
data from the related indices. The below search request enriches its
search hits with the location of each IP address from the `ip_location`
index.
```
POST logs/_search
{
"runtime_mappings": {
"location": {
"type": "lookup",
"lookup_index": "ip_location",
"query_type": "term",
"query_input_field": "ip",
"query_target_field": "_id",
"fetch_fields": [
"country",
"city"
]
}
},
"fields": [
"timestamp",
"message",
"location"
]
}
```
Response:
```
{
"hits": {
"hits": [
{
"_index": "logs",
"_id": "1",
"fields": {
"location": [
{
"city": [ "Montreal" ],
"country": [ "Canada" ]
}
],
"message": [ "the first message" ]
}
}
]
}
}
```
Clarifies that the `orientation` mapping parameter only applies to WKT polygons. GeoJSON polygons use a default orientation of `RIGHT`, regardless of the mapping parameter.
Also notes that the document-level `orientation` parameter overrides the default orientation for both WKT and GeoJSON polygons.
Closes https://github.com/elastic/elasticsearch/issues/84009.
Similar to #82409, but for geo_point fields.
Allows searching on geo_point fields when those fields are not indexed (index: false) but just doc values are enabled.
Also adds distance feature query support for date fields (bringing date field to feature parity with runtime fields)
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
* [DOCS] Update dynamic mapping docs to clarify supported match_mapping_type
* Add ES data type column header
* Remove sentence about always choosing the larger data type
* Clarify that JSON doesn't distinguish types
* Add frame to table
Updates and reuses a warning against creating multi-level `join` fields to make it more prominent.
The current warning is low on the page, where some users may not seeing until they've already begun mapping fields.
Closes https://github.com/elastic/elasticsearch/issues/82818.
Allows searching on ip fields when those fields are not indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
Allows searching on boolean fields when those fields are not indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
Allows searching on keyword fields when those fields are not indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
Similar to #82409, but for date fields.
Allows searching on date field types (date, date_nanos) when those fields are not indexed (index: false) but just doc
values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Relates #81210 and #52728
Allows searching on number field types (long, short, int, float, double, byte, half_float) when those fields are not
indexed (index: false) but just doc values are enabled.
This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.
Note to reviewers:
I have split isSearchable into two separate methods isIndexed and isSearchable on MappedFieldType. The former one is
about whether actual indexing data structures have been used (postings or points), and the latter one on whether you
can run queries on the given field (e.g. used by field caps). For number field types, queries are now allowed whenever
points are available or when doc values are available (i.e. searchability is expanded).
Relates #81210 and #52728
Cosine similarity is not defined when one of the vectors has zero magnitude.
Before, the kNN search endpoint threw a confusing exception related to top docs
collection. Now we reject vectors early with a clear error message, failing
indexing if the vector has zero magnitude.
This commit adds docs for the new `_knn_search` endpoint.
It focuses on being an API reference and is light on details in terms of how
exactly the kNN search works, and how the endpoint contrasts with
`script_score` queries. We plan to add a high-level guide on kNN search that
will explain this in depth.
Relates to #78473.
Currently, we don't support kNN search against fields in a `nested` mapping.
Before, we were checking this at search-time. This commit moves it earlier, so
you aren't even allowed to set `index: true` if the vector is in a nested
mapping. That way, users are aware of the limitation before they start to index
documents.
Relates to #78473.
This reverts the change to use segment ordinals in composite terms aggregations due to a performance degradation when the field is high cardinality.
Co-authored-by: Mark Tozzi <mark.tozzi@elastic.co>
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.
Relates to #79309, #31619
This commit updates the `dense_vector` docs to include information on the new
`index`, `similarity`, and `index_options` parameters. It also tries to clarify
the difference between `similarity` and `index_options` with the existing
parameters that have the same name.
Relates to #78473.
Adds an `experimental` annotation to the following:
* `time_series_metric` mapping parameter
* `time_series_dimension` mapping parameter
* `index.mapping.dimension_fields.limit` index setting
* `time_series_dimension` and `time_series_metric` properties in the field caps API response
Changes:
* Documents the `time_series_metric` mapping parameter for PR #76766.
* Renames the `dimension` parameter to `time_series_dimension` for PR #78012.
* Adds support for `unsigned_long` to `time_series_dimension` for PR #78204.
Adds additional information about how Elasticsearch uses polygon orientation. Elasticsearch only uses a polygon's orientation to determine if it crosses the international dateline. If so, Elasticsearch splits the polygon at the dateline.
Closes#74891
* Mention match_only_text in disk usage docs
Previously we explained how to manually disable norms, freqs, and positions. We
now have a ready-made solution in the new `match_only_text` field type.
* Fixing typo and minor grammar changes
Co-authored-by: Adam Locke <adam.locke@elastic.co>
The `_routing` metadata field docs currently include formulas for how
Elasticsearch routes documents to shards. However, these formulas were not
updated for #18699. This updates the routing formulas and adds xrefs for
related settings.
Closes#76072
Changes:
* Use "geopoint" when not referring to the literal field type
* Use "geoshape" when not referring to the literal field type or query type
* Use "GeoJSON" consistently
