mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-25 15:47:23 -04:00
* First steps in docs for runtime fields. * Adding new page for runtime fields. * Adding page for runtime fields. * Adding more to the runtime fields topic. * Adding parameters and retrieval options for runtime fields. * Adding TESTSETUP for index creation. * Incorporating review feedback. * Incorporating reviewer feedback. * Adding examples for runtime fields. * Adding more context and simplifying the example. * Changing timestamp to @timestamp throughout. * Removing duplicate @timestamp field. * Expanding example to hopefully fix CI builds. * Adding skip test for result. * Adding missing callout. * Adding TESTRESPONSEs, which are currently broken. * Fixing TESTRESPONSEs. * Incorporating review feedback. * Several clarifications, better test cases, and other changes. * Adding missing callout in example. * Adding substitutions to TESTRESPONSE for shorter results shown. * Shuffling some information and adding link to script-fields. * Fixing typo. * Updates for API redesign -- will break builds. * Updating examples and including info about overriding fields. * Updating examples. * Adding info for using runtime fields in the search request. * Adding that queries against runtime fields are expensive. * Incorporating feedback from reviewers. * Minor changes from reviews. * Adding alias for test case. * Adding aliases to PUT example. * Fixing test cases, for real this time. * Updating use cases and introducing overlay throughout. * Edits, adding 'shadowing', and explaining shadowing better. * Streamlining tests and other changes. * Fix formatting in example for test. * Apply suggestions from code review * Incorporating reviewer feedback 7 Dec * Shifting structure of mapping page to fix cross links. * Revisions for shadowing, overview, and other sections. * Removing dot notation section and incorporating review changes. * Adding updated example for shadowing. * Streamlining shadowing example and TESTRESPONSEs.
234 lines
6 KiB
Text
234 lines
6 KiB
Text
[[mapping]]
|
|
= Mapping
|
|
|
|
[partintro]
|
|
--
|
|
|
|
Mapping is the process of defining how a document, and the fields it contains,
|
|
are stored and indexed. For instance, use mappings to define:
|
|
|
|
* which string fields should be treated as full text fields.
|
|
* which fields contain numbers, dates, or geolocations.
|
|
* the <<mapping-date-format,format>> of date values.
|
|
* custom rules to control the mapping for
|
|
<<dynamic-mapping,dynamically added fields>>.
|
|
|
|
A mapping definition includes metadata fields and fields:
|
|
|
|
<<mapping-fields,Metadata fields>>::
|
|
|
|
Metadata fields are used to customize how a document's associated metadata is
|
|
treated. Examples of metadata fields include the document's
|
|
<<mapping-index-field,`_index`>>, <<mapping-id-field,`_id`>>, and
|
|
<<mapping-source-field,`_source`>> fields.
|
|
|
|
<<mapping-types,Fields>>::
|
|
|
|
A mapping contains a list of fields or `properties` pertinent to the
|
|
document. Each field has its own <<mapping-types, data type>>.
|
|
|
|
NOTE: Before 7.0.0, the 'mappings' definition used to include a type name.
|
|
For more details, please see <<removal-of-types>>.
|
|
|
|
[discrete]
|
|
[[mapping-limit-settings]]
|
|
== Settings to prevent mapping explosion
|
|
Defining too many fields in an index can lead to a mapping explosion, which can
|
|
cause out of memory errors and difficult situations to recover from.
|
|
|
|
Consider a situation where every new document inserted
|
|
introduces new fields, such as with <<dynamic-mapping,dynamic mapping>>.
|
|
Each new field is added to the index mapping, which can become a
|
|
problem as the mapping grows.
|
|
|
|
Use the <<mapping-settings-limit,mapping limit settings>> to limit the number
|
|
of field mappings (created manually or dynamically) and prevent documents from
|
|
causing a mapping explosion.
|
|
|
|
[discrete]
|
|
[[runtime-fields]]
|
|
== Runtime fields
|
|
Typically, you index data into {es} to promote faster search. However, indexing
|
|
can be slow and requires more disk space, and you have to reindex your data to
|
|
add fields to existing documents.
|
|
|
|
<<runtime,Runtime fields>> are not indexed, which saves disk space and makes
|
|
data ingest faster. You can add runtime fields to existing documents without
|
|
reindexing your data and calculate field values dynamically at search time.
|
|
|
|
[discrete]
|
|
[[dynamic-mapping-intro]]
|
|
== Dynamic mapping
|
|
|
|
Fields and mapping types do not need to be defined before being used. Thanks
|
|
to _dynamic mapping_, new field names will be added automatically, just by
|
|
indexing a document. New fields can be added both to the top-level mapping
|
|
type, and to inner <<object,`object`>> and <<nested,`nested`>> fields.
|
|
|
|
The <<dynamic-mapping,dynamic mapping>> rules can be configured to customise
|
|
the mapping that is used for new fields.
|
|
|
|
[discrete]
|
|
== Explicit mappings
|
|
|
|
You know more about your data than Elasticsearch can guess, so while dynamic
|
|
mapping can be useful to get started, at some point you will want to specify
|
|
your own explicit mappings.
|
|
|
|
You can create field mappings when you <<create-mapping,create an index>> and
|
|
<<add-field-mapping,add fields to an existing index>>.
|
|
|
|
[discrete]
|
|
[[create-mapping]]
|
|
=== Create an index with an explicit mapping
|
|
|
|
You can use the <<indices-create-index,create index>> API to create a new index
|
|
with an explicit mapping.
|
|
|
|
[source,console]
|
|
----
|
|
PUT /my-index-000001
|
|
{
|
|
"mappings": {
|
|
"properties": {
|
|
"age": { "type": "integer" }, <1>
|
|
"email": { "type": "keyword" }, <2>
|
|
"name": { "type": "text" } <3>
|
|
}
|
|
}
|
|
}
|
|
----
|
|
|
|
<1> Creates `age`, an <<number,`integer`>> field
|
|
<2> Creates `email`, a <<keyword,`keyword`>> field
|
|
<3> Creates `name`, a <<text,`text`>> field
|
|
|
|
[discrete]
|
|
[[add-field-mapping]]
|
|
== Add a field to an existing mapping
|
|
|
|
You can use the <<indices-put-mapping, put mapping>> API to add one or more new
|
|
fields to an existing index.
|
|
|
|
The following example adds `employee-id`, a `keyword` field with an
|
|
<<mapping-index,`index`>> mapping parameter value of `false`. This means values
|
|
for the `employee-id` field are stored but not indexed or available for search.
|
|
|
|
[source,console]
|
|
----
|
|
PUT /my-index-000001/_mapping
|
|
{
|
|
"properties": {
|
|
"employee-id": {
|
|
"type": "keyword",
|
|
"index": false
|
|
}
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
[discrete]
|
|
[[update-mapping]]
|
|
=== Update the mapping of a field
|
|
|
|
include::{es-repo-dir}/indices/put-mapping.asciidoc[tag=change-field-mapping]
|
|
|
|
include::{es-repo-dir}/indices/put-mapping.asciidoc[tag=rename-field]
|
|
|
|
[discrete]
|
|
[[view-mapping]]
|
|
== View the mapping of an index
|
|
|
|
You can use the <<indices-get-mapping, get mapping>> API to view the mapping of
|
|
an existing index.
|
|
|
|
[source,console]
|
|
----
|
|
GET /my-index-000001/_mapping
|
|
----
|
|
// TEST[continued]
|
|
|
|
The API returns the following response:
|
|
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"my-index-000001" : {
|
|
"mappings" : {
|
|
"properties" : {
|
|
"age" : {
|
|
"type" : "integer"
|
|
},
|
|
"email" : {
|
|
"type" : "keyword"
|
|
},
|
|
"employee-id" : {
|
|
"type" : "keyword",
|
|
"index" : false
|
|
},
|
|
"name" : {
|
|
"type" : "text"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
----
|
|
|
|
|
|
[discrete]
|
|
[[view-field-mapping]]
|
|
== View the mapping of specific fields
|
|
|
|
If you only want to view the mapping of one or more specific fields, you can use
|
|
the <<indices-get-field-mapping, get field mapping>> API.
|
|
|
|
This is useful if you don't need the complete mapping of an index or your index
|
|
contains a large number of fields.
|
|
|
|
The following request retrieves the mapping for the `employee-id` field.
|
|
|
|
[source,console]
|
|
----
|
|
GET /my-index-000001/_mapping/field/employee-id
|
|
----
|
|
// TEST[continued]
|
|
|
|
The API returns the following response:
|
|
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"my-index-000001" : {
|
|
"mappings" : {
|
|
"employee-id" : {
|
|
"full_name" : "employee-id",
|
|
"mapping" : {
|
|
"employee-id" : {
|
|
"type" : "keyword",
|
|
"index" : false
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
----
|
|
|
|
--
|
|
|
|
include::mapping/removal_of_types.asciidoc[]
|
|
|
|
include::mapping/mapping-settings-limit.asciidoc[]
|
|
|
|
include::mapping/types.asciidoc[]
|
|
|
|
include::mapping/runtime.asciidoc[]
|
|
|
|
include::mapping/fields.asciidoc[]
|
|
|
|
include::mapping/params.asciidoc[]
|
|
|
|
include::mapping/dynamic-mapping.asciidoc[]
|