mirror of
https://github.com/elastic/elasticsearch.git
synced 2025-04-25 07:37:19 -04:00
555 lines
14 KiB
Text
555 lines
14 KiB
Text
[[getting-started]]
|
|
== Index and search data using {es} APIs
|
|
++++
|
|
<titleabbrev>Basics: Index and search using APIs</titleabbrev>
|
|
++++
|
|
|
|
This quick start guide is a hands-on introduction to the fundamental concepts of Elasticsearch: <<documents-indices,indices, documents and field type mappings>>.
|
|
|
|
You'll learn how to create an index, add data as documents, work with dynamic and explicit mappings, and perform your first basic searches.
|
|
|
|
[TIP]
|
|
====
|
|
The code examples in this tutorial are in {kibana-ref}/console-kibana.html[Console] syntax by default.
|
|
You can {kibana-ref}/console-kibana.html#import-export-console-requests[convert into other programming languages] in the Console UI.
|
|
====
|
|
|
|
[discrete]
|
|
[[getting-started-requirements]]
|
|
=== Requirements
|
|
|
|
You'll need a running {es} cluster, together with {kib} to use the Dev Tools API Console.
|
|
Run the following command in your terminal to set up a <<run-elasticsearch-locally,single-node local cluster in Docker>>:
|
|
|
|
[source,sh]
|
|
----
|
|
curl -fsSL https://elastic.co/start-local | sh
|
|
----
|
|
// NOTCONSOLE
|
|
|
|
////
|
|
[source,console]
|
|
----
|
|
PUT books
|
|
PUT my-explicit-mappings-books
|
|
----
|
|
// TESTSETUP
|
|
|
|
[source,console]
|
|
--------------------------------------------------
|
|
DELETE books
|
|
DELETE my-explicit-mappings-books
|
|
--------------------------------------------------
|
|
// TEARDOWN
|
|
|
|
////
|
|
|
|
[discrete]
|
|
[[getting-started-index-creation]]
|
|
=== Step 1: Create an index
|
|
|
|
Create a new index named `books`:
|
|
|
|
[source,console]
|
|
----
|
|
PUT /books
|
|
----
|
|
// TEST[skip: index already setup]
|
|
|
|
The following response indicates the index was created successfully.
|
|
|
|
.Example response
|
|
[%collapsible]
|
|
===============
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"acknowledged": true,
|
|
"shards_acknowledged": true,
|
|
"index": "books"
|
|
}
|
|
----
|
|
// TEST[skip: index already setup]
|
|
===============
|
|
|
|
[discrete]
|
|
[[getting-started-add-documents]]
|
|
=== Step 2: Add data to your index
|
|
|
|
[TIP]
|
|
====
|
|
This tutorial uses {es} APIs, but there are many other ways to
|
|
<<es-ingestion-overview,add data to {es}>>.
|
|
====
|
|
|
|
You add data to {es} as JSON objects called documents.
|
|
{es} stores these
|
|
documents in searchable indices.
|
|
|
|
[discrete]
|
|
[[getting-started-add-single-document]]
|
|
==== Add a single document
|
|
|
|
Submit the following indexing request to add a single document to the
|
|
`books` index.
|
|
|
|
[TIP]
|
|
====
|
|
If the index didn't already exist, this request would automatically create it.
|
|
====
|
|
|
|
[source,console]
|
|
----
|
|
POST books/_doc
|
|
{
|
|
"name": "Snow Crash",
|
|
"author": "Neal Stephenson",
|
|
"release_date": "1992-06-01",
|
|
"page_count": 470
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
The response includes metadata that {es} generates for the document, including a unique `_id` for the document within the index.
|
|
|
|
.Example response
|
|
[%collapsible]
|
|
===============
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"_index": "books", <1>
|
|
"_id": "O0lG2IsBaSa7VYx_rEia", <2>
|
|
"_version": 1, <3>
|
|
"result": "created", <4>
|
|
"_shards": { <5>
|
|
"total": 2, <6>
|
|
"successful": 2, <7>
|
|
"failed": 0 <8>
|
|
},
|
|
"_seq_no": 0, <9>
|
|
"_primary_term": 1 <10>
|
|
}
|
|
----
|
|
// TEST[s/O0lG2IsBaSa7VYx_rEia/*/]
|
|
<1> The `_index` field indicates the index the document was added to.
|
|
<2> The `_id` field is the unique identifier for the document.
|
|
<3> The `_version` field indicates the version of the document.
|
|
<4> The `result` field indicates the result of the indexing operation.
|
|
<5> The `_shards` field contains information about the number of <<scalability,shards>> that the indexing operation was executed on and the number that succeeded.
|
|
<6> The `total` field indicates the total number of shards for the index.
|
|
<7> The `successful` field indicates the number of shards that the indexing operation was executed on.
|
|
<8> The `failed` field indicates the number of shards that failed during the indexing operation. '0' indicates no failures.
|
|
<9> The `_seq_no` field holds a monotonically increasing number incremented for each indexing operation on a shard.
|
|
<10> The `_primary_term` field is a monotonically increasing number incremented each time a primary shard is assigned to a different node.
|
|
===============
|
|
|
|
[discrete]
|
|
[[getting-started-add-multiple-documents]]
|
|
==== Add multiple documents
|
|
|
|
Use the <<docs-bulk,`_bulk` endpoint>> to add multiple documents in one request. Bulk data
|
|
must be formatted as newline-delimited JSON (NDJSON).
|
|
|
|
[source,console]
|
|
----
|
|
POST /_bulk
|
|
{ "index" : { "_index" : "books" } }
|
|
{"name": "Revelation Space", "author": "Alastair Reynolds", "release_date": "2000-03-15", "page_count": 585}
|
|
{ "index" : { "_index" : "books" } }
|
|
{"name": "1984", "author": "George Orwell", "release_date": "1985-06-01", "page_count": 328}
|
|
{ "index" : { "_index" : "books" } }
|
|
{"name": "Fahrenheit 451", "author": "Ray Bradbury", "release_date": "1953-10-15", "page_count": 227}
|
|
{ "index" : { "_index" : "books" } }
|
|
{"name": "Brave New World", "author": "Aldous Huxley", "release_date": "1932-06-01", "page_count": 268}
|
|
{ "index" : { "_index" : "books" } }
|
|
{"name": "The Handmaids Tale", "author": "Margaret Atwood", "release_date": "1985-06-01", "page_count": 311}
|
|
----
|
|
// TEST[continued]
|
|
|
|
You should receive a response indicating there were no errors.
|
|
|
|
.Example response
|
|
[%collapsible]
|
|
===============
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"errors": false,
|
|
"took": 29,
|
|
"items": [
|
|
{
|
|
"index": {
|
|
"_index": "books",
|
|
"_id": "QklI2IsBaSa7VYx_Qkh-",
|
|
"_version": 1,
|
|
"result": "created",
|
|
"_shards": {
|
|
"total": 2,
|
|
"successful": 2,
|
|
"failed": 0
|
|
},
|
|
"_seq_no": 1,
|
|
"_primary_term": 1,
|
|
"status": 201
|
|
}
|
|
},
|
|
{
|
|
"index": {
|
|
"_index": "books",
|
|
"_id": "Q0lI2IsBaSa7VYx_Qkh-",
|
|
"_version": 1,
|
|
"result": "created",
|
|
"_shards": {
|
|
"total": 2,
|
|
"successful": 2,
|
|
"failed": 0
|
|
},
|
|
"_seq_no": 2,
|
|
"_primary_term": 1,
|
|
"status": 201
|
|
}
|
|
},
|
|
{
|
|
"index": {
|
|
"_index": "books",
|
|
"_id": "RElI2IsBaSa7VYx_Qkh-",
|
|
"_version": 1,
|
|
"result": "created",
|
|
"_shards": {
|
|
"total": 2,
|
|
"successful": 2,
|
|
"failed": 0
|
|
},
|
|
"_seq_no": 3,
|
|
"_primary_term": 1,
|
|
"status": 201
|
|
}
|
|
},
|
|
{
|
|
"index": {
|
|
"_index": "books",
|
|
"_id": "RUlI2IsBaSa7VYx_Qkh-",
|
|
"_version": 1,
|
|
"result": "created",
|
|
"_shards": {
|
|
"total": 2,
|
|
"successful": 2,
|
|
"failed": 0
|
|
},
|
|
"_seq_no": 4,
|
|
"_primary_term": 1,
|
|
"status": 201
|
|
}
|
|
},
|
|
{
|
|
"index": {
|
|
"_index": "books",
|
|
"_id": "RklI2IsBaSa7VYx_Qkh-",
|
|
"_version": 1,
|
|
"result": "created",
|
|
"_shards": {
|
|
"total": 2,
|
|
"successful": 2,
|
|
"failed": 0
|
|
},
|
|
"_seq_no": 5,
|
|
"_primary_term": 1,
|
|
"status": 201
|
|
}
|
|
}
|
|
]
|
|
}
|
|
----
|
|
// TEST[skip:TODO]
|
|
===============
|
|
|
|
[discrete]
|
|
[[getting-started-mappings-and-data-types]]
|
|
=== Step 3: Define mappings and data types
|
|
|
|
<<elasticsearch-intro-documents-fields-mappings,Mappings>> define how data is stored and indexed in {es}, like a schema in a relational database.
|
|
|
|
[discrete]
|
|
[[getting-started-dynamic-mapping]]
|
|
==== Use dynamic mapping
|
|
|
|
When using dynamic mapping, {es} automatically creates mappings for new fields by default.
|
|
The documents we've added so far have used dynamic mapping, because we didn't specify a mapping when creating the index.
|
|
|
|
To see how dynamic mapping works, add a new document to the `books` index with a field that doesn't appear in the existing documents.
|
|
|
|
[source,console]
|
|
----
|
|
POST /books/_doc
|
|
{
|
|
"name": "The Great Gatsby",
|
|
"author": "F. Scott Fitzgerald",
|
|
"release_date": "1925-04-10",
|
|
"page_count": 180,
|
|
"language": "EN" <1>
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
<1> The new field.
|
|
|
|
View the mapping for the `books` index with the <<indices-get-mapping, Get mapping API>>. The new field `new_field` has been added to the mapping with a `text` data type.
|
|
|
|
[source,console]
|
|
----
|
|
GET /books/_mapping
|
|
----
|
|
// TEST[continued]
|
|
|
|
.Example response
|
|
[%collapsible]
|
|
===============
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"books": {
|
|
"mappings": {
|
|
"properties": {
|
|
"author": {
|
|
"type": "text",
|
|
"fields": {
|
|
"keyword": {
|
|
"type": "keyword",
|
|
"ignore_above": 256
|
|
}
|
|
}
|
|
},
|
|
"name": {
|
|
"type": "text",
|
|
"fields": {
|
|
"keyword": {
|
|
"type": "keyword",
|
|
"ignore_above": 256
|
|
}
|
|
}
|
|
},
|
|
"new_field": {
|
|
"type": "text",
|
|
"fields": {
|
|
"keyword": {
|
|
"type": "keyword",
|
|
"ignore_above": 256
|
|
}
|
|
}
|
|
},
|
|
"page_count": {
|
|
"type": "long"
|
|
},
|
|
"release_date": {
|
|
"type": "date"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
===============
|
|
|
|
[discrete]
|
|
[[getting-started-explicit-mapping]]
|
|
==== Define explicit mapping
|
|
|
|
Create an index named `my-explicit-mappings-books` with explicit mappings.
|
|
Pass each field's properties as a JSON object. This object should contain the <<mapping-types,field data type>> and any additional <<mapping-params,mapping parameters>>.
|
|
|
|
[source,console]
|
|
----
|
|
PUT /my-explicit-mappings-books
|
|
{
|
|
"mappings": {
|
|
"dynamic": false, <1>
|
|
"properties": { <2>
|
|
"name": { "type": "text" },
|
|
"author": { "type": "text" },
|
|
"release_date": { "type": "date", "format": "yyyy-MM-dd" },
|
|
"page_count": { "type": "integer" }
|
|
}
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
<1> Disables dynamic mapping for the index. Documents containing fields not defined in the mapping will be rejected.
|
|
<2> The `properties` object defines the fields and their data types for documents in this index.
|
|
|
|
.Example response
|
|
[%collapsible]
|
|
===============
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"acknowledged": true,
|
|
"shards_acknowledged": true,
|
|
"index": "my-explicit-mappings-books"
|
|
}
|
|
----
|
|
// TEST[skip:already created in setup]
|
|
===============
|
|
|
|
[discrete]
|
|
[[getting-started-combined-mapping]]
|
|
==== Combine dynamic and explicit mappings
|
|
|
|
Explicit mappings are defined at index creation, and documents must conform to these mappings.
|
|
You can also use the <<indices-put-mapping,Update mapping API>>.
|
|
When an index has the `dynamic` flag set to `true`, you can add new fields to documents without updating the mapping.
|
|
|
|
This allows you to combine explicit and dynamic mappings.
|
|
Learn more about <<mapping-manage-update,managing and updating mappings>>.
|
|
|
|
[discrete]
|
|
[[getting-started-search-data]]
|
|
=== Step 4: Search your index
|
|
|
|
Indexed documents are available for search in near real-time, using the <<search-your-data,`_search` API>>.
|
|
// TODO: You'll find more detailed quick start guides in TODO
|
|
|
|
[discrete]
|
|
[[getting-started-search-all-documents]]
|
|
==== Search all documents
|
|
|
|
Run the following command to search the `books` index for all documents:
|
|
|
|
[source,console]
|
|
----
|
|
GET books/_search
|
|
----
|
|
// TEST[continued]
|
|
|
|
.Example response
|
|
[%collapsible]
|
|
===============
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"took": 2, <1>
|
|
"timed_out": false, <2>
|
|
"_shards": { <3>
|
|
"total": 5,
|
|
"successful": 5,
|
|
"skipped": 0,
|
|
"failed": 0
|
|
},
|
|
"hits": { <4>
|
|
"total": { <5>
|
|
"value": 7,
|
|
"relation": "eq"
|
|
},
|
|
"max_score": 1, <6>
|
|
"hits": [
|
|
{
|
|
"_index": "books", <7>
|
|
"_id": "CwICQpIBO6vvGGiC_3Ls", <8>
|
|
"_score": 1, <9>
|
|
"_source": { <10>
|
|
"name": "Brave New World",
|
|
"author": "Aldous Huxley",
|
|
"release_date": "1932-06-01",
|
|
"page_count": 268
|
|
}
|
|
},
|
|
... (truncated)
|
|
]
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
<1> The `took` field indicates the time in milliseconds for {es} to execute the search
|
|
<2> The `timed_out` field indicates whether the search timed out
|
|
<3> The `_shards` field contains information about the number of <<glossary,shards>> that the search was executed on and the number that succeeded
|
|
<4> The `hits` object contains the search results
|
|
<5> The `total` object provides information about the total number of matching documents
|
|
<6> The `max_score` field indicates the highest relevance score among all matching documents
|
|
<7> The `_index` field indicates the index the document belongs to
|
|
<8> The `_id` field is the document's unique identifier
|
|
<9> The `_score` field indicates the relevance score of the document
|
|
<10> The `_source` field contains the original JSON object submitted during indexing
|
|
===============
|
|
|
|
[discrete]
|
|
[[getting-started-match-query]]
|
|
==== `match` query
|
|
|
|
You can use the <<query-dsl-match-query,`match` query>> to search for documents that contain a specific value in a specific field.
|
|
This is the standard query for full-text searches.
|
|
|
|
Run the following command to search the `books` index for documents containing `brave` in the `name` field:
|
|
|
|
[source,console]
|
|
----
|
|
GET books/_search
|
|
{
|
|
"query": {
|
|
"match": {
|
|
"name": "brave"
|
|
}
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
|
|
.Example response
|
|
[%collapsible]
|
|
===============
|
|
[source,console-result]
|
|
----
|
|
{
|
|
"took": 9,
|
|
"timed_out": false,
|
|
"_shards": {
|
|
"total": 5,
|
|
"successful": 5,
|
|
"skipped": 0,
|
|
"failed": 0
|
|
},
|
|
"hits": {
|
|
"total": {
|
|
"value": 1,
|
|
"relation": "eq"
|
|
},
|
|
"max_score": 0.6931471, <1>
|
|
"hits": [
|
|
{
|
|
"_index": "books",
|
|
"_id": "CwICQpIBO6vvGGiC_3Ls",
|
|
"_score": 0.6931471,
|
|
"_source": {
|
|
"name": "Brave New World",
|
|
"author": "Aldous Huxley",
|
|
"release_date": "1932-06-01",
|
|
"page_count": 268
|
|
}
|
|
}
|
|
]
|
|
}
|
|
}
|
|
----
|
|
// TEST[continued]
|
|
<1> The `max_score` is the score of the highest-scoring document in the results. In this case, there is only one matching document, so the `max_score` is the score of that document.
|
|
===============
|
|
|
|
[discrete]
|
|
[[getting-started-delete-indices]]
|
|
=== Step 5: Delete your indices (optional)
|
|
|
|
When following along with examples, you might want to delete an index to start from scratch.
|
|
You can delete indices using the <<indices-delete-index,Delete index API>>.
|
|
|
|
For example, run the following command to delete the indices created in this tutorial:
|
|
|
|
[source,console]
|
|
----
|
|
DELETE /books
|
|
DELETE /my-explicit-mappings-books
|
|
----
|
|
// TEST[skip:handled by setup/teardown]
|
|
|
|
[CAUTION]
|
|
====
|
|
Deleting an index permanently deletes its documents, shards, and metadata.
|
|
====
|