github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-24 06:37:19 -04:00
Code Issues Projects Releases Packages Wiki Activity

Add notes about doc type bug in ES output to upgrading guide

Browse source 
This is the workaround for https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/708

Fixes #8708
This commit is contained in:
Andrew Cholakian 2017-11-22 04:41:50 -06:00
parent 055de8e9c3
commit 908e2d46b6
1 changed files with 11 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

15
docs/static/upgrading.asciidoc vendored
View file

@ -66,12 +66,19 @@ Before upgrading Logstash, remember to read the <<breaking-changes,breaking chan
If you are installing Logstash with other components in the Elastic Stack, also see the
{stack-ref}/index.html[Elastic Stack installation and upgrade documentation].
If you are using the default mapping templates in Logstash, to continue using these after migrating Elasticsearch to 6.0, you must override the existing template with the 6.x template.
This can be done by starting a pipeline with the `overwrite_template => true` option in the Elasticsearch output definition in the Logstash config.
==== Logstash 6.0.0 `document_type` Handling Bug
Note that multiple doctypes are no longer supported in Elasticsearch 6.0. Please refer to
{ref}/removal-of-types.html[Removal of mapping types] and {ref}/breaking-changes.html[Breaking changes] for more information.
We’d like to alert users to a bug in Logstash 6.0.0 that can cause errors writing to Elasticsearch 6.0+ clusters. When the `type` field is set to more than one value per Elasticsearch index Logstash will encounter errors while indexing documents. Logstash 6.0 uses the value of the `type` field to set the Elasticsearch `type` by default. Elasticsearch 6 [no longer supports more than one type]((https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html) per index. In Logstash 6.0.1 we will fix this bug by changing the behavior so that when communicating with an Elasticsearch 6.0+ cluster Logstash will not set the document type by default, but rather, will set it to the value `doc`, regardless of whether an event’s `[type]` field has been set. The type information of a document will still be available under the field `type`.
The workaround for Logstash 6.0.0 is to add the setting `document_type => doc` to the Elasticsearch output configuration.
Please read on for more information about document types with Logstash and Elasticsearch 6.0
==== Handling Document Types in Elasticsearch 6.0+
As of Elasticsearch 6.0 document types are [on the way out](https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html) with only a single mapping type per index supported. For Logstash users this means transitioning to using the `type` field inside of the document instead of the document type. The effect is the same, but the usage is slightly different. This may mean reconfiguring existing Kibana dashboards to use this new field in lieu of the document type.
If you are using the default mapping templates in Logstash you will need to upgrade your mapping templates. To do this, after migrating Elasticsearch to 6.0, you must override the existing template with the 6.x template. This can be done by starting a pipeline with the `overwrite_template => true` option in the Elasticsearch output definition in the Logstash config.
==== When to Upgrade