github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-24 06:37:19 -04:00
Code Issues Projects Releases Packages Wiki Activity

Fix renotes for 5.0

Browse source 
Fix typo

Fixes #6132
This commit is contained in:
DeDe Morton 2016-10-26 10:09:43 -07:00
parent 39fa1904d7
commit b0c54b135f
2 changed files with 57 additions and 110 deletions
Download patch file Download diff file Expand all files Collapse all files

47
docs/static/breaking-changes.asciidoc vendored
View file

@ -8,7 +8,8 @@ This section discusses the changes that you need to be aware of when migrating y
These changes can impact any instance of Logstash and are plugin agnostic, but only if you are using the features that are impacted.
Application Settings::
[float]
==== Application Settings
[IMPORTANT]
Logstash 5.0 introduces a new way to <<logstash-settings-file, configure application settings>> for Logstash through a
@ -23,7 +24,8 @@ after installing it via a package (RPM, DEB).
bin/logstash --path.settings /path/to/settings.yml
----------------------------------
Release Packages::
[float]
==== Release Packages
When Logstash 5.0 is installed via DEB or RPM packages, it now uses `/usr/share/logstash` and `/var/lib/logstash` to
install binaries. Previously it used to install in `/opt/logstash` directory. This change was done to make the user experience consistent with other products in the Elastic Stack.
@ -42,7 +44,8 @@ install binaries. Previously it used to install in `/opt/logstash` directory. Th
A complete directory layout is described in <<dir-layout>>. This will likely impact any scripts that you may have written
to support installing or manipulating Logstash, such as via Puppet.
Default Logging Level::
[float]
==== Default Logging Level
The default log severity level changed to `INFO` instead of `WARN` to match Elasticsearch. Existing logs
(in core and plugins) were too noisy at the `INFO` level, so we auditted our log messages and switched some of them to
@ -56,7 +59,8 @@ You can use the new `settings.yml` file to configure the `log.level` setting or
bin/logstash --log.level warn
----------------------------------
Plugin Manager Renamed::
[float]
==== Plugin Manager Renamed
`bin/plugin` has been renamed to `bin/logstash-plugin`. This occurred in Logstash 2.3 and it was mainly prevent `PATH` being
polluted when other components of the Elastic Stack are installed on the same machine. Also, this provides a foundation
@ -79,7 +83,8 @@ Like earlier releases of Logstash, most plugins are bundled directly with Logsta
while upgrading from earlier Logstash releases. However, if you are attempting to install a non-bundled plugin, then make
sure that it supports Logstash 5.0 before upgrading!
Logstash with All Plugins Download::
[float]
==== Logstash with All Plugins Download
The Logstash All Plugins download option has been removed. For users previously using this option as a convenience for
offline plugin management purposes (air-gapped environments), please see the <<offline-plugins>> documentation page.
@ -104,7 +109,8 @@ There were 17 plugins removed from 5.0 default bundle. These plugins can still b
* logstash-output-opentsdb
* logstash-output-zeromq
Command Line Interface::
[float]
==== Command Line Interface
Some CLI Options changed in Logstash 5.0. If you were using the “long form” of the <<command-line-flags,options>>,
then this will impact the way that you launch Logstash. They were changed to match the `settings.yml` format used to
@ -134,14 +140,11 @@ NOTE: None of the short form options have changed!
[float]
=== Breaking Changes in Plugins
<<<<<<< HEAD
* **Elasticsearch Output Index Template:** The index template for 5.0 has been changed to reflect {ref}breaking_50_mapping_changes.html[Elasticsearch's mapping changes]. Most
=======
Elasticsearch Output Index Template::
[float]
==== Elasticsearch Output Index Template
The index template for Elasticsearch 5.0 has been changed to reflect
https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_mapping_changes.html[Elasticsearch's mapping changes]. Most
>>>>>>> 5442d06... Update Breaking Changes to provide extra details (#6100)
importantly, the subfield for string multi-fields has changed from `.raw` to `.keyword` to match Elasticsearch's default
behavior. The impact of this change to various user groups is detailed below:
@ -157,8 +160,9 @@ also uses the `.keyword` field, unless you are able to transition from `.raw` to
transition time. You _can_ use a custom template to get both `.raw` and `.keyword` so that you can wait until all `.raw` data
has stopped existing before transitioning to only using `.keyword`; this will waste some storage space and memory, but it does
help users to avoid having to relearn operations.
Plugin Versions::
[float]
==== Plugin Versions
Logstash is unique amongst the Elastic Stack with respect to its plugins. Unlike Elasticsearch and Kibana, which both
require plugins to be targeted to a specific release, Logstash’s plugin ecosystem provides more flexibility so that it can
@ -189,7 +193,8 @@ bin/logstash-plugin install --version 4.0.1 logstash-output-kafka
The version numbers were found by checking the compatibility matrix for the individual plugins.
File Input::
[float]
==== File Input
The <<plugins-inputs-file, File Input>> `SinceDB` file is now saved at `<path.data>/plugins/inputs/file` location,
where `path.data` is the path defined in the new `settings.yml` file.
@ -203,32 +208,18 @@ where `path.data` is the path defined in the new `settings.yml` file.
|`<path.data>/plugins/inputs/file`
|===
<<<<<<< HEAD
* **File Input:** SinceDB file is now saved in `<path.data>/plugins/inputs/file` location, not user's home. If you have manually specified `sincedb_path`
configuration, this change will not affect you. If you are moving from 2.x to 5.x, and would like to use the existing SinceDB file, it
has to be copied over to `path.data` manually to use the save state.
=======
If you have manually specified `sincedb_path` as part of the configuration, this change will not affect you.
If you are moving from Logstash 2.x to Logstash 5.0, and you would like to use the existing SinceDB file,
then it must be copied over to `path.data` manually to use the save state (or the path needs to be changed to point to it).
>>>>>>> 5442d06... Update Breaking Changes to provide extra details (#6100)
[float]
=== Ruby Filter and Custom Plugin Developers
With the migration to the new <<event-api>>, we have changed how you can access internal data compared to previous release.
<<<<<<< HEAD
The Event object no longer returns a reference to the data. Instead, it returns a copy. This might change how you do manipulation of
your data, especially when working with nested hashes. When working with nested hashes, it’s recommended that you
use the `fieldref` syntax instead of using multiple brackets. Also note that we have introduced new Getter/Setter APIs
for accessing information in the Event object. Refer <<event-api>> for details.
=======
The `event` object no longer returns a reference to the data. Instead, it returns a copy. This might change how you perform
manipulation of your data, especially when working with nested hashes. When working with nested hashes, it’s recommended that
you use the <<logstash-config-field-references, `field reference` syntax>> instead of using multiple square brackets.
As part of this change, Logstash has introduced new Getter/Setter APIs for accessing information in the `event` object.
>>>>>>> 5442d06... Update Breaking Changes to provide extra details (#6100)
**Examples:**

120
docs/static/upgrading.asciidoc vendored
View file

@ -9,41 +9,6 @@ Before upgrading Logstash:
* Test upgrades in a development environment before upgrading your production cluster.
===========================================
[float]
=== When to Upgrade
Fresh installations can and should start with the same version across the Elastic Stack.
Elasticsearch 5.0 does not require Logstash 5.0. An Elasticsearch 5.0 cluster will happily receive data from a
Logstash 2.x instance via the default HTTP communication layer. This provides some flexibility to decide when to upgrade
Logstash relative to an Elasticsearch upgrade. It may or may not be convenient for you to upgrade them together, and it
is
not required to be done at the same time as long as Elasticsearch is upgraded first.
You should upgrade in a timely manner to get the performance improvements that come with Logstash 5.0, but do so in
the way that makes the most sense for your environment.
[float]
=== When Not to Upgrade
If any Logstash plugin that you require is not compatible with Logstash 5.0, then you should wait until it is ready
before upgrading.
////
BEFORE 5.0 is released, we need to add the link to the Elastic Stack upgrade page.
////
Although we make great efforts to ensure compatibility, Logstash 5.0 is not completely backwards compatible. As noted
in the Elastic Stack upgrade guide, Logstash 5.0 should not be upgraded before Elasticsearch 5.0. This is both
practical and because some Logstash 5.0 plugins may attempt to use features of Elasticsearch 5.0 that did not exist
in earlier versions. For example, if you attempt to send the 5.x template to a cluster before Elasticsearch 5.0, then it
will not be able to use it and all indexing will fail likely fail. If you use your own, custom template with Logstash, t
hen
this issue can be ignored.
Note the Elasticsearch Output Index Template change in the <<breaking-changes>> documentation for further insight into
this change and how it impacts operations.
=== Upgrading Using Package Managers
This procedure uses <<package-repositories,package managers>> to upgrade Logstash.
@ -67,52 +32,6 @@ This procedure downloads the relevant Logstash binaries directly from Elastic.
some Logstash plugins have changed in the 2.0 release.
5. Restart your Logstash pipeline after updating your configuration file.
=== Upgrading Logstash and Elasticsearch to 2.0
If you are using Elasticsearch as an output, and wish to upgrade to Elasticsearch 2.0, please be
aware of https://www.elastic.co/guide/en/elasticsearch/reference/2.0/breaking-changes-2.0.html[breaking changes]
before you upgrade. In addition, the following steps needs to be performed after upgrading to Elasticsearch 2.0:
**Mapping changes:** Users may have custom template changes, so by default a Logstash upgrade will
leave the template as is. Even if you don't have a custom template, Logstash will not overwrite an existing
template by default.
There is one known issue (removal of https://www.elastic.co/guide/en/elasticsearch/reference/1.4/mapping-object-type.html#_path_3[path]) with using GeoIP filter that needs a manual update to the template.
Note: If you have custom template changes, please make sure to save it and merge any changes. You can
get the existing template by running:
[source,shell]
curl -XGET localhost:9200/_template/logstash
Add the following option to your Logstash config:
[source,json]
output {
elasticsearch {
template_overwrite => true
}
}
Restart Logstash.
**Dots in fields:** Elasticsearch 2.0 does not allow field names to contain the `.` character.
Further details about this change https://www.elastic.co/guide/en/elasticsearch/reference/2.0/breaking_20_mapping_changes.html#_field_names_may_not_contain_dots[here]. Some plugins already have been updated to compensate
for this breaking change, including logstash-filter-metrics and logstash-filter-elapsed.
These plugin updates are available for Logstash 2.0. To upgrade to the latest version of these
plugins, the command is:
[source,shell]
bin/logstash-plugin update <plugin_name>
**Multiline Filter:** If you are using the Multiline Filter in your configuration and upgrade to Logstash 2.0,
you will get an error. Make sure to explicitly set the number of filter workers (`-w`) to `1`. You can set the number
of workers by passing a command line flag such as:
[source,shell]
bin/logstash -w 1
[[upgrading-logstash-2.2]]
=== Upgrading Logstash to 2.2
@ -141,5 +60,42 @@ Thus, in 2.2, you can safely set `-w` to a number which is a multiple of the num
A common way to tune performance is keep increasing the `-w` beyond the # of cores until performance no longer
improves. A note of caution - make sure you also keep heapsize in mind, because the number of in-flight events
are `#workers * batch_size * average_event size`. More in-flight events could add to memory pressure, eventually
leading to Out of Memory errors. You can change the heapsize in Logstash by setting `LS_HEAP_SIZE`
leading to Out of Memory errors. You can change the heapsize in Logstash by setting `LS_HEAP_SIZE`.
[[upgrading-logstash-5.0]]
=== Upgrading Logstash to 5.0
Before upgrading Logstash, remember to read the <<breaking-changes,breaking changes>>.
If you are installing Logstash with other components in the Elastic Stack, also see the
{stack}index.html[Elastic Stack installation and upgrade documentation].
==== When to Upgrade
Fresh installations can and should start with the same version across the Elastic Stack.
Elasticsearch 5.0 does not require Logstash 5.0. An Elasticsearch 5.0 cluster will happily receive data from a
Logstash 2.x instance via the default HTTP communication layer. This provides some flexibility to decide when to upgrade
Logstash relative to an Elasticsearch upgrade. It may or may not be convenient for you to upgrade them together, and it
is
not required to be done at the same time as long as Elasticsearch is upgraded first.
You should upgrade in a timely manner to get the performance improvements that come with Logstash 5.0, but do so in
the way that makes the most sense for your environment.
==== When Not to Upgrade
If any Logstash plugin that you require is not compatible with Logstash 5.0, then you should wait until it is ready
before upgrading.
Although we make great efforts to ensure compatibility, Logstash 5.0 is not completely backwards compatible. As noted
in the Elastic Stack upgrade guide, Logstash 5.0 should not be upgraded before Elasticsearch 5.0. This is both
practical and because some Logstash 5.0 plugins may attempt to use features of Elasticsearch 5.0 that did not exist
in earlier versions. For example, if you attempt to send the 5.x template to a cluster before Elasticsearch 5.0, then it
will not be able to use it and all indexing will fail likely fail. If you use your own, custom template with Logstash,
then this issue can be ignored.
Note the Elasticsearch Output Index Template change in the <<breaking-changes>> documentation for further insight into
this change and how it impacts operations.