From 3a7d46da4f817653ed0db4e84d14a8fc4e75566a Mon Sep 17 00:00:00 2001 From: Karen Metts Date: Mon, 1 Apr 2019 13:56:09 -0400 Subject: [PATCH] Update instructions for 7.0 upgrade Fixes #10621 --- docs/static/upgrading.asciidoc | 115 +++++++++++++++++++++++++-------- 1 file changed, 89 insertions(+), 26 deletions(-) diff --git a/docs/static/upgrading.asciidoc b/docs/static/upgrading.asciidoc index 6a7863d0f..cc0a4bddb 100644 --- a/docs/static/upgrading.asciidoc +++ b/docs/static/upgrading.asciidoc @@ -6,6 +6,7 @@ Before upgrading Logstash: * Consult the <> docs. +* Read the <>. * Test upgrades in a development environment before upgrading your production cluster. While upgrading Logstash: @@ -16,16 +17,15 @@ and becomes a new node in the monitoring data. =========================================== If you're upgrading other products in the stack, also read the -{stack-ref}/index.html[Elastic Stack Installation and Upgrade Guide]. Want an -upgrade list that's tailored to your stack? Try out our -{upgrade_guide}[Interactive Upgrade Guide]. +{stack-ref}/index.html[Elastic Stack Installation and Upgrade Guide]. See the following topics for information about upgrading Logstash: * <> * <> +* <> +* <> * <> -* <> [[upgrading-using-package-managers]] === Upgrading Using Package Managers @@ -33,12 +33,12 @@ See the following topics for information about upgrading Logstash: This procedure uses <> to upgrade Logstash. 1. Shut down your Logstash pipeline, including any inputs that send events to Logstash. -2. Using the directions in the _Package Repositories_ section, update your repository links to point to the 5.x repositories -instead of the previous version. +2. Using the directions in the <> section, update your repository +links to point to the 7.x repositories. 3. Run the `apt-get upgrade logstash` or `yum update logstash` command as appropriate for your operating system. 4. Test your configuration file with the `logstash --config.test_and_exit -f ` command. Configuration options for -some Logstash plugins have changed in the 5.x release. -5. Restart your Logstash pipeline after updating your configuration file. +some Logstash plugins have changed in the 7.x release. +5. Restart your Logstash pipeline after you have updated your configuration file. [[upgrading-using-direct-download]] === Upgrading Using a Direct Download @@ -48,12 +48,70 @@ This procedure downloads the relevant Logstash binaries directly from Elastic. 1. Shut down your Logstash pipeline, including any inputs that send events to Logstash. 2. Download the https://www.elastic.co/downloads/logstash[Logstash installation file] that matches your host environment. 3. Unpack the installation file into your Logstash directory. -4. Test your configuration file with the `logstash --config.test_and_exit -f ` command. Configuration options for -some Logstash plugins have changed in the 5.x release. +4. Test your configuration file with the `logstash --config.test_and_exit -f ` command. +Configuration options for +some Logstash plugins have changed in the 7.x release. 5. Restart your Logstash pipeline after updating your configuration file. + + +[[upgrading-minor-versions]] +=== Upgrading between minor versions + +As a general rule, you can upgrade between minor versions (for example, 7.x to +7.y, where x < y) by simply installing the new release and restarting {ls}. +{ls} typically maintains backwards compatibility for configuration +settings and exported fields. Please review the +<> for potential exceptions. + +Upgrading between non-consecutive major versions (5.x to 7.x, for example) is not +supported. + [[upgrading-logstash-pqs]] -=== Upgrading with Persistent Queues Enabled +=== Upgrading with the Persistent Queue Enabled + +If you have the persistent queue (PQ) enabled, please read the section that applies +for your upgrade scenario. + +* If you are upgrading from version 6.2.x or earlier, we recommend that you +<> before you upgrade. + +* If you are upgrading from version 6.3.0 or later, see +<> for information. + +[float] +[[drain-pq]] +==== Drain the Persistent Queue (version 6.2.x and earlier) + +The following applies only if you are upgrading from Logstash version 6.2.x or +earlier with the persistent queue (PQ) enabled. + +We strive to maintain backward compatibility within a given major release. +Serialization issues in Logstash 6.2.x and earlier required us to break +that compatibility in version 6.3.0 to ensure correctness of operation. For more +technical details, please check our tracking github issue for this +matter, https://github.com/elastic/logstash/issues/9494[#9494]. + +We strongly recommend that you drain or delete +the persistent queue before you upgrade from version 6.2.x and earlier. + +To drain the queue: + +. In the logstash.yml file, set `queue.drain:true`. +. Restart Logstash for this setting to take effect. +. Shutdown Logstash (using CTRL+C or SIGTERM), and wait for the queue to empty. + +When the queue is empty: + +. Complete the upgrade. +. Restart Logstash. + +We have resolved issues with data incompatibilities for version 6.3 and later. +These steps won’t be required for future upgrades. + +[float] +[[upgrading-logstash-pqs-6.3]] +==== Upgrading from version 6.3 (and later) with Persistent Queues Enabled Upgrading Logstash with persistent queues enabled is supported. The persistent queue directory is self-contained and can be read by a new Logstash instance @@ -66,37 +124,42 @@ Keep in mind that only one Logstash instance can write to `path.queue`. You cannot have the original instance and the new instance writing to the queue at the same time. -[[upgrading-logstash-5.0]] -=== Upgrading Logstash to 5.0 +[[upgrading-logstash-7.0]] +=== Upgrading Logstash to 7.0 -Before upgrading Logstash, remember to read the <>. +Before upgrading Logstash, remember to read the <> docs and the <>. If you are installing Logstash with other components in the Elastic Stack, also see the {stack-ref}/index.html[Elastic Stack installation and upgrade documentation]. +NOTE: Upgrading between non-consecutive major versions (5.x to 7.x, for example) is not +supported. We recommend that you upgrade to 6.x, and then upgrade to 7.x. + ==== When to Upgrade Fresh installations can and should start with the same version across the Elastic Stack. -Elasticsearch 5.0 does not require Logstash 5.0. An Elasticsearch 5.0 cluster will happily receive data from a -Logstash 2.x instance via the default HTTP communication layer. This provides some flexibility to decide when to upgrade -Logstash relative to an Elasticsearch upgrade. It may or may not be convenient for you to upgrade them together, and it -is -not required to be done at the same time as long as Elasticsearch is upgraded first. +Elasticsearch 7.0 does not require Logstash 7.0. An Elasticsearch 7.0 cluster +will happily receive data from earlier versions of Logstash via the default +HTTP communication layer. This provides some flexibility to decide when to +upgrade Logstash relative to an Elasticsearch upgrade. It may or may not be +convenient for you to upgrade them together, and it is not required to be done +at the same time as long as Elasticsearch is upgraded first. -You should upgrade in a timely manner to get the performance improvements that come with Logstash 5.0, but do so in +You should upgrade in a timely manner to get the performance improvements that come with Logstash 7.0, but do so in the way that makes the most sense for your environment. ==== When Not to Upgrade -If any Logstash plugin that you require is not compatible with Logstash 5.0, then you should wait until it is ready +If any Logstash plugin that you require is not compatible with Logstash 7.0, then you should wait until it is ready before upgrading. -Although we make great efforts to ensure compatibility, Logstash 5.0 is not completely backwards compatible. As noted -in the Elastic Stack upgrade guide, Logstash 5.0 should not be upgraded before Elasticsearch 5.0. This is both -practical and because some Logstash 5.0 plugins may attempt to use features of Elasticsearch 5.0 that did not exist -in earlier versions. For example, if you attempt to send the 5.x template to a cluster before Elasticsearch 5.0, then it -will not be able to use it and all indexing will fail likely fail. If you use your own, custom template with Logstash, +Although we make great efforts to ensure compatibility, Logstash 7.0 is not completely backwards compatible. As noted +in the Elastic Stack upgrade guide, Logstash 7.0 should not be upgraded before Elasticsearch 7.0. This is both +practical and because some Logstash 7.0 plugins may attempt to use features of Elasticsearch 7.0 that did not exist +in earlier versions. For example, if you attempt to send the 7.x template to a cluster before Elasticsearch 7.0, then +all indexing will fail likely fail. If you use your own custom template with Logstash, then this issue can be ignored. Note the Elasticsearch Output Index Template change in the <> documentation for further insight into