logstash/docs/static/plugin-manager.asciidoc

[[working-with-plugins]]
== Working with plugins

[IMPORTANT]
.macOS Gatekeeper warnings
====
Apple's rollout of stricter notarization requirements affected the notarization
of the {version} {ls} artifacts. If macOS Catalina displays a dialog when you
first run {ls}, you need to take an action to allow it to run.
To prevent Gatekeeper checks on the {ls} files, run the following command on the
downloaded `.tar.gz` archive or the directory to which was extracted:
[source,sh]
----
xattr -d -r com.apple.quarantine <archive-or-directory>
----
For example, if the `.tar.gz` file was extracted to the default
logstash-{version} directory, the command is:
[source,sh,subs="attributes"]
----
xattr -d -r com.apple.quarantine logstash-{version}
----
Alternatively, you can add a security override if a Gatekeeper popup appears by
following the instructions in the _How to open an app that hasn’t been notarized
or is from an unidentified developer_ section of
https://support.apple.com/en-us/HT202491[Safely open apps on your Mac].
====


Logstash has a rich collection of input, filter, codec, and output plugins.
Check out the https://www.elastic.co/support/matrix#matrix_logstash_plugins[Elastic Support Matrix] 
to see which plugins are supported at various levels. 

Plugins are available in self-contained packages called gems and hosted on
https://rubygems.org/[RubyGems.org]. Use the plugin manager
script--`bin/logstash-plugin`--to manage plugins:  

* <<listing-plugins>>
* <<installing-plugins>>
* <<updating-plugins>>
* <<removing-plugins>>
* <<installing-local-plugins>>
* <<installing-local-plugins-path>>

[discrete]
[[pointer-to-offline]]
=== No internet connection? 

If you don't have an internet connection, check out <<offline-plugins>> for
information on <<building-offline-packs,building>>,
<<installing-offline-packs,installing>>, and <<updating-offline-packs,updating>>
offline plugin packs.

[discrete]
[[http-proxy]]
==== Proxy configuration

Most plugin manager commands require access to the internet to reach https://rubygems.org[RubyGems.org].
If your organization is behind a firewall, you can set these environments variables to configure Logstash to use your proxy.

[source, shell]
----------------------------------
export http_proxy=http://localhost:3128
export https_proxy=http://localhost:3128
----------------------------------

[discrete]
[[listing-plugins]]
=== Listing plugins

Logstash release packages bundle common plugins. To list the plugins currently
available in your deployment:

[source,shell]
----------------------------------
bin/logstash-plugin list <1>
bin/logstash-plugin list --verbose <2>
bin/logstash-plugin list '*namefragment*' <3>
bin/logstash-plugin list --group output <4>
----------------------------------
<1> Lists all installed plugins
<2> Lists installed plugins with version information
<3> Lists all installed plugins containing a namefragment
<4> Lists all installed plugins for a particular group (input, filter, codec, output)

[discrete]
[[installing-plugins]]
=== Adding plugins to your deployment

When you have access to internet, you can retrieve plugins hosted on the
https://rubygems.org/[RubyGems.org]public repository and install them on top of
your Logstash installation.

[source,shell]
----------------------------------
bin/logstash-plugin install logstash-input-github
----------------------------------

After a plugin is successfully installed, you can use it in your configuration file.

[discrete]
[[updating-plugins]]
=== Updating plugins

Plugins have their own release cycles and are often released independently of Logstash’s core release cycle. 
Using the update subcommand you can get the latest version of the plugin.

[source,shell]
----------------------------------
bin/logstash-plugin update <1>
bin/logstash-plugin update logstash-input-github <2>
----------------------------------
<1> updates all installed plugins
<2> updates only the plugin you specify

[discrete]
[[updating-major]]
==== Major version plugin updates

To avoid introducing breaking changes, the plugin manager updates only plugins for which newer _minor_ or _patch_ versions exist by default.
If you wish to also include breaking changes, specify `--level=major`.

[source,shell]
----------------------------------
bin/logstash-plugin update --level=major <1>
bin/logstash-plugin update --level=major logstash-input-github <2>
----------------------------------
<1> updates all installed plugins to latest, including major versions with breaking changes
<2> updates only the plugin you specify to latest, including major versions with breaking changes


[discrete]
[[removing-plugins]]
=== Removing plugins

If you need to remove plugins from your Logstash installation:

[source,shell]
----------------------------------
bin/logstash-plugin remove logstash-input-github
----------------------------------

[discrete]
[[installing-local-plugins]]
==== Advanced: Adding a locally built plugin

In some cases, you may want to install plugins which are not yet released and
not hosted on RubyGems.org. Logstash provides you the option to install a
locally built plugin which is packaged as a ruby gem. Using a file location:

[source,shell]
----------------------------------
bin/logstash-plugin install /path/to/logstash-output-kafka-1.0.0.gem
----------------------------------

[discrete]
[[installing-local-plugins-path]]
==== Advanced: Using `--path.plugins`

Using the Logstash `--path.plugins` flag, you can load a plugin source code located on your file system. Typically this is used by
developers who are iterating on a custom plugin and want to test it before creating a ruby gem.

The path needs to be in a  specific directory hierarchy: `PATH/logstash/TYPE/NAME.rb`, where TYPE is 'inputs' 'filters', 'outputs' or 'codecs' and NAME is the name of the plugin.

[source,shell]
----------------------------------
# supposing the code is in /opt/shared/lib/logstash/inputs/my-custom-plugin-code.rb
bin/logstash --path.plugins /opt/shared/lib
----------------------------------


include::cross-plugin-concepts.asciidoc[]

include::plugin-generator.asciidoc[]

include::offline-plugins.asciidoc[]

include::private-gem-repo.asciidoc[]

include::event-api.asciidoc[]