mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 14:47:19 -04:00
Jordan Sissel
parent
4d5bdeb891
commit
378068173b
166 changed files with 0 additions and 35133 deletions
|
|
@ -1,30 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/dots
|
||||
layout: content_right
|
||||
---
|
||||
<h2>dots</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => dots {
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/dots.rb">lib/logstash/codecs/dots.rb</a>
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/json
|
||||
layout: content_right
|
||||
---
|
||||
<h2>json</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>This is the base class for logstash codecs.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => json {
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/json.rb">lib/logstash/codecs/json.rb</a>
|
||||
|
|
@ -1,45 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/json_spooler
|
||||
layout: content_right
|
||||
---
|
||||
<h2>json_spooler</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => json_spooler {
|
||||
<a href="#spool_size">spool_size</a> => ... # number (optional), default: 50
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="spool_size">
|
||||
spool_size
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 50 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/json_spooler.rb">lib/logstash/codecs/json_spooler.rb</a>
|
||||
|
|
@ -1,70 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/line
|
||||
layout: content_right
|
||||
---
|
||||
<h2>line</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>Line-oriented text data.</p>
|
||||
|
||||
<p>Decoding behavior: Only whole line events will be emitted.</p>
|
||||
|
||||
<p>Encoding behavior: Each event will be emitted with a trailing newline.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => line {
|
||||
<a href="#charset">charset</a> => ... # string, one of ["ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal"] (optional), default: "UTF-8"
|
||||
<a href="#format">format</a> => ... # string (optional)
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> Default value is "UTF-8" </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Set the desired text format for encoding.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/line.rb">lib/logstash/codecs/line.rb</a>
|
||||
|
|
@ -1,45 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/msgpack
|
||||
layout: content_right
|
||||
---
|
||||
<h2>msgpack</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => msgpack {
|
||||
<a href="#format">format</a> => ... # string (optional), default: nil
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/msgpack.rb">lib/logstash/codecs/msgpack.rb</a>
|
||||
|
|
@ -1,179 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/multiline
|
||||
layout: content_right
|
||||
---
|
||||
<h2>multiline</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>The multiline codec is for taking line-oriented text and merging them into a
|
||||
single event.</p>
|
||||
|
||||
<p>The original goal of this codec was to allow joining of multi-line messages
|
||||
from files into a single event. For example - joining java exception and
|
||||
stacktrace messages into a single event.</p>
|
||||
|
||||
<p>The config looks like this:</p>
|
||||
|
||||
<pre><code>input {
|
||||
stdin {
|
||||
codec => multiline {
|
||||
pattern => "pattern, a regexp"
|
||||
negate => true or false
|
||||
what => "previous" or "next"
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The 'pattern' should match what you believe to be an indicator that the field
|
||||
is part of a multi-line event.</p>
|
||||
|
||||
<p>The 'what' must be "previous" or "next" and indicates the relation
|
||||
to the multi-line event.</p>
|
||||
|
||||
<p>The 'negate' can be "true" or "false" (defaults false). If true, a
|
||||
message not matching the pattern will constitute a match of the multiline
|
||||
filter and the what will be applied. (vice-versa is also true)</p>
|
||||
|
||||
<p>For example, java stack traces are multiline and usually have the message
|
||||
starting at the far-left, then each subsequent line indented. Do this:</p>
|
||||
|
||||
<pre><code>input {
|
||||
stdin {
|
||||
codec => multiline {
|
||||
pattern => "^\s"
|
||||
what => "previous"
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>This says that any line starting with whitespace belongs to the previous line.</p>
|
||||
|
||||
<p>Another example is C line continuations (backslash). Here's how to do that:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
multiline {
|
||||
type => "somefiletype "
|
||||
pattern => "\\$"
|
||||
what => "next"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>This is the base class for logstash codecs.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => multiline {
|
||||
<a href="#charset">charset</a> => ... # string, one of ["ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal"] (optional), default: "UTF-8"
|
||||
<a href="#negate">negate</a> => ... # boolean (optional), default: false
|
||||
<a href="#pattern">pattern</a> => ... # string (required)
|
||||
<a href="#patterns_dir">patterns_dir</a> => ... # array (optional), default: []
|
||||
<a href="#what">what</a> => ... # string, one of ["previous", "next"] (required)
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> Default value is "UTF-8" </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="negate">
|
||||
negate
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Negate the regexp pattern ('if not matched')</p>
|
||||
|
||||
<h4>
|
||||
<a name="pattern">
|
||||
pattern (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The regular expression to match</p>
|
||||
|
||||
<h4>
|
||||
<a name="patterns_dir">
|
||||
patterns_dir
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>logstash ships by default with a bunch of patterns, so you don't
|
||||
necessarily need to define this yourself unless you are adding additional
|
||||
patterns.</p>
|
||||
|
||||
<p>Pattern files are plain text with format:</p>
|
||||
|
||||
<pre><code>NAME PATTERN
|
||||
</code></pre>
|
||||
|
||||
<p>For example:</p>
|
||||
|
||||
<pre><code>NUMBER \d+
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="what">
|
||||
what (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "previous", "next" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If the pattern matched, does event belong to the next or previous event?</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/multiline.rb">lib/logstash/codecs/multiline.rb</a>
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/noop
|
||||
layout: content_right
|
||||
---
|
||||
<h2>noop</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => noop {
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/noop.rb">lib/logstash/codecs/noop.rb</a>
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/oldlogstashjson
|
||||
layout: content_right
|
||||
---
|
||||
<h2>oldlogstashjson</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => oldlogstashjson {
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/oldlogstashjson.rb">lib/logstash/codecs/oldlogstashjson.rb</a>
|
||||
|
|
@ -1,72 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/plain
|
||||
layout: content_right
|
||||
---
|
||||
<h2>plain</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>The "plain" codec is for plain text with no delimiting between events.</p>
|
||||
|
||||
<p>This is mainly useful on inputs and outputs that already have a defined
|
||||
framing in their transport protocol (such as zeromq, rabbitmq, redis, etc)</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => plain {
|
||||
<a href="#charset">charset</a> => ... # string, one of ["ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal"] (optional), default: "UTF-8"
|
||||
<a href="#format">format</a> => ... # string (optional)
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> Default value is "UTF-8" </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Set the message you which to emit for each event. This supports sprintf
|
||||
strings.</p>
|
||||
|
||||
<p>This setting only affects outputs (encoding of events).</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/plain.rb">lib/logstash/codecs/plain.rb</a>
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/rubydebug
|
||||
layout: content_right
|
||||
---
|
||||
<h2>rubydebug</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => rubydebug {
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/rubydebug.rb">lib/logstash/codecs/rubydebug.rb</a>
|
||||
|
|
@ -1,45 +0,0 @@
|
|||
---
|
||||
title: logstash docs for codecs/spool
|
||||
layout: content_right
|
||||
---
|
||||
<h2>spool</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code># with an input plugin:
|
||||
# you can also use this codec with an output.
|
||||
input {
|
||||
file {
|
||||
codec => spool {
|
||||
<a href="#spool_size">spool_size</a> => ... # number (optional), default: 50
|
||||
}
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="spool_size">
|
||||
spool_size
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 50 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/codecs/spool.rb">lib/logstash/codecs/spool.rb</a>
|
||||
|
|
@ -1,247 +0,0 @@
|
|||
---
|
||||
title: Configuration Language - logstash
|
||||
layout: content_right
|
||||
---
|
||||
# LogStash Config Language
|
||||
|
||||
The logstash config language aims to be simple.
|
||||
|
||||
There's 3 main sections: inputs, filters, outputs. Each section has
|
||||
configurations for each plugin available in that section.
|
||||
|
||||
Example:
|
||||
|
||||
# This is a comment. You should use comments to describe
|
||||
# parts of your configuration.
|
||||
input {
|
||||
...
|
||||
}
|
||||
|
||||
filter {
|
||||
...
|
||||
}
|
||||
|
||||
output {
|
||||
...
|
||||
}
|
||||
|
||||
## Filters and Ordering
|
||||
|
||||
For a given event, are applied in the order of appearance in the
|
||||
configuration file.
|
||||
|
||||
## Comments
|
||||
|
||||
Comments are as in ruby, perl, and python. Starts with a '#' character. Example:
|
||||
|
||||
# this is a comment
|
||||
|
||||
input { # comments can appear at the end of a line, too
|
||||
# ...
|
||||
}
|
||||
|
||||
## Plugins
|
||||
|
||||
The input, filter, and output sections all let you configure plugins. Plugins
|
||||
configuration consists of the plugin name followed by a block of settings for
|
||||
that plugin. For example, how about two file inputs:
|
||||
|
||||
input {
|
||||
file {
|
||||
path => "/var/log/messages"
|
||||
type => "syslog"
|
||||
}
|
||||
|
||||
file {
|
||||
path => "/var/log/apache/access.log"
|
||||
type => "apache"
|
||||
}
|
||||
}
|
||||
|
||||
The above configures a two file separate inputs. Both set two
|
||||
configuration settings each: path and type. Each plugin has different
|
||||
settings for configuring it, seek the documentation for your plugin to
|
||||
learn what settings are available and what they mean. For example, the
|
||||
[file input][fileinput] documentation will explain the meanings of the
|
||||
path and type settings.
|
||||
|
||||
[fileinput]: inputs/file
|
||||
|
||||
## Value Types
|
||||
|
||||
The documentation for a plugin may say that a configuration field has a
|
||||
certain type. Examples include boolean, string, array, number, hash,
|
||||
etc.
|
||||
|
||||
### <a name="boolean"></a>Boolean
|
||||
|
||||
A boolean must be either `true` or `false`.
|
||||
|
||||
Examples:
|
||||
|
||||
debug => true
|
||||
|
||||
### <a name="string"></a>String
|
||||
|
||||
A string must be a single value.
|
||||
|
||||
Example:
|
||||
|
||||
name => "Hello world"
|
||||
|
||||
Single, unquoted words are valid as strings, too, but you should use quotes.
|
||||
|
||||
### <a name="number"></a>Number
|
||||
|
||||
Numbers must be valid numerics (floating point or integer are OK)
|
||||
|
||||
Example:
|
||||
|
||||
port => 33
|
||||
|
||||
### <a name="array"></a>Array
|
||||
|
||||
An array can be a single string value or multiple. If you specify the same
|
||||
field multiple times, it appends to the array.
|
||||
|
||||
Examples:
|
||||
|
||||
path => [ "/var/log/messages", "/var/log/*.log" ]
|
||||
path => "/data/mysql/mysql.log"
|
||||
|
||||
The above makes 'path' a 3-element array including all 3 strings.
|
||||
|
||||
### <a name="hash"></a>Hash
|
||||
|
||||
A hash is basically the same syntax as Ruby hashes.
|
||||
The key and value are simply pairs, such as:
|
||||
|
||||
match => { "field1" => "value1", "field2" => "value2", ... }
|
||||
|
||||
## <a name="fieldreferences"></a>Field References
|
||||
|
||||
All events have properties. For example, an apache access log would have things
|
||||
like status code, request path, http verb, client ip, etc. Logstash calls these
|
||||
properties "fields."
|
||||
|
||||
In many cases, it is useful to be able to refer to a field by name. To do this,
|
||||
you can use the logstash field reference syntax.
|
||||
|
||||
By way of example, let us suppose we have this event:
|
||||
|
||||
{
|
||||
"agent": "Mozilla/5.0 (compatible; MSIE 9.0)",
|
||||
"ip": "192.168.24.44",
|
||||
"request": "/index.html"
|
||||
"response": {
|
||||
"status": 200,
|
||||
"bytes": 52353
|
||||
},
|
||||
"ua": {
|
||||
"os": "Windows 7"
|
||||
}
|
||||
}
|
||||
|
||||
The syntax to access fields is `[fieldname]`. If you are only referring to a
|
||||
top-level field, you can omit the `[]` and simply say `fieldname`. In the case
|
||||
of nested fields,
|
||||
like the "os" field above, you need the full path to that field: `[ua][os]`.
|
||||
|
||||
## <a name="sprintf"></a>sprintf format
|
||||
|
||||
This syntax is also used in what logstash calls 'sprintf format'. This format
|
||||
allows you to refer to field values from within other strings. For example, the
|
||||
statsd output has an 'increment' setting, to allow you to keep a count of
|
||||
apache logs by status code:
|
||||
|
||||
output {
|
||||
statsd {
|
||||
increment => "apache.%{[response][status]}"
|
||||
}
|
||||
}
|
||||
|
||||
You can also do time formatting in this sprintf format. Instead of specifying a field name, use the `+FORMAT` syntax where `FORMAT` is a [time format](http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html).
|
||||
|
||||
For example, if you want to use the file output to write to logs based on the
|
||||
hour and the 'type' field:
|
||||
|
||||
output {
|
||||
file {
|
||||
path => "/var/log/%{type}.%{+yyyy.MM.dd.HH}"
|
||||
}
|
||||
}
|
||||
|
||||
## <a name="conditionals"></a>Conditionals
|
||||
|
||||
Sometimes you only want a filter or output to process an even under
|
||||
certain conditions. For that, you'll want to use a conditional!
|
||||
|
||||
Conditionals in logstash look and act the same way they do in programming
|
||||
languages. You have `if`, `else if` and `else` statements. Conditionals may be
|
||||
nested if you need that.
|
||||
|
||||
The syntax is follows:
|
||||
|
||||
if EXPRESSION {
|
||||
...
|
||||
} else if EXPRESSION {
|
||||
...
|
||||
} else {
|
||||
...
|
||||
}
|
||||
|
||||
What's an expression? Comparison tests, boolean logic, etc!
|
||||
|
||||
The following comparison operators are supported:
|
||||
|
||||
* equality, etc: == != < > <= >=
|
||||
* regexp: =~ !~
|
||||
* inclusion: in
|
||||
|
||||
The following boolean operators are supported:
|
||||
|
||||
* and, or, nand, xor
|
||||
|
||||
The following unary operators are supported:
|
||||
|
||||
* !
|
||||
|
||||
Expressions may contain expressions. Expressions may be negated with `!`.
|
||||
Expressions may be grouped with parentheses `(...)`.
|
||||
|
||||
For example, if we want to remove the field `secret` if the field
|
||||
`action` has a value of `login`:
|
||||
|
||||
filter {
|
||||
if [action] == "login" {
|
||||
mutate { remove => "secret" }
|
||||
}
|
||||
}
|
||||
|
||||
The above uses the field reference syntax to get the value of the
|
||||
`action` field. It is compared against the text `login` and, when equal,
|
||||
allows the mutate filter to do delete the field named `secret`
|
||||
|
||||
How about a more complex example?
|
||||
|
||||
* alert nagios of any apache events with status 5xx
|
||||
* record any 4xx status to elasticsearch
|
||||
* record all status code hits via statsd
|
||||
|
||||
How about telling nagios of any http event that has a status code of 5xx?
|
||||
|
||||
output {
|
||||
if [type] == "apache" {
|
||||
if [status] =~ /^5\d\d/ {
|
||||
nagios { ... }
|
||||
} else if [status] =~ /^4\d\d/ {
|
||||
elasticsearch { ... }
|
||||
}
|
||||
|
||||
statsd { increment => "apache.%{status}" }
|
||||
}
|
||||
}
|
||||
|
||||
## Further Reading
|
||||
|
||||
For more information, see [the plugin docs index](index)
|
||||
|
|
@ -1,241 +0,0 @@
|
|||
require "rubygems"
|
||||
require "erb"
|
||||
require "optparse"
|
||||
require "bluecloth" # for markdown parsing
|
||||
|
||||
$: << Dir.pwd
|
||||
$: << File.join(File.dirname(__FILE__), "..", "lib")
|
||||
|
||||
require "logstash/config/mixin"
|
||||
require "logstash/inputs/base"
|
||||
require "logstash/codecs/base"
|
||||
require "logstash/filters/base"
|
||||
require "logstash/outputs/base"
|
||||
require "logstash/version"
|
||||
|
||||
class LogStashConfigDocGenerator
|
||||
COMMENT_RE = /^ *#(?: (.*)| *$)/
|
||||
|
||||
def initialize
|
||||
@rules = {
|
||||
COMMENT_RE => lambda { |m| add_comment(m[1]) },
|
||||
/^ *class.*< *LogStash::(Outputs|Filters|Inputs|Codecs)::(Base|Threadable)/ => \
|
||||
lambda { |m| set_class_description },
|
||||
/^ *config +[^=].*/ => lambda { |m| add_config(m[0]) },
|
||||
/^ *milestone .*/ => lambda { |m| set_milestone(m[0]) },
|
||||
/^ *config_name .*/ => lambda { |m| set_config_name(m[0]) },
|
||||
/^ *flag[( ].*/ => lambda { |m| add_flag(m[0]) },
|
||||
/^ *(class|def|module) / => lambda { |m| clear_comments },
|
||||
}
|
||||
end
|
||||
|
||||
def parse(string)
|
||||
clear_comments
|
||||
buffer = ""
|
||||
string.split(/\r\n|\n/).each do |line|
|
||||
# Join long lines
|
||||
if line =~ COMMENT_RE
|
||||
# nothing
|
||||
else
|
||||
# Join extended lines
|
||||
if line =~ /(, *$)|(\\$)|(\[ *$)/
|
||||
buffer += line.gsub(/\\$/, "")
|
||||
next
|
||||
end
|
||||
end
|
||||
|
||||
line = buffer + line
|
||||
buffer = ""
|
||||
|
||||
@rules.each do |re, action|
|
||||
m = re.match(line)
|
||||
if m
|
||||
action.call(m)
|
||||
end
|
||||
end # RULES.each
|
||||
end # string.split("\n").each
|
||||
end # def parse
|
||||
|
||||
def set_class_description
|
||||
@class_description = @comments.join("\n")
|
||||
clear_comments
|
||||
end # def set_class_description
|
||||
|
||||
def add_comment(comment)
|
||||
@comments << comment
|
||||
end # def add_comment
|
||||
|
||||
def add_config(code)
|
||||
# I just care about the 'config :name' part
|
||||
code = code.sub(/,.*/, "")
|
||||
|
||||
# call the code, which calls 'config' in this class.
|
||||
# This will let us align comments with config options.
|
||||
name, opts = eval(code)
|
||||
|
||||
# TODO(sissel): This hack is only required until regexp configs
|
||||
# are gone from logstash.
|
||||
name = name.to_s unless name.is_a?(Regexp)
|
||||
|
||||
description = BlueCloth.new(@comments.join("\n")).to_html
|
||||
@attributes[name][:description] = description
|
||||
clear_comments
|
||||
end # def add_config
|
||||
|
||||
def add_flag(code)
|
||||
# call the code, which calls 'config' in this class.
|
||||
# This will let us align comments with config options.
|
||||
#p :code => code
|
||||
fixed_code = code.gsub(/ do .*/, "")
|
||||
#p :fixedcode => fixed_code
|
||||
name, description = eval(fixed_code)
|
||||
@flags[name] = description
|
||||
clear_comments
|
||||
end # def add_flag
|
||||
|
||||
def set_config_name(code)
|
||||
name = eval(code)
|
||||
@name = name
|
||||
end # def set_config_name
|
||||
|
||||
def set_milestone(code)
|
||||
@milestone = eval(code)
|
||||
end
|
||||
|
||||
# pretend to be the config DSL and just get the name
|
||||
def config(name, opts={})
|
||||
return name, opts
|
||||
end # def config
|
||||
|
||||
# Pretend to support the flag DSL
|
||||
def flag(*args, &block)
|
||||
name = args.first
|
||||
description = args.last
|
||||
return name, description
|
||||
end # def config
|
||||
|
||||
# pretend to be the config dsl's 'config_name' method
|
||||
def config_name(name)
|
||||
return name
|
||||
end # def config_name
|
||||
|
||||
# pretend to be the config dsl's 'milestone' method
|
||||
def milestone(m)
|
||||
return m
|
||||
end # def milestone
|
||||
|
||||
def clear_comments
|
||||
@comments.clear
|
||||
end # def clear_comments
|
||||
|
||||
def generate(file, settings)
|
||||
@class_description = ""
|
||||
@milestone = ""
|
||||
@comments = []
|
||||
@attributes = Hash.new { |h,k| h[k] = {} }
|
||||
@flags = {}
|
||||
|
||||
# local scoping for the monkeypatch belowg
|
||||
attributes = @attributes
|
||||
# Monkeypatch the 'config' method to capture
|
||||
# Note, this monkeypatch requires us do the config processing
|
||||
# one at a time.
|
||||
#LogStash::Config::Mixin::DSL.instance_eval do
|
||||
#define_method(:config) do |name, opts={}|
|
||||
#p name => opts
|
||||
#attributes[name].merge!(opts)
|
||||
#end
|
||||
#end
|
||||
|
||||
# Loading the file will trigger the config dsl which should
|
||||
# collect all the config settings.
|
||||
load file
|
||||
|
||||
# parse base first
|
||||
parse(File.new(File.join(File.dirname(file), "base.rb"), "r").read)
|
||||
|
||||
# Now parse the real library
|
||||
code = File.new(file).read
|
||||
|
||||
# inputs either inherit from Base or Threadable.
|
||||
if code =~ /\< LogStash::Inputs::Threadable/
|
||||
parse(File.new(File.join(File.dirname(file), "threadable.rb"), "r").read)
|
||||
end
|
||||
|
||||
if code =~ /include LogStash::PluginMixins/
|
||||
mixin = code.gsub(/.*include LogStash::PluginMixins::(\w+)\s.*/m, '\1')
|
||||
mixin.gsub!(/(.)([A-Z])/, '\1_\2')
|
||||
mixin.downcase!
|
||||
parse(File.new(File.join(File.dirname(file), "..", "plugin_mixins", "#{mixin}.rb")).read)
|
||||
end
|
||||
|
||||
parse(code)
|
||||
|
||||
puts "Generating docs for #{file}"
|
||||
|
||||
if @name.nil?
|
||||
$stderr.puts "Missing 'config_name' setting in #{file}?"
|
||||
return nil
|
||||
end
|
||||
|
||||
klass = LogStash::Config::Registry.registry[@name]
|
||||
if klass.ancestors.include?(LogStash::Inputs::Base)
|
||||
section = "input"
|
||||
elsif klass.ancestors.include?(LogStash::Filters::Base)
|
||||
section = "filter"
|
||||
elsif klass.ancestors.include?(LogStash::Outputs::Base)
|
||||
section = "output"
|
||||
elsif klass.ancestors.include?(LogStash::Codecs::Base)
|
||||
section = "codec"
|
||||
end
|
||||
|
||||
template_file = File.join(File.dirname(__FILE__), "plugin-doc.html.erb")
|
||||
template = ERB.new(File.new(template_file).read, nil, "-")
|
||||
|
||||
# descriptions are assumed to be markdown
|
||||
description = BlueCloth.new(@class_description).to_html
|
||||
|
||||
klass.get_config.each do |name, settings|
|
||||
@attributes[name].merge!(settings)
|
||||
end
|
||||
sorted_attributes = @attributes.sort { |a,b| a.first.to_s <=> b.first.to_s }
|
||||
klassname = LogStash::Config::Registry.registry[@name].to_s
|
||||
name = @name
|
||||
|
||||
synopsis_file = File.join(File.dirname(__FILE__), "plugin-synopsis.html.erb")
|
||||
synopsis = ERB.new(File.new(synopsis_file).read, nil, "-").result(binding)
|
||||
|
||||
if settings[:output]
|
||||
dir = File.join(settings[:output], section + "s")
|
||||
path = File.join(dir, "#{name}.html")
|
||||
Dir.mkdir(settings[:output]) if !File.directory?(settings[:output])
|
||||
Dir.mkdir(dir) if !File.directory?(dir)
|
||||
File.open(path, "w") do |out|
|
||||
html = template.result(binding)
|
||||
html.gsub!("1.2.0.beta1", LOGSTASH_VERSION)
|
||||
html.gsub!("%PLUGIN%", @name)
|
||||
out.puts(html)
|
||||
end
|
||||
else
|
||||
puts template.result(binding)
|
||||
end
|
||||
end # def generate
|
||||
|
||||
end # class LogStashConfigDocGenerator
|
||||
|
||||
if __FILE__ == $0
|
||||
opts = OptionParser.new
|
||||
settings = {}
|
||||
opts.on("-o DIR", "--output DIR",
|
||||
"Directory to output to; optional. If not specified,"\
|
||||
"we write to stdout.") do |val|
|
||||
settings[:output] = val
|
||||
end
|
||||
|
||||
args = opts.parse(ARGV)
|
||||
|
||||
args.each do |arg|
|
||||
gen = LogStashConfigDocGenerator.new
|
||||
gen.generate(arg, settings)
|
||||
end
|
||||
end
|
||||
|
|
@ -1,120 +0,0 @@
|
|||
---
|
||||
title: How to extend - logstash
|
||||
layout: content_right
|
||||
---
|
||||
# Add a new filter
|
||||
|
||||
This document shows you how to add a new filter to logstash.
|
||||
|
||||
For a general overview of how to add a new plugin, see [the extending
|
||||
logstash](.) overview.
|
||||
|
||||
## Write code.
|
||||
|
||||
Let's write a 'hello world' filter. This filter will replace the 'message' in
|
||||
the event with "Hello world!"
|
||||
|
||||
First, logstash expects plugins in a certain directory structure: `logstash/TYPE/PLUGIN_NAME.rb`
|
||||
|
||||
Since we're creating a filter, let's mkdir this:
|
||||
|
||||
mkdir -p logstash/filters/
|
||||
cd logstash/filters
|
||||
|
||||
Now add the code:
|
||||
|
||||
# Call this file 'foo.rb' (in logstash/filters, as above)
|
||||
require "logstash/filters/base"
|
||||
require "logstash/namespace"
|
||||
|
||||
class LogStash::Filters::Foo < LogStash::Filters::Base
|
||||
|
||||
# Setting the config_name here is required. This is how you
|
||||
# configure this filter from your logstash config.
|
||||
#
|
||||
# filter {
|
||||
# foo { ... }
|
||||
# }
|
||||
config_name "foo"
|
||||
# need to set a plugin_status
|
||||
plugin_status "experimental"
|
||||
|
||||
# Replace the message with this value.
|
||||
config :message, :validate => :string
|
||||
|
||||
public
|
||||
def register
|
||||
# nothing to do
|
||||
end # def register
|
||||
|
||||
public
|
||||
def filter(event)
|
||||
# return nothing unless there's an actual filter event
|
||||
return unless filter?(event)
|
||||
if @message
|
||||
# Replace the event message with our message as configured in the
|
||||
# config file.
|
||||
# If no message is specified, do nothing.
|
||||
event.message = @message
|
||||
end
|
||||
# filter_matched should go in the last line of our successful code
|
||||
filter_matched(event)
|
||||
end # def filter
|
||||
end # class LogStash::Filters::Foo
|
||||
|
||||
## Add it to your configuration
|
||||
|
||||
For this simple example, let's just use stdin input and stdout output.
|
||||
The config file looks like this:
|
||||
|
||||
input {
|
||||
stdin { type => "foo" }
|
||||
}
|
||||
filter {
|
||||
foo {
|
||||
type => "foo"
|
||||
message => "Hello world!"
|
||||
}
|
||||
}
|
||||
output {
|
||||
stdout { }
|
||||
}
|
||||
|
||||
Call this file 'example.conf'
|
||||
|
||||
## Tell logstash about it.
|
||||
|
||||
Depending on how you installed logstash, you have a few ways of including this
|
||||
plugin.
|
||||
|
||||
You can use the agent flag --pluginpath flag to specify where the root of your
|
||||
plugin tree is. In our case, it's the current directory.
|
||||
|
||||
% logstash --pluginpath . -f example.conf
|
||||
|
||||
If you use the jar release of logstash, you have an additional option - you can
|
||||
include the plugin right in the jar file.
|
||||
|
||||
# This command will take your 'logstash/filters/foo.rb' file
|
||||
# and add it into the jar file.
|
||||
% jar -uf logstash-1.2.0.beta1-flatjar.jar logstash/filters/foo.rb
|
||||
|
||||
# Verify it's in the right location in the jar!
|
||||
% jar tf logstash-1.2.0.beta1-flatjar.jar | grep foo.rb
|
||||
logstash/filters/foo.rb
|
||||
|
||||
% java -jar logstash-1.2.0.beta1-flatjar.jar agent -f example.conf
|
||||
|
||||
## Example running
|
||||
|
||||
In the example below, I typed in "the quick brown fox" after running the java
|
||||
command.
|
||||
|
||||
% java -jar logstash-1.2.0.beta1-flatjar.jar agent -f example.conf
|
||||
the quick brown fox
|
||||
2011-05-12T01:05:09.495000Z stdin://snack.home/: Hello world!
|
||||
|
||||
The output is the standard logstash stdout output, but in this case our "the
|
||||
quick brown fox" message was replaced with "Hello world!"
|
||||
|
||||
All done! :)
|
||||
|
|
@ -1,93 +0,0 @@
|
|||
---
|
||||
title: How to extend - logstash
|
||||
layout: content_right
|
||||
---
|
||||
# Extending logstash
|
||||
|
||||
You can add your own input, output, or filter plugins to logstash.
|
||||
|
||||
If you're looking to extend logstash today, please look at the existing plugins.
|
||||
|
||||
## Good examples of plugins
|
||||
|
||||
* [inputs/tcp](https://github.com/logstash/logstash/blob/master/lib/logstash/inputs/tcp.rb)
|
||||
* [filters/multiline](https://github.com/logstash/logstash/blob/master/lib/logstash/filters/multiline.rb)
|
||||
* [outputs/mongodb](https://github.com/logstash/logstash/blob/master/lib/logstash/outputs/mongodb.rb)
|
||||
|
||||
## Common concepts
|
||||
|
||||
* The `config_name` sets the name used in the config file.
|
||||
* The `plugin_status` sets the status of the plugin for example `beta`.
|
||||
* The `config` lines define config options.
|
||||
* The `register` method is called per plugin instantiation. Do any of your initialization here.
|
||||
|
||||
### Required modules
|
||||
|
||||
All plugins should require the Logstash module.
|
||||
|
||||
require 'logstash/namespace'
|
||||
|
||||
### Plugin name
|
||||
|
||||
Every plugin must have a name set with the `config_name` method. If this
|
||||
is not specified plugins will fail to load with an error.
|
||||
|
||||
### Plugin status
|
||||
|
||||
Every plugin needs a status set using `plugin_status`. Valid values are
|
||||
`stable`, `beta`, `experimental`, and `unsupported`. Plugins with either
|
||||
the `experimental` and `unsupported` status will generate warnings when
|
||||
used.
|
||||
|
||||
### Config lines
|
||||
|
||||
The `config` lines define configuration options and are constructed like
|
||||
so:
|
||||
|
||||
config :host, :validate => :string, :default => "0.0.0.0"
|
||||
|
||||
The name of the option is specified, here `:host` and then the
|
||||
attributes of the option. They can include `:validate`, `:default`,
|
||||
`:required` (a Boolean `true` or `false`), and `:deprecated` (also a
|
||||
Boolean).
|
||||
|
||||
## Inputs
|
||||
|
||||
All inputs require the LogStash::Inputs::Base class:
|
||||
|
||||
require 'logstash/inputs/base'
|
||||
|
||||
Inputs have two methods: `register` and `run`.
|
||||
|
||||
* Each input runs as its own thread.
|
||||
* The `run` method is expected to run-forever.
|
||||
|
||||
## Filters
|
||||
|
||||
All filters require the LogStash::Filters::Base class:
|
||||
|
||||
require 'logstash/filters/base'
|
||||
|
||||
Filters have two methods: `register` and `filter`.
|
||||
|
||||
* The `filter` method gets an event.
|
||||
* Call `event.cancel` to drop the event.
|
||||
* To modify an event, simply make changes to the event you are given.
|
||||
* The return value is ignored.
|
||||
|
||||
## Outputs
|
||||
|
||||
All outputs require the LogStash::Outputs::Base class:
|
||||
|
||||
require 'logstash/outputs/base'
|
||||
|
||||
Outputs have two methods: `register` and `receive`.
|
||||
|
||||
* The `register` method is called per plugin instantiation. Do any of your initialization here.
|
||||
* The `receive` method is called when an event gets pushed to your output
|
||||
|
||||
## Example: a new filter
|
||||
|
||||
Learn by example how to [add a new filter to logstash](example-add-a-new-filter)
|
||||
|
||||
|
||||
|
|
@ -1,244 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/advisor
|
||||
layout: content_right
|
||||
---
|
||||
<h2>advisor</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>INFORMATION:
|
||||
The filter Advisor is designed for capture and confrontation the events.
|
||||
The events must be grep by a filter first, then it can pull out a copy of it, like clone, whit tags "advisor<em>first",
|
||||
this copy is the first occurrence of this event verified in time</em>adv.
|
||||
After time<em>adv Advisor will pull out an event tagged "advisor</em>info" who will tell you the number of same events verified in time_adv.
|
||||
INFORMATION ABOUT CLASS:
|
||||
For do this job, i used a thread that will sleep time adv. I assume that events coming on advisor are tagged, then i use an array for storing different events.
|
||||
If an events is not present on array, then is the first and if the option is activate then advisor push out a copy of event.
|
||||
Else if the event is present on array, then is another same event and not the first, let's count it.<br/>
|
||||
USAGE:
|
||||
This is an example of logstash config:
|
||||
filter{
|
||||
advisor {</p>
|
||||
|
||||
<pre><code>time_adv => 1 #(optional)
|
||||
send_first => true #(optional)
|
||||
</code></pre>
|
||||
|
||||
<p> }
|
||||
}
|
||||
We analize this:
|
||||
time<em>adv => 1
|
||||
Means the time when the events matched and collected are pushed on outputs with tag "advisor</em>info".
|
||||
send<em>first => true
|
||||
Means you can push out the first events different who came in advisor like clone copy and tagged with "advisor</em>first"</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
advisor {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#send_first">send_first</a> => ... # boolean (optional), default: true
|
||||
<a href="#time_adv">time_adv</a> => ... # number (optional), default: 0
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
advisor {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
advisor {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
advisor {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
advisor {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="send_first">
|
||||
send_first
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>If you want the first different event will be pushed out like a copy</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="time_adv">
|
||||
time_adv
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 0 </li>
|
||||
</ul>
|
||||
|
||||
<p>If you do not set time_adv the plugin does nothing.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/advisor.rb">lib/logstash/filters/advisor.rb</a>
|
||||
|
|
@ -1,278 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/alter
|
||||
layout: content_right
|
||||
---
|
||||
<h2>alter</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>The alter filter allows you to do general alterations to fields
|
||||
that are not included in the normal mutate filter.</p>
|
||||
|
||||
<p>NOTE: The functionality provided by this plugin is likely to
|
||||
be merged into the 'mutate' filter in future versions.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#coalesce">coalesce</a> => ... # array (optional)
|
||||
<a href="#condrewrite">condrewrite</a> => ... # array (optional)
|
||||
<a href="#condrewriteother">condrewriteother</a> => ... # array (optional)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="coalesce">
|
||||
coalesce
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Sets the value of field_name to the first nonnull expression among its arguments.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
coalesce => [
|
||||
"field_name", "value1", "value2", "value3", ...
|
||||
]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="condrewrite">
|
||||
condrewrite
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Change the content of the field to the specified value
|
||||
if the actual content is equal to the expected one.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
condrewrite => [
|
||||
"field_name", "expected_value", "new_value"
|
||||
"field_name2", "expected_value2, "new_value2"
|
||||
....
|
||||
]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="condrewriteother">
|
||||
condrewriteother
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Change the content of the field to the specified value
|
||||
if the content of another field is equal to the expected one.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
condrewriteother => [
|
||||
"field_name", "expected_value", "field_name_to_change", "value",
|
||||
"field_name2", "expected_value2, "field_name_to_change2", "value2",
|
||||
....
|
||||
]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
alter {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/alter.rb">lib/logstash/filters/alter.rb</a>
|
||||
|
|
@ -1,237 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/anonymize
|
||||
layout: content_right
|
||||
---
|
||||
<h2>anonymize</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Anonymize fields using by replacing values with a consistent hash.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
anonymize {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#algorithm">algorithm</a> => ... # string, one of ["SHA1", "SHA256", "SHA384", "SHA512", "MD5", "MURMUR3", "IPV4_NETWORK"] (required), default: "SHA1"
|
||||
<a href="#fields">fields</a> => ... # array (required)
|
||||
<a href="#key">key</a> => ... # string (required)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
anonymize {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
anonymize {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="algorithm">
|
||||
algorithm (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "SHA1", "SHA256", "SHA384", "SHA512", "MD5", "MURMUR3", "IPV4_NETWORK" </li>
|
||||
<li> Default value is "SHA1" </li>
|
||||
</ul>
|
||||
|
||||
<p>digest/hash type</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="fields">
|
||||
fields (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The fields to be anonymized</p>
|
||||
|
||||
<h4>
|
||||
<a name="key">
|
||||
key (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Hashing key
|
||||
When using MURMUR3 the key is ignored but must still be set.
|
||||
When using IPV4_NETWORK key is the subnet prefix lentgh</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
anonymize {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
anonymize {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/anonymize.rb">lib/logstash/filters/anonymize.rb</a>
|
||||
|
|
@ -1,228 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/checksum
|
||||
layout: content_right
|
||||
---
|
||||
<h2>checksum</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>This filter let's you create a checksum based on various parts
|
||||
of the logstash event.
|
||||
This can be useful for deduplication of messages or simply to provide
|
||||
a custom unique identifier.</p>
|
||||
|
||||
<p>This is VERY experimental and is largely a proof-of-concept</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
checksum {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#algorithm">algorithm</a> => ... # string, one of ["md5", "sha128", "sha256", "sha384"] (optional), default: "sha256"
|
||||
<a href="#keys">keys</a> => ... # array (optional), default: ["message", "@timestamp", "type"]
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
checksum {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
checksum {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="algorithm">
|
||||
algorithm
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "md5", "sha128", "sha256", "sha384" </li>
|
||||
<li> Default value is "sha256" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="keys">
|
||||
keys
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["message", "@timestamp", "type"] </li>
|
||||
</ul>
|
||||
|
||||
<p>A list of keys to use in creating the string to checksum
|
||||
Keys will be sorted before building the string
|
||||
keys and values will then be concatenated with pipe delimeters
|
||||
and checksummed</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
checksum {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
checksum {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/checksum.rb">lib/logstash/filters/checksum.rb</a>
|
||||
|
|
@ -1,389 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/cipher
|
||||
layout: content_right
|
||||
---
|
||||
<h2>cipher</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>This filter parses a source and apply a cipher or decipher before
|
||||
storing it in the target.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
cipher {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#algorithm">algorithm</a> => ... # string (required)
|
||||
<a href="#base64">base64</a> => ... # boolean (optional), default: true
|
||||
<a href="#cipher_padding">cipher_padding</a> => ... # string (optional)
|
||||
<a href="#iv">iv</a> => ... # string (optional)
|
||||
<a href="#key">key</a> => ... # string (optional)
|
||||
<a href="#key_pad">key_pad</a> => ... # (optional), default: "\x00"
|
||||
<a href="#key_size">key_size</a> => ... # number (optional), default: 32
|
||||
<a href="#mode">mode</a> => ... # string (required)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#source">source</a> => ... # string (optional), default: "message"
|
||||
<a href="#target">target</a> => ... # string (optional), default: "message"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
cipher {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
cipher {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="algorithm">
|
||||
algorithm (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The cipher algorythm</p>
|
||||
|
||||
<p>A list of supported algorithms can be obtained by</p>
|
||||
|
||||
<pre><code>puts OpenSSL::Cipher.ciphers
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="base64">
|
||||
base64
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Do we have to perform a base64 decode or encode?</p>
|
||||
|
||||
<p>If we are decrypting, base64 decode will be done before.
|
||||
If we are encrypting, base64 will be done after.</p>
|
||||
|
||||
<h4>
|
||||
<a name="cipher_padding">
|
||||
cipher_padding
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Cypher padding to use. Enables or disables padding.</p>
|
||||
|
||||
<p>By default encryption operations are padded using standard block padding
|
||||
and the padding is checked and removed when decrypting. If the pad
|
||||
parameter is zero then no padding is performed, the total amount of data
|
||||
encrypted or decrypted must then be a multiple of the block size or an
|
||||
error will occur.</p>
|
||||
|
||||
<p>See EVP<em>CIPHER</em>CTX<em>set</em>padding for further information.</p>
|
||||
|
||||
<p>We are using Openssl jRuby which uses default padding to PKCS5Padding
|
||||
If you want to change it, set this parameter. If you want to disable
|
||||
it, Set this parameter to 0</p>
|
||||
|
||||
<pre><code>filter { cipher { padding => 0 }}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="iv">
|
||||
iv
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The initialization vector to use</p>
|
||||
|
||||
<p>The cipher modes CBC, CFB, OFB and CTR all need an "initialization
|
||||
vector", or short, IV. ECB mode is the only mode that does not require
|
||||
an IV, but there is almost no legitimate use case for this mode
|
||||
because of the fact that it does not sufficiently hide plaintext patterns.</p>
|
||||
|
||||
<h4>
|
||||
<a name="key">
|
||||
key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The key to use</p>
|
||||
|
||||
<h4>
|
||||
<a name="key_pad">
|
||||
key_pad
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "\x00" </li>
|
||||
</ul>
|
||||
|
||||
<p>The character used to pad the key</p>
|
||||
|
||||
<h4>
|
||||
<a name="key_size">
|
||||
key_size
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 32 </li>
|
||||
</ul>
|
||||
|
||||
<p>The key size to pad</p>
|
||||
|
||||
<p>It depends of the cipher algorythm.I your key don't need
|
||||
padding, don't set this parameter</p>
|
||||
|
||||
<p>Example, for AES-256, we must have 32 char long key</p>
|
||||
|
||||
<pre><code>filter { cipher { key_size => 32 }
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Encrypting or decrypting some data</p>
|
||||
|
||||
<p>Valid values are encrypt or decrypt</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
cipher {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
cipher {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "message" </li>
|
||||
</ul>
|
||||
|
||||
<p>The field to perform filter</p>
|
||||
|
||||
<p>Example, to use the @message field (default) :</p>
|
||||
|
||||
<pre><code>filter { cipher { source => "message" } }
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="target">
|
||||
target
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "message" </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the container to put the result</p>
|
||||
|
||||
<p>Example, to place the result into crypt :</p>
|
||||
|
||||
<pre><code>filter { cipher { target => "crypt" } }
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/cipher.rb">lib/logstash/filters/cipher.rb</a>
|
||||
|
|
@ -1,207 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/clone
|
||||
layout: content_right
|
||||
---
|
||||
<h2>clone</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>The clone filter is for duplicating events.
|
||||
A clone will be made for each type in the clone list.
|
||||
The original event is left unchanged.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
clone {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#clones">clones</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
clone {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
clone {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="clones">
|
||||
clones
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>A new clone will be created with the given type for each type in this list.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
clone {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
clone {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/clone.rb">lib/logstash/filters/clone.rb</a>
|
||||
|
|
@ -1,258 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/csv
|
||||
layout: content_right
|
||||
---
|
||||
<h2>csv</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>CSV filter. Takes an event field containing CSV data, parses it,
|
||||
and stores it as individual fields (can optionally specify the names).</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
csv {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#columns">columns</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#separator">separator</a> => ... # string (optional), default: ","
|
||||
<a href="#source">source</a> => ... # string (optional), default: "message"
|
||||
<a href="#target">target</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
csv {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
csv {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="columns">
|
||||
columns
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Define a list of column names (in the order they appear in the CSV,
|
||||
as if it were a header line). If this is not specified or there
|
||||
are not enough columns specified, the default column name is "columnX"
|
||||
(where X is the field number, starting from 1).</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
csv {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
csv {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="separator">
|
||||
separator
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "," </li>
|
||||
</ul>
|
||||
|
||||
<p>Define the column separator value. If this is not specified the default
|
||||
is a comma ','
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "message" </li>
|
||||
</ul>
|
||||
|
||||
<p>The CSV data in the value of the source field will be expanded into a
|
||||
datastructure.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="target">
|
||||
target
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Define target for placing the data
|
||||
Defaults to writing to the root of the event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/csv.rb">lib/logstash/filters/csv.rb</a>
|
||||
|
|
@ -1,298 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/date
|
||||
layout: content_right
|
||||
---
|
||||
<h2>date</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>The date filter is used for parsing dates from fields and using that
|
||||
date or timestamp as the timestamp for the event.</p>
|
||||
|
||||
<p>For example, syslog events usually have timestamps like this:</p>
|
||||
|
||||
<pre><code>"Apr 17 09:32:01"
|
||||
</code></pre>
|
||||
|
||||
<p>You would use the date format "MMM dd HH:mm:ss" to parse this.</p>
|
||||
|
||||
<p>The date filter is especially important for sorting events and for
|
||||
backfilling old data. If you don't get the date correct in your
|
||||
event, then searching for them later will likely sort out of order.</p>
|
||||
|
||||
<p>In the absence of this filter, logstash will choose a timestamp based on the
|
||||
first time it sees the event (at input time), if the timestamp is not already
|
||||
set in the event. For example, with file input, the timestamp is set to the
|
||||
time of each read.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
date {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#locale">locale</a> => ... # string (optional)
|
||||
<a href="#match">match</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#timezone">timezone</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
date {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
date {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="locale">
|
||||
locale
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>specify a locale to be used for date parsing. If this is not specified the
|
||||
platform default will be used</p>
|
||||
|
||||
<p>The locale is mostly necessary to be set for parsing month names and
|
||||
weekday names</p>
|
||||
|
||||
<h4>
|
||||
<a name="match">
|
||||
match
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>The date formats allowed are anything allowed by Joda-Time (java time
|
||||
library): You can see the docs for this format here:</p>
|
||||
|
||||
<p><a href="http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html">joda.time.format.DateTimeFormat</a></p>
|
||||
|
||||
<p>An array with field name first, and format patterns following, <code>[ field,
|
||||
formats... ]</code></p>
|
||||
|
||||
<p>If your time field has multiple possible formats, you can do this:</p>
|
||||
|
||||
<pre><code>match => [ "logdate", "MMM dd YYY HH:mm:ss",
|
||||
"MMM d YYY HH:mm:ss", "ISO8601" ]
|
||||
</code></pre>
|
||||
|
||||
<p>The above will match a syslog (rfc3164) or iso8601 timestamp.</p>
|
||||
|
||||
<p>There are a few special exceptions, the following format literals exist
|
||||
to help you save time and ensure correctness of date parsing.</p>
|
||||
|
||||
<ul>
|
||||
<li>"ISO8601" - should parse any valid ISO8601 timestamp, such as
|
||||
2011-04-19T03:44:01.103Z</li>
|
||||
<li>"UNIX" - will parse unix time in seconds since epoch</li>
|
||||
<li>"UNIX_MS" - will parse unix time in milliseconds since epoch</li>
|
||||
<li>"TAI64N" - will parse tai64n time values</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<p>For example, if you have a field 'logdate' and with a value that looks like
|
||||
'Aug 13 2010 00:03:44', you would use this configuration:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
date {
|
||||
match => [ "logdate", "MMM dd YYYY HH:mm:ss" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
date {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
date {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="timezone">
|
||||
timezone
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>specify a timezone canonical ID to be used for date parsing.
|
||||
The valid ID are listed on http://joda-time.sourceforge.net/timezones.html
|
||||
Useful in case the timezone cannot be extracted from the value,
|
||||
and is not the platform default
|
||||
If this is not specified the platform default will be used.
|
||||
Canonical ID is good as it takes care of daylight saving time for you
|
||||
For example, America/Los_Angeles or Europe/France are valid IDs</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/date.rb">lib/logstash/filters/date.rb</a>
|
||||
|
|
@ -1,294 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/dns
|
||||
layout: content_right
|
||||
---
|
||||
<h2>dns</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>DNS Filter</p>
|
||||
|
||||
<p>This filter will resolve any IP addresses from a field of your choosing.</p>
|
||||
|
||||
<p>The DNS filter performs a lookup (either an A record/CNAME record lookup
|
||||
or a reverse lookup at the PTR record) on records specified under the
|
||||
"reverse" and "resolve" arrays.</p>
|
||||
|
||||
<p>The config should look like this:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
dns {
|
||||
type => 'type'
|
||||
reverse => [ "source_host", "field_with_address" ]
|
||||
resolve => [ "field_with_fqdn" ]
|
||||
action => "replace"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>Caveats: at the moment, there's no way to tune the timeout with the 'resolv'
|
||||
core library. It does seem to be fixed in here:</p>
|
||||
|
||||
<p> http://redmine.ruby-lang.org/issues/5100</p>
|
||||
|
||||
<p>but isn't currently in JRuby.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
dns {
|
||||
<a href="#action">action</a> => ... # string, one of ["append", "replace"] (optional), default: "append"
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#nameserver">nameserver</a> => ... # string (optional)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#resolve">resolve</a> => ... # array (optional)
|
||||
<a href="#reverse">reverse</a> => ... # array (optional)
|
||||
<a href="#timeout">timeout</a> => ... # int (optional), default: 2
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="action">
|
||||
action
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "append", "replace" </li>
|
||||
<li> Default value is "append" </li>
|
||||
</ul>
|
||||
|
||||
<p>Determine what action to do: append or replace the values in the fields
|
||||
specified under "reverse" and "resolve."</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
dns {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
dns {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="nameserver">
|
||||
nameserver
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Use custom nameserver.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
dns {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
dns {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="resolve">
|
||||
resolve
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Forward resolve one or more fields.</p>
|
||||
|
||||
<h4>
|
||||
<a name="reverse">
|
||||
reverse
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Reverse resolve one or more fields.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="timeout">
|
||||
timeout
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#int">int</a> </li>
|
||||
<li> Default value is 2 </li>
|
||||
</ul>
|
||||
|
||||
<p>TODO(sissel): make 'action' required? This was always the intent, but it
|
||||
due to a typo it was never enforced. Thus the default behavior in past
|
||||
versions was 'append' by accident.
|
||||
resolv calls will be wrapped in a timeout instance</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/dns.rb">lib/logstash/filters/dns.rb</a>
|
||||
|
|
@ -1,204 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/drop
|
||||
layout: content_right
|
||||
---
|
||||
<h2>drop</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Drop filter.</p>
|
||||
|
||||
<p>Drops everything that gets to this filter.</p>
|
||||
|
||||
<p>This is best used in combination with conditionals, for example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
if [loglevel] == "debug" {
|
||||
drop { }
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above will only pass events to the drop filter if the loglevel field is
|
||||
"debug". This will cause all events matching to be dropped.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
drop {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
drop {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
drop {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
drop {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
drop {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/drop.rb">lib/logstash/filters/drop.rb</a>
|
||||
|
|
@ -1,206 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/environment
|
||||
layout: content_right
|
||||
---
|
||||
<h2>environment</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Set fields from environment variables</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
environment {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_field_from_env">add_field_from_env</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
environment {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_field_from_env">
|
||||
add_field_from_env
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Specify a hash of fields to the environment variable
|
||||
A hash of matches of field => environment variable</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
environment {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
environment {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
environment {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/environment.rb">lib/logstash/filters/environment.rb</a>
|
||||
|
|
@ -1,191 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/gelfify
|
||||
layout: content_right
|
||||
---
|
||||
<h2>gelfify</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>The GELFify filter parses RFC3164 severity levels to
|
||||
corresponding GELF levels.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
gelfify {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
gelfify {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
gelfify {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
gelfify {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
gelfify {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/gelfify.rb">lib/logstash/filters/gelfify.rb</a>
|
||||
|
|
@ -1,272 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/geoip
|
||||
layout: content_right
|
||||
---
|
||||
<h2>geoip</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Add GeoIP fields from Maxmind database</p>
|
||||
|
||||
<p>GeoIP filter, adds information about geographical location of IP addresses.
|
||||
This filter uses Maxmind GeoIP databases, have a look at
|
||||
https://www.maxmind.com/app/geolite</p>
|
||||
|
||||
<p>Logstash releases ship with the GeoLiteCity database made available from
|
||||
Maxmind with a CCA-ShareAlike 3.0 license. For more details on geolite, see
|
||||
<a href="http://www.maxmind.com/en/geolite">http://www.maxmind.com/en/geolite</a>.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
geoip {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#database">database</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#fields">fields</a> => ... # array (optional)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#source">source</a> => ... # string (optional)
|
||||
<a href="#target">target</a> => ... # string (optional), default: "geoip"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
geoip {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
geoip {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="database">
|
||||
database
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>GeoIP database file to use, Country, City, ASN, ISP and organization
|
||||
databases are supported</p>
|
||||
|
||||
<p>If not specified, this will default to the GeoLiteCity database that ships
|
||||
with logstash.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="fields">
|
||||
fields
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Array of geoip fields that we want to be included in our event.</p>
|
||||
|
||||
<p>Possible fields depend on the database type. By default, all geoip fields
|
||||
are included in the event.</p>
|
||||
|
||||
<p>For the built in GeoLiteCity database, the following are available:
|
||||
city_name, continent_code, country_code2, country_code3, country_name,
|
||||
dma_code, ip, latitude, longitude, postal_code, region_name, timezone</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
geoip {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
geoip {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The field containing IP address, hostname is also OK. If this field is an
|
||||
array, only the first value will be used.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="target">
|
||||
target
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "geoip" </li>
|
||||
</ul>
|
||||
|
||||
<p>Specify into what field you want the geoip data.
|
||||
This can be useful for example if you have a src_ip and dst_ip and want
|
||||
information of both IP's</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/geoip.rb">lib/logstash/filters/geoip.rb</a>
|
||||
|
|
@ -1,278 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/grep
|
||||
layout: content_right
|
||||
---
|
||||
<h2>grep</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>Grep filter. Useful for dropping events you don't want to pass, or
|
||||
adding tags or fields to events that match.</p>
|
||||
|
||||
<p>Events not matched are dropped. If 'negate' is set to true (defaults false),
|
||||
then matching events are dropped.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
grep {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#drop">drop</a> => ... # boolean (optional), default: true
|
||||
<a href="#ignore_case">ignore_case</a> => ... # boolean (optional), default: false
|
||||
<a href="#match">match</a> => ... # hash (optional), default: {}
|
||||
<a href="#negate">negate</a> => ... # boolean (optional), default: false
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grep {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grep {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="drop">
|
||||
drop
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Drop events that don't match</p>
|
||||
|
||||
<p>If this is set to false, no events will be dropped at all. Rather, the
|
||||
requested tags and fields will be added to matching events, and
|
||||
non-matching events will be passed through unchanged.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="ignore_case">
|
||||
ignore_case
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Use case-insensitive matching. Similar to 'grep -i'</p>
|
||||
|
||||
<p>If enabled, ignore case distinctions in the patterns.</p>
|
||||
|
||||
<h4>
|
||||
<a name="match">
|
||||
match
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>A hash of matches of field => regexp. If multiple matches are specified,
|
||||
all must match for the grep to be considered successful. Normal regular
|
||||
expressions are supported here.</p>
|
||||
|
||||
<p>For example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grep {
|
||||
match => [ "message", "hello world" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above will drop all events with a message not matching "hello world" as
|
||||
a regular expression.</p>
|
||||
|
||||
<h4>
|
||||
<a name="negate">
|
||||
negate
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Negate the match. Similar to 'grep -v'</p>
|
||||
|
||||
<p>If this is set to true, then any positive matches will result in the
|
||||
event being cancelled and dropped. Non-matching will be allowed
|
||||
through.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grep {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grep {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/grep.rb">lib/logstash/filters/grep.rb</a>
|
||||
|
|
@ -1,533 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/grok
|
||||
layout: content_right
|
||||
---
|
||||
<h2>grok</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>Parse arbitrary text and structure it.</p>
|
||||
|
||||
<p>Grok is currently the best way in logstash to parse crappy unstructured log
|
||||
data into something structured and queryable.</p>
|
||||
|
||||
<p>This tool is perfect for syslog logs, apache and other webserver logs, mysql
|
||||
logs, and in general, any log format that is generally written for humans
|
||||
and not computer consumption.</p>
|
||||
|
||||
<p>Logstash ships with about 120 patterns by default. You can find them here:
|
||||
<a href="https://github.com/logstash/logstash/tree/v1.2.0.beta1/patterns">https://github.com/logstash/logstash/tree/v1.2.0.beta1/patterns</a>. You can add
|
||||
your own trivially. (See the patterns_dir setting)</p>
|
||||
|
||||
<p>If you need help building patterns to match your logs, you will find the
|
||||
<a href="http://grokdebug.herokuapp.com">http://grokdebug.herokuapp.com</a> too quite useful!</p>
|
||||
|
||||
<h4>Grok Basics</h4>
|
||||
|
||||
<p>Grok works by using combining text patterns into something that matches your
|
||||
logs.</p>
|
||||
|
||||
<p>The syntax for a grok pattern is <code>%{SYNTAX:SEMANTIC}</code></p>
|
||||
|
||||
<p>The <code>SYNTAX</code> is the name of the pattern that will match your text. For
|
||||
example, "3.44" will be matched by the NUMBER pattern and "55.3.244.1" will
|
||||
be matched by the IP pattern. The syntax is how you match.</p>
|
||||
|
||||
<p>The <code>SEMANTIC</code> is the identifier you give to the piece of text being matched.
|
||||
For example, "3.44" could be the duration of an event, so you could call it
|
||||
simply 'duration'. Further, a string "55.3.244.1" might identify the client
|
||||
making a request.</p>
|
||||
|
||||
<p>Optionally you can add a data type conversion to your grok pattern. By default
|
||||
all semantics are saved as strings. If you wish to convert a semnatic's data type,
|
||||
for example change a string to an integer then suffix it with the target data type.
|
||||
For example <code>${NUMBER:num:int}</code> which converts the 'num' semantic from a string to an
|
||||
integer. Currently the only supporting conversions are <code>int</code> and <code>float</code>.</p>
|
||||
|
||||
<h4>Example</h4>
|
||||
|
||||
<p>With that idea of a syntax and semantic, we can pull out useful fields from a
|
||||
sample log like this fictional http request log:</p>
|
||||
|
||||
<pre><code>55.3.244.1 GET /index.html 15824 0.043
|
||||
</code></pre>
|
||||
|
||||
<p>The pattern for this could be:</p>
|
||||
|
||||
<pre><code>%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}
|
||||
</code></pre>
|
||||
|
||||
<p>A more realistic example, let's read these logs from a file:</p>
|
||||
|
||||
<pre><code>input {
|
||||
file {
|
||||
path => "/var/log/http.log"
|
||||
type => "examplehttp"
|
||||
}
|
||||
}
|
||||
filter {
|
||||
grok {
|
||||
type => "examplehttp"
|
||||
match => [ "message", "%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>After the grok filter, the event will have a few extra fields in it:</p>
|
||||
|
||||
<ul>
|
||||
<li>client: 55.3.244.1</li>
|
||||
<li>method: GET</li>
|
||||
<li>request: /index.html</li>
|
||||
<li>bytes: 15824</li>
|
||||
<li>duration: 0.043</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h4>Regular Expressions</h4>
|
||||
|
||||
<p>Grok sits on top of regular expressions, so any regular expressions are valid
|
||||
in grok as well. The regular expression library is Oniguruma, and you can see
|
||||
the full supported regexp syntax <a href="http://www.geocities.jp/kosako3/oniguruma/doc/RE.txt">on the Onigiruma
|
||||
site</a></p>
|
||||
|
||||
<h4>Custom Patterns</h4>
|
||||
|
||||
<p>Sometimes logstash doesn't have a pattern you need. For this, you have
|
||||
a few options.</p>
|
||||
|
||||
<p>First, you can use the Oniguruma syntax for 'named capture' which will
|
||||
let you match a piece of text and save it as a field:</p>
|
||||
|
||||
<pre><code>(?<field_name>the pattern here)
|
||||
</code></pre>
|
||||
|
||||
<p>For example, postfix logs have a 'queue id' that is an 11-character
|
||||
hexadecimal value. I can capture that easily like this:</p>
|
||||
|
||||
<pre><code>(?<queue_id>[0-9A-F]{11})
|
||||
</code></pre>
|
||||
|
||||
<p>Alternately, you can create a custom patterns file.</p>
|
||||
|
||||
<ul>
|
||||
<li>Create a directory called <code>patterns</code> with a file in it called <code>extra</code>
|
||||
(the file name doesn't matter, but name it meaningfully for yourself)</li>
|
||||
<li>In that file, write the pattern you need as the pattern name, a space, then
|
||||
the regexp for that pattern.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<p>For example, doing the postfix queue id example as above:</p>
|
||||
|
||||
<pre><code># in ./patterns/postfix
|
||||
POSTFIX_QUEUEID [0-9A-F]{11}
|
||||
</code></pre>
|
||||
|
||||
<p>Then use the <code>patterns_dir</code> setting in this plugin to tell logstash where
|
||||
your custom patterns directory is. Here's a full example with a sample log:</p>
|
||||
|
||||
<pre><code>Jan 1 06:25:43 mailserver14 postfix/cleanup[21403]: BEF25A72965: message-id=<20130101142543.5828399CCAF@mailserver14.example.com>
|
||||
|
||||
filter {
|
||||
grok {
|
||||
patterns_dir => "./patterns"
|
||||
match => [ "message", "%{SYSLOGBASE} %{POSTFIX_QUEUEID:queue_id}: %{GREEDYDATA:message}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above will match and result in the following fields:</p>
|
||||
|
||||
<ul>
|
||||
<li>timestamp: Jan 1 06:25:43</li>
|
||||
<li>logsource: mailserver14</li>
|
||||
<li>program: postfix/cleanup</li>
|
||||
<li>pid: 21403</li>
|
||||
<li>queue_id: BEF25A72965</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<p>The <code>timestamp</code>, <code>logsource</code>, <code>program</code>, and <code>pid</code> fields come from the
|
||||
SYSLOGBASE pattern which itself is defined by other patterns.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#break_on_match">break_on_match</a> => ... # boolean (optional), default: true
|
||||
<a href="#drop_if_match">drop_if_match</a> => ... # boolean (optional), default: false
|
||||
<a href="#keep_empty_captures">keep_empty_captures</a> => ... # boolean (optional), default: false
|
||||
<a href="#match">match</a> => ... # hash (optional), default: {}
|
||||
<a href="#named_captures_only">named_captures_only</a> => ... # boolean (optional), default: true
|
||||
<a href="#overwrite">overwrite</a> => ... # array (optional), default: []
|
||||
<a href="#patterns_dir">patterns_dir</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#singles">singles</a> => ... # boolean (optional), default: true
|
||||
<a href="#tag_on_failure">tag_on_failure</a> => ... # array (optional), default: ["_grokparsefailure"]
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="break_on_match">
|
||||
break_on_match
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Break on first match. The first successful match by grok will result in the
|
||||
filter being finished. If you want grok to try all patterns (maybe you are
|
||||
parsing different things), then set this to false.</p>
|
||||
|
||||
<h4>
|
||||
<a name="drop_if_match">
|
||||
drop_if_match
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Drop if matched. Note, this feature may not stay. It is preferable to combine
|
||||
grok + grep filters to do parsing + dropping.</p>
|
||||
|
||||
<p>requested in: googlecode/issue/26</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="keep_empty_captures">
|
||||
keep_empty_captures
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>If true, keep empty captures as event fields.</p>
|
||||
|
||||
<h4>
|
||||
<a name="match">
|
||||
match
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>A hash of matches of field => value</p>
|
||||
|
||||
<p>For example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
match => [ "message", "Duration: %{NUMBER:duration}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="named_captures_only">
|
||||
named_captures_only
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>If true, only store named captures from grok.</p>
|
||||
|
||||
<h4>
|
||||
<a name="overwrite">
|
||||
overwrite
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>The fields to overwrite.</p>
|
||||
|
||||
<p>This allows you to overwrite a value in a field that already exists.</p>
|
||||
|
||||
<p>For example, if you have a syslog line in the 'message' field, you can
|
||||
overwrite the 'message' field with part of the match like so:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
match => [
|
||||
"message",
|
||||
"%{SYSLOGBASE} %{DATA:message}
|
||||
]
|
||||
overwrite => [ "message" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p> In this case, a line like "May 29 16:37:11 sadness logger: hello world"
|
||||
will be parsed and 'hello world' will overwrite the original message.</p>
|
||||
|
||||
<h4>
|
||||
<a name="pattern">
|
||||
pattern
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Specify a pattern to parse with. This will match the 'message' field.</p>
|
||||
|
||||
<p>If you want to match other fields than message, use the 'match' setting.
|
||||
Multiple patterns is fine.</p>
|
||||
|
||||
<h4>
|
||||
<a name="patterns_dir">
|
||||
patterns_dir
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>logstash ships by default with a bunch of patterns, so you don't
|
||||
necessarily need to define this yourself unless you are adding additional
|
||||
patterns.</p>
|
||||
|
||||
<p>Pattern files are plain text with format:</p>
|
||||
|
||||
<pre><code>NAME PATTERN
|
||||
</code></pre>
|
||||
|
||||
<p>For example:</p>
|
||||
|
||||
<pre><code>NUMBER \d+
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="singles">
|
||||
singles
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>If true, make single-value fields simply that value, not an array
|
||||
containing that one value.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tag_on_failure">
|
||||
tag_on_failure
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["_grokparsefailure"] </li>
|
||||
</ul>
|
||||
|
||||
<p>If true, ensure the '_grokparsefailure' tag is present when there has been no
|
||||
successful match</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/grok.rb">lib/logstash/filters/grok.rb</a>
|
||||
|
|
@ -1,191 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/grokdiscovery
|
||||
layout: content_right
|
||||
---
|
||||
<h2>grokdiscovery</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>TODO(sissel): This is not supported yet. There is a bug in grok discovery
|
||||
that causes segfaults in libgrok.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
grokdiscovery {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grokdiscovery {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grokdiscovery {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grokdiscovery {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grokdiscovery {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/grokdiscovery.rb">lib/logstash/filters/grokdiscovery.rb</a>
|
||||
|
|
@ -1,250 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/json
|
||||
layout: content_right
|
||||
---
|
||||
<h2>json</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>JSON filter. Takes a field that contains JSON and expands it into
|
||||
an actual datastructure.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
json {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#source">source</a> => ... # string (required)
|
||||
<a href="#target">target</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Config for json is:</p>
|
||||
|
||||
<pre><code>source => source_field
|
||||
</code></pre>
|
||||
|
||||
<p>For example, if you have json data in the @message field:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json {
|
||||
source => "message"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above would parse the json from the @message field</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="target">
|
||||
target
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Define target for placing the data. If this setting is omitted,
|
||||
the json data will be stored at the root of the event.</p>
|
||||
|
||||
<p>For example if you want the data to be put in the 'doc' field:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json {
|
||||
target => "doc"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>json in the value of the source field will be expanded into a
|
||||
datastructure in the "target" field.</p>
|
||||
|
||||
<p>Note: if the "target" field already exists, it will be overwritten.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/json.rb">lib/logstash/filters/json.rb</a>
|
||||
|
|
@ -1,223 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/json_encode
|
||||
layout: content_right
|
||||
---
|
||||
<h2>json_encode</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>JSON encode filter. Takes a field and serializes it into JSON</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
json_encode {
|
||||
<a href="#/[A-Za-z0-9_@-]+/">/[A-Za-z0-9_@-]+/</a> => ... # string (optional)
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="/[A-Za-z0-9_@-]+/">
|
||||
/[A-Za-z0-9_@-]+/
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> The configuration attribute name here is anything that matches the above regular expression. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Config for json_encode is:</p>
|
||||
|
||||
<ul>
|
||||
<li>source => dest</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<p>For example, if you have a field named 'foo', and you want to store the
|
||||
JSON encoded string in 'bar', do this:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json_encode {
|
||||
foo => bar
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>Note: if the "dest" field already exists, it will be overridden.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json_encode {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json_encode {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json_encode {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
json_encode {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/json_encode.rb">lib/logstash/filters/json_encode.rb</a>
|
||||
|
|
@ -1,476 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/kv
|
||||
layout: content_right
|
||||
---
|
||||
<h2>kv</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>This filter helps automatically parse messages which are of the 'foo=bar'
|
||||
variety.</p>
|
||||
|
||||
<p>For example, if you have a log message which contains 'ip=1.2.3.4
|
||||
error=REFUSED', you can parse those automatically by doing:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv { }
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above will result in a message of "ip=1.2.3.4 error=REFUSED" having
|
||||
the fields:</p>
|
||||
|
||||
<ul>
|
||||
<li>ip: 1.2.3.4</li>
|
||||
<li>error: REFUSED</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<p>This is great for postfix, iptables, and other types of logs that
|
||||
tend towards 'key=value' syntax.</p>
|
||||
|
||||
<p>Further, this can often be used to parse query parameters like
|
||||
'foo=bar&baz=fizz' by setting the field_split to "&"</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#default_keys">default_keys</a> => ... # hash (optional), default: {}
|
||||
<a href="#exclude_keys">exclude_keys</a> => ... # array (optional), default: []
|
||||
<a href="#field_split">field_split</a> => ... # string (optional), default: " "
|
||||
<a href="#include_keys">include_keys</a> => ... # array (optional), default: []
|
||||
<a href="#prefix">prefix</a> => ... # string (optional), default: ""
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#source">source</a> => ... # string (optional), default: "message"
|
||||
<a href="#target">target</a> => ... # string (optional)
|
||||
<a href="#trim">trim</a> => ... # string (optional)
|
||||
<a href="#trimkey">trimkey</a> => ... # string (optional)
|
||||
<a href="#value_split">value_split</a> => ... # string (optional), default: "="
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="default_keys">
|
||||
default_keys
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>A hash that specifies the default keys and their values that should be added to event
|
||||
in case these keys do no exist in the source field being parsed.</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
default_keys = [ "from", "logstash@example.com",
|
||||
"to", "default@dev.null" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_keys">
|
||||
exclude_keys
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>An array that specifies the parsed keys which should not be added to event.
|
||||
By default no keys will be excluded.</p>
|
||||
|
||||
<p>Example, to exclude "from" and "to" from a source like "Hey, from=<abc>, to=def foo=bar"
|
||||
while "foo" key will be added to event.</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
exclude_keys = [ "from", "to" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="field_split">
|
||||
field_split
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is " " </li>
|
||||
</ul>
|
||||
|
||||
<p>A string of characters to use as delimiters for parsing out key-value pairs.</p>
|
||||
|
||||
<p>These characters form a regex character class and thus you must escape special regex
|
||||
characters like [ or ] using .</p>
|
||||
|
||||
<h4>Example with URL Query Strings</h4>
|
||||
|
||||
<p>Example, to split out the args from a url query string such as
|
||||
'?pin=12345~0&d=123&e=foo@bar.com&oq=bobo&ss=12345':</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
field_split => "&?"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above splits on both "&" and "?" characters, giving you the following
|
||||
fields:</p>
|
||||
|
||||
<ul>
|
||||
<li>pin: 12345~0</li>
|
||||
<li>d: 123</li>
|
||||
<li>e: foo@bar.com</li>
|
||||
<li>oq: bobo</li>
|
||||
<li>ss: 12345</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="include_keys">
|
||||
include_keys
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>An array that specifies the parsed keys which should be added to event.
|
||||
By default all keys will be added.</p>
|
||||
|
||||
<p>Example, to include only "from" and "to" from a source like "Hey, from=<abc>, to=def foo=bar"
|
||||
while "foo" key will not be added to event.</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
include_keys = [ "from", "to" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="prefix">
|
||||
prefix
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>A string to prepend to all of the extracted keys</p>
|
||||
|
||||
<p>Example, to prepend arg_ to all keys:</p>
|
||||
|
||||
<pre><code>filter { kv { prefix => "arg_" } }
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "message" </li>
|
||||
</ul>
|
||||
|
||||
<p>The fields to perform 'key=value' searching on</p>
|
||||
|
||||
<p>Example, to use the message field:</p>
|
||||
|
||||
<pre><code>filter { kv { source => "message" } }
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="target">
|
||||
target
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the container to put all of the key-value pairs into</p>
|
||||
|
||||
<p>If this setting is omitted, fields will be written to the root of the
|
||||
event.</p>
|
||||
|
||||
<p>Example, to place all keys into field kv:</p>
|
||||
|
||||
<pre><code>filter { kv { target => "kv" } }
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="trim">
|
||||
trim
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>A string of characters to trim from the value. This is useful if your
|
||||
values are wrapped in brackets or are terminated by comma (like postfix
|
||||
logs)</p>
|
||||
|
||||
<p>These characters form a regex character class and thus you must escape special regex
|
||||
characters like [ or ] using .</p>
|
||||
|
||||
<p>Example, to strip '<' '>' '[' ']' and ',' characters from values:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
trim => "<>\[\],"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="trimkey">
|
||||
trimkey
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>A string of characters to trim from the key. This is useful if your
|
||||
key are wrapped in brackets or starts with space</p>
|
||||
|
||||
<p>These characters form a regex character class and thus you must escape special regex
|
||||
characters like [ or ] using .</p>
|
||||
|
||||
<p>Example, to strip '<' '>' '[' ']' and ',' characters from keys:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
kv {
|
||||
trimkey => "<>\[\],"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="value_split">
|
||||
value_split
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "=" </li>
|
||||
</ul>
|
||||
|
||||
<p>A string of characters to use as delimiters for identifying key-value relations.</p>
|
||||
|
||||
<p>These characters form a regex character class and thus you must escape special regex
|
||||
characters like [ or ] using .</p>
|
||||
|
||||
<p>Example, to identify key-values such as
|
||||
'key1:value1 key2:value2':</p>
|
||||
|
||||
<pre><code>filter { kv { value_split => ":" } }
|
||||
</code></pre>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/kv.rb">lib/logstash/filters/kv.rb</a>
|
||||
|
|
@ -1,220 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/metaevent
|
||||
layout: content_right
|
||||
---
|
||||
<h2>metaevent</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
metaevent {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#followed_by_tags">followed_by_tags</a> => ... # array (required)
|
||||
<a href="#period">period</a> => ... # number (optional), default: 5
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metaevent {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metaevent {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="followed_by_tags">
|
||||
followed_by_tags (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>syntax: <code>followed_by_tags => [ "tag", "tag" ]</code></p>
|
||||
|
||||
<h4>
|
||||
<a name="period">
|
||||
period
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5 </li>
|
||||
</ul>
|
||||
|
||||
<p>syntax: <code>period => 60</code></p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metaevent {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metaevent {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/metaevent.rb">lib/logstash/filters/metaevent.rb</a>
|
||||
|
|
@ -1,348 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/metrics
|
||||
layout: content_right
|
||||
---
|
||||
<h2>metrics</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>The metrics filter is useful for aggregating metrics.</p>
|
||||
|
||||
<p>For example, if you have a field 'response' that is
|
||||
a http response code, and you want to count each
|
||||
kind of response, you can do this:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metrics {
|
||||
meter => [ "http.%{response}" ]
|
||||
add_tag => "metric"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>Metrics are flushed every 5 seconds. Metrics appear as
|
||||
new events in the event stream and go through any filters
|
||||
that occur after as well as outputs.</p>
|
||||
|
||||
<p>In general, you will want to add a tag to your metrics and have an output
|
||||
explicitly look for that tag.</p>
|
||||
|
||||
<p>The event that is flushed will include every 'meter' and 'timer'
|
||||
metric in the following way:</p>
|
||||
|
||||
<h4>'meter' values</h4>
|
||||
|
||||
<p>For a <code>meter => "something"</code> you will receive the following fields:</p>
|
||||
|
||||
<ul>
|
||||
<li>"thing.count" - the total count of events</li>
|
||||
<li>"thing.rate_1m" - the 1-minute rate (sliding)</li>
|
||||
<li>"thing.rate_5m" - the 5-minute rate (sliding)</li>
|
||||
<li>"thing.rate_15m" - the 15-minute rate (sliding)</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h4>'timer' values</h4>
|
||||
|
||||
<p>For a <code>timer => [ "thing", "%{duration}" ]</code> you will receive the following fields:</p>
|
||||
|
||||
<ul>
|
||||
<li>"thing.count" - the total count of events</li>
|
||||
<li>"thing.rate_1m" - the 1-minute rate of events (sliding)</li>
|
||||
<li>"thing.rate_5m" - the 5-minute rate of events (sliding)</li>
|
||||
<li>"thing.rate_15m" - the 15-minute rate of events (sliding)</li>
|
||||
<li>"thing.min" - the minimum value seen for this metric</li>
|
||||
<li>"thing.max" - the maximum value seen for this metric</li>
|
||||
<li>"thing.stddev" - the standard deviation for this metric</li>
|
||||
<li>"thing.mean" - the mean for this metric</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h4>Example: computing event rate</h4>
|
||||
|
||||
<p>For a simple example, let's track how many events per second are running
|
||||
through logstash:</p>
|
||||
|
||||
<pre><code>input {
|
||||
generator {
|
||||
type => "generated"
|
||||
}
|
||||
}
|
||||
|
||||
filter {
|
||||
metrics {
|
||||
type => "generated"
|
||||
meter => "events"
|
||||
add_tag => "metric"
|
||||
}
|
||||
}
|
||||
|
||||
output {
|
||||
stdout {
|
||||
# only emit events with the 'metric' tag
|
||||
tags => "metric"
|
||||
message => "rate: %{events.rate_1m}"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>Running the above:</p>
|
||||
|
||||
<pre><code>% java -jar logstash.jar agent -f example.conf
|
||||
rate: 23721.983566819246
|
||||
rate: 24811.395722536377
|
||||
rate: 25875.892745934525
|
||||
rate: 26836.42375967113
|
||||
</code></pre>
|
||||
|
||||
<p>We see the output includes our 'events' 1-minute rate.</p>
|
||||
|
||||
<p>In the real world, you would emit this to graphite or another metrics store,
|
||||
like so:</p>
|
||||
|
||||
<pre><code>output {
|
||||
graphite {
|
||||
metrics => [ "events.rate_1m", "%{events.rate_1m}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
metrics {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#ignore_older_than">ignore_older_than</a> => ... # number (optional), default: 0
|
||||
<a href="#meter">meter</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#timer">timer</a> => ... # hash (optional), default: {}
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metrics {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metrics {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="ignore_older_than">
|
||||
ignore_older_than
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 0 </li>
|
||||
</ul>
|
||||
|
||||
<p>Don't track events that have @timestamp older than some number of seconds.</p>
|
||||
|
||||
<p>This is useful if you want to only include events that are near real-time
|
||||
in your metrics.</p>
|
||||
|
||||
<p>Example, to only count events that are within 10 seconds of real-time, you
|
||||
would do this:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metrics {
|
||||
meter => [ "hits" ]
|
||||
ignore_older_than => 10
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="meter">
|
||||
meter
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>syntax: <code>meter => [ "name of metric", "name of metric" ]</code></p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metrics {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
metrics {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="timer">
|
||||
timer
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>syntax: <code>timer => [ "name of metric", "%{time_value}" ]</code></p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/metrics.rb">lib/logstash/filters/metrics.rb</a>
|
||||
|
|
@ -1,284 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/multiline
|
||||
layout: content_right
|
||||
---
|
||||
<h2>multiline</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<h2>This filter was replaced by a codec.</h2>
|
||||
|
||||
<p>See the multiline codec instead.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
multiline {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#negate">negate</a> => ... # boolean (optional), default: false
|
||||
<a href="#pattern">pattern</a> => ... # string (required)
|
||||
<a href="#patterns_dir">patterns_dir</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#source">source</a> => ... # string (optional), default: "message"
|
||||
<a href="#stream_identity">stream_identity</a> => ... # string (optional), default: "%{host}-%{path}-%{type}"
|
||||
<a href="#what">what</a> => ... # string, one of ["previous", "next"] (required)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
multiline {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
multiline {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="negate">
|
||||
negate
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="pattern">
|
||||
pattern (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Leave these config settings until we remove this filter entirely.
|
||||
THe idea is that we want the register method to cause an abort
|
||||
giving the user a clue to use the codec instead of the filter.</p>
|
||||
|
||||
<h4>
|
||||
<a name="patterns_dir">
|
||||
patterns_dir
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
multiline {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
multiline {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "message" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="stream_identity">
|
||||
stream_identity
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "%{host}-%{path}-%{type}" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="what">
|
||||
what (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "previous", "next" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/multiline.rb">lib/logstash/filters/multiline.rb</a>
|
||||
|
|
@ -1,511 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/mutate
|
||||
layout: content_right
|
||||
---
|
||||
<h2>mutate</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>The mutate filter allows you to do general mutations to fields. You
|
||||
can rename, remove, replace, and modify fields in your events.</p>
|
||||
|
||||
<p>TODO(sissel): Support regexp replacements like String#gsub ?</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#convert">convert</a> => ... # hash (optional)
|
||||
<a href="#gsub">gsub</a> => ... # array (optional)
|
||||
<a href="#join">join</a> => ... # hash (optional)
|
||||
<a href="#lowercase">lowercase</a> => ... # array (optional)
|
||||
<a href="#merge">merge</a> => ... # hash (optional)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#rename">rename</a> => ... # hash (optional)
|
||||
<a href="#replace">replace</a> => ... # hash (optional)
|
||||
<a href="#split">split</a> => ... # hash (optional)
|
||||
<a href="#strip">strip</a> => ... # array (optional)
|
||||
<a href="#update">update</a> => ... # hash (optional)
|
||||
<a href="#uppercase">uppercase</a> => ... # array (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="convert">
|
||||
convert
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Convert a field's value to a different type, like turning a string to an
|
||||
integer. If the field value is an array, all members will be converted.
|
||||
If the field is a hash, no action will be taken.</p>
|
||||
|
||||
<p>Valid conversion targets are: integer, float, string</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
convert => [ "fieldname", "integer" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="gsub">
|
||||
gsub
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Convert a string field by applying a regular expression and a replacement
|
||||
if the field is not a string, no action will be taken</p>
|
||||
|
||||
<p>This configuration takes an array consisting of 3 elements per
|
||||
field/substitution.</p>
|
||||
|
||||
<p>be aware of escaping any backslash in the config file</p>
|
||||
|
||||
<p>for example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
gsub => [
|
||||
# replace all forward slashes with underscore
|
||||
"fieldname", "/", "_",
|
||||
|
||||
# replace backslashes, question marks, hashes, and minuses with
|
||||
# dot
|
||||
"fieldname2", "[\\?#-]", "."
|
||||
]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="join">
|
||||
join
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Join an array with a separator character, does nothing on non-array fields</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<p> filter {</p>
|
||||
|
||||
<pre><code> mutate {
|
||||
join => ["fieldname", ","]
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p> }</p>
|
||||
|
||||
<h4>
|
||||
<a name="lowercase">
|
||||
lowercase
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Convert a string to its lowercase equivalent</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
lowercase => [ "fieldname" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="merge">
|
||||
merge
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>merge two fields or arrays or hashes
|
||||
String fields will be converted in array, so
|
||||
array + string will work
|
||||
string + string will result in an 2 entry array in dest_field
|
||||
array and hash will not work</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
merge => ["dest_field", "added_field"]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="remove">
|
||||
remove
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Remove one or more fields.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
remove => [ "client" ] # Removes the 'client' field
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>This option is deprecated, instead use remove_field option available in all
|
||||
filters.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="rename">
|
||||
rename
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Rename one or more fields.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
# Renames the 'HOSTORIP' field to 'client_ip'
|
||||
rename => [ "HOSTORIP", "client_ip" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="replace">
|
||||
replace
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Replace a field with a new value. The new value can include %{foo} strings
|
||||
to help you build a new value from other parts of the event.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
replace => [ "message", "%{source_host}: My new message" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="split">
|
||||
split
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Split a field to an array using a separator character. Only works on string
|
||||
fields.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
split => ["fieldname", ","]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="strip">
|
||||
strip
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Strip whitespaces</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
strip => ["field1", "field2"]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="update">
|
||||
update
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Update an existing field with a new value. If the field does not exist,
|
||||
then no action will be taken.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
update => [ "sample", "My new message" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="uppercase">
|
||||
uppercase
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Convert a string to its uppercase equivalent</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
mutate {
|
||||
uppercase => [ "fieldname" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/mutate.rb">lib/logstash/filters/mutate.rb</a>
|
||||
|
|
@ -1,190 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/noop
|
||||
layout: content_right
|
||||
---
|
||||
<h2>noop</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>No-op filter. This is used generally for internal/dev testing.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
noop {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
noop {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
noop {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
noop {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
noop {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/noop.rb">lib/logstash/filters/noop.rb</a>
|
||||
|
|
@ -1,308 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/prune
|
||||
layout: content_right
|
||||
---
|
||||
<h2>prune</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>The prune filter is for pruning event data from @fileds based on whitelist/blacklist
|
||||
of field names or their values (names and values can also be regular expressions).</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#blacklist_names">blacklist_names</a> => ... # array (optional), default: ["%{[^}]+}"]
|
||||
<a href="#blacklist_values">blacklist_values</a> => ... # hash (optional), default: {}
|
||||
<a href="#interpolate">interpolate</a> => ... # boolean (optional), default: false
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#whitelist_names">whitelist_names</a> => ... # array (optional), default: []
|
||||
<a href="#whitelist_values">whitelist_values</a> => ... # hash (optional), default: {}
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="blacklist_names">
|
||||
blacklist_names
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["%{[^}]+}"] </li>
|
||||
</ul>
|
||||
|
||||
<p>Exclude fields which names match specified regexps, by default exclude unresolved %{field} strings.</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
tags => [ "apache-accesslog" ]
|
||||
blacklist_names => [ "method", "(referrer|status)", "${some}_field" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="blacklist_values">
|
||||
blacklist_values
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Exclude specified fields if their values match regexps.
|
||||
In case field values are arrays, the fields are pruned on per array item
|
||||
in case all array items are matched whole field will be deleted.</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
tags => [ "apache-accesslog" ]
|
||||
blacklist_values => [ "uripath", "/index.php",
|
||||
"method", "(HEAD|OPTIONS)",
|
||||
"status", "^[^2]" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="interpolate">
|
||||
interpolate
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Trigger whether configation fields and values should be interpolated for
|
||||
dynamic values.
|
||||
Probably adds some performance overhead. Defaults to false.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="whitelist_names">
|
||||
whitelist_names
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Include only fields only if their names match specified regexps, default to empty list which means include everything.</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
tags => [ "apache-accesslog" ]
|
||||
whitelist_names => [ "method", "(referrer|status)", "${some}_field" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="whitelist_values">
|
||||
whitelist_values
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Include specified fields only if their values match regexps.
|
||||
In case field values are arrays, the fields are pruned on per array item
|
||||
thus only matching array items will be included.</p>
|
||||
|
||||
<pre><code>filter {
|
||||
prune {
|
||||
tags => [ "apache-accesslog" ]
|
||||
whitelist_values => [ "uripath", "/index.php",
|
||||
"method", "(GET|POST)",
|
||||
"status", "^[^2]" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/prune.rb">lib/logstash/filters/prune.rb</a>
|
||||
|
|
@ -1,192 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/railsparallelrequest
|
||||
layout: content_right
|
||||
---
|
||||
<h2>railsparallelrequest</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>parallel request filter</p>
|
||||
|
||||
<p>This filter will separate out the parallel requests into separate events.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
railsparallelrequest {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
railsparallelrequest {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
railsparallelrequest {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
railsparallelrequest {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
railsparallelrequest {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/railsparallelrequest.rb">lib/logstash/filters/railsparallelrequest.rb</a>
|
||||
|
|
@ -1,251 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/range
|
||||
layout: content_right
|
||||
---
|
||||
<h2>range</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>This filter is used to check that certain fields are within expected size/length ranges.
|
||||
Supported types are numbers and strings.
|
||||
Numbers are checked to be within numeric value range.
|
||||
Strings are checked to be within string length range.
|
||||
More than one range can be specified for same fieldname, actions will be applied incrementally.
|
||||
Then field value is with in a specified range and action will be taken
|
||||
supported actions are drop event add tag or add field with specified value.</p>
|
||||
|
||||
<p>Example usecases are for histogram like tagging of events
|
||||
or for finding anomaly values in fields or too big events that should be dropped.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
range {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#negate">negate</a> => ... # boolean (optional), default: false
|
||||
<a href="#ranges">ranges</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
range {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
range {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="negate">
|
||||
negate
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Negate the range match logic, events should be outsize of the specificed range to match.</p>
|
||||
|
||||
<h4>
|
||||
<a name="ranges">
|
||||
ranges
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>An array of field, min, max ,action tuples.
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
range {
|
||||
ranges => [ "message", 0, 10, "tag:short",
|
||||
"message", 11, 100, "tag:medium",
|
||||
"message", 101, 1000, "tag:long",
|
||||
"message", 1001, 1e1000, "drop",
|
||||
"duration", 0, 100, "field:latency:fast",
|
||||
"duration", 101, 200, "field:latency:normal",
|
||||
"duration", 201, 1000, "field:latency:slow",
|
||||
"duration", 1001, 1e1000, "field:latency:outlier"
|
||||
"requests", 0, 10, "tag:to_few_%{source}_requests" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>Supported actions are drop tag or field with specified value.
|
||||
Added tag names and field names and field values can have %{dynamic} values.</p>
|
||||
|
||||
<p>TODO(piavlo): The action syntax is ugly at the moment due to logstash grammar limitations - arrays grammar should support
|
||||
TODO(piavlo): simple not nested hashses as values in addition to numaric and string values to prettify the syntax.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
range {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
range {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/range.rb">lib/logstash/filters/range.rb</a>
|
||||
|
|
@ -1,231 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/ruby
|
||||
layout: content_right
|
||||
---
|
||||
<h2>ruby</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Execute ruby code.</p>
|
||||
|
||||
<p>For example, to cancel 90% of events, you can do this:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
ruby {
|
||||
# Cancel 90% of events
|
||||
code => "event.cancel if rand <= 0.90"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
ruby {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#code">code</a> => ... # string (required)
|
||||
<a href="#init">init</a> => ... # string (optional)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
ruby {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
ruby {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="code">
|
||||
code (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The code to execute for every event.
|
||||
You will have an 'event' variable available that is the event itself.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="init">
|
||||
init
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Any code to execute at logstash startup-time</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
ruby {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
ruby {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/ruby.rb">lib/logstash/filters/ruby.rb</a>
|
||||
|
|
@ -1,286 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/sleep
|
||||
layout: content_right
|
||||
---
|
||||
<h2>sleep</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Sleep a given amount of time. This will cause logstash
|
||||
to stall for the given amount of time. This is useful
|
||||
for rate limiting, etc.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#every">every</a> => ... # string (optional), default: 1
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#replay">replay</a> => ... # boolean (optional), default: false
|
||||
<a href="#time">time</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="every">
|
||||
every
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>Sleep on every N'th. This option is ignored in replay mode.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
time => "1" # Sleep 1 second
|
||||
every => 10 # on every 10th event
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="replay">
|
||||
replay
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Enable replay mode.</p>
|
||||
|
||||
<p>Replay mode tries to sleep based on timestamps in each event.</p>
|
||||
|
||||
<p>The amount of time to sleep is computed by subtracting the
|
||||
previous event's timestamp from the current event's timestamp.
|
||||
This helps you replay events in the same timeline as original.</p>
|
||||
|
||||
<p>If you specify a <code>time</code> setting as well, this filter will
|
||||
use the <code>time</code> value as a speed modifier. For example,
|
||||
a <code>time</code> value of 2 will replay at double speed, while a
|
||||
value of 0.25 will replay at 1/4th speed.</p>
|
||||
|
||||
<p>For example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
time => 2
|
||||
replay => true
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above will sleep in such a way that it will perform
|
||||
replay 2-times faster than the original time speed.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="time">
|
||||
time
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The length of time to sleep, in seconds, for every event.</p>
|
||||
|
||||
<p>This can be a number (eg, 0.5), or a string (eg, "%{foo}")
|
||||
The second form (string with a field value) is useful if
|
||||
you have an attribute of your event that you want to use
|
||||
to indicate the amount of time to sleep.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
sleep {
|
||||
# Sleep 1 second for every event.
|
||||
time => "1"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/sleep.rb">lib/logstash/filters/sleep.rb</a>
|
||||
|
|
@ -1,228 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/split
|
||||
layout: content_right
|
||||
---
|
||||
<h2>split</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>The split filter is for splitting multiline messages into separate events.</p>
|
||||
|
||||
<p>An example use case of this filter is for taking output from the 'exec' input
|
||||
which emits one event for the whole output of a command and splitting that
|
||||
output by newline - making each line an event.</p>
|
||||
|
||||
<p>The end result of each split is a complete copy of the event
|
||||
with only the current split section of the given field changed.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
split {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#field">field</a> => ... # string (optional), default: "message"
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#terminator">terminator</a> => ... # string (optional), default: "\n"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
split {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
split {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="field">
|
||||
field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "message" </li>
|
||||
</ul>
|
||||
|
||||
<p>The field which value is split by the terminator</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
split {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
split {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="terminator">
|
||||
terminator
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "\n" </li>
|
||||
</ul>
|
||||
|
||||
<p>The string to split on. This is usually a line terminator, but can be any
|
||||
string.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/split.rb">lib/logstash/filters/split.rb</a>
|
||||
|
|
@ -1,256 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/syslog_pri
|
||||
layout: content_right
|
||||
---
|
||||
<h2>syslog_pri</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Filter plugin for logstash to parse the PRI field from the front
|
||||
of a Syslog (RFC3164) message. If no priority is set, it will
|
||||
default to 13 (per RFC).</p>
|
||||
|
||||
<p>This filter is based on the original syslog.rb code shipped
|
||||
with logstash.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
syslog_pri {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#facility_labels">facility_labels</a> => ... # array (optional), default: ["kernel", "user-level", "mail", "daemon", "security/authorization", "syslogd", "line printer", "network news", "uucp", "clock", "security/authorization", "ftp", "ntp", "log audit", "log alert", "clock", "local0", "local1", "local2", "local3", "local4", "local5", "local6", "local7"]
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#severity_labels">severity_labels</a> => ... # array (optional), default: ["emergency", "alert", "critical", "error", "warning", "notice", "informational", "debug"]
|
||||
<a href="#syslog_pri_field_name">syslog_pri_field_name</a> => ... # string (optional), default: "syslog_pri"
|
||||
<a href="#use_labels">use_labels</a> => ... # boolean (optional), default: true
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
syslog_pri {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
syslog_pri {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="facility_labels">
|
||||
facility_labels
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["kernel", "user-level", "mail", "daemon", "security/authorization", "syslogd", "line printer", "network news", "uucp", "clock", "security/authorization", "ftp", "ntp", "log audit", "log alert", "clock", "local0", "local1", "local2", "local3", "local4", "local5", "local6", "local7"] </li>
|
||||
</ul>
|
||||
|
||||
<p>Labels for facility levels. This comes from RFC3164.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
syslog_pri {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
syslog_pri {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="severity_labels">
|
||||
severity_labels
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["emergency", "alert", "critical", "error", "warning", "notice", "informational", "debug"] </li>
|
||||
</ul>
|
||||
|
||||
<p>Labels for severity levels. This comes from RFC3164.</p>
|
||||
|
||||
<h4>
|
||||
<a name="syslog_pri_field_name">
|
||||
syslog_pri_field_name
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "syslog_pri" </li>
|
||||
</ul>
|
||||
|
||||
<p>Name of field which passes in the extracted PRI part of the syslog message</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="use_labels">
|
||||
use_labels
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>set the status to experimental/beta/stable
|
||||
Add human-readable names after parsing severity and facility from PRI</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/syslog_pri.rb">lib/logstash/filters/syslog_pri.rb</a>
|
||||
|
|
@ -1,340 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/translate
|
||||
layout: content_right
|
||||
---
|
||||
<h2>translate</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Originally written to translate HTTP response codes
|
||||
but turned into a general translation tool which uses
|
||||
configured has or/and .yaml files as a dictionary.
|
||||
response codes in default dictionary were scraped from
|
||||
'gem install cheat; cheat status_codes'</p>
|
||||
|
||||
<p>Alternatively for simple string search and replacements for just a few values
|
||||
use the gsub function of the mutate filter.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
translate {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#destination">destination</a> => ... # string (optional), default: "translation"
|
||||
<a href="#dictionary">dictionary</a> => ... # hash (optional), default: {}
|
||||
<a href="#dictionary_path">dictionary_path</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#exact">exact</a> => ... # boolean (optional), default: true
|
||||
<a href="#fallback">fallback</a> => ... # string (optional)
|
||||
<a href="#field">field</a> => ... # string (required)
|
||||
<a href="#override">override</a> => ... # boolean (optional), default: false
|
||||
<a href="#regex">regex</a> => ... # boolean (optional), default: false
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
translate {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
translate {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="destination">
|
||||
destination
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "translation" </li>
|
||||
</ul>
|
||||
|
||||
<p>The destination field you wish to populate with the translation code.
|
||||
default is "translation".
|
||||
Set to the same value as source if you want to do a substitution, in this case filter will allways succeed.</p>
|
||||
|
||||
<h4>
|
||||
<a name="dictionary">
|
||||
dictionary
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Dictionary to use for translation.
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
translate {
|
||||
dictionary => [ "100", "Continue",
|
||||
"101", "Switching Protocols",
|
||||
"200", "OK",
|
||||
"201", "Created",
|
||||
"202", "Accepted" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="dictionary_path">
|
||||
dictionary_path
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>name with full path of external dictionary file. <br/>
|
||||
format of the table should be a YAML file which will be merged with the @dictionary.
|
||||
make sure you encase any integer based keys in quotes.
|
||||
The YAML file should look something like this:</p>
|
||||
|
||||
<pre><code>100: Continue
|
||||
101: Switching Protocols
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="exact">
|
||||
exact
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>set to false if you want to match multiple terms
|
||||
a large dictionary could get expensive if set to false.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="fallback">
|
||||
fallback
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Incase no translation was made add default translation string</p>
|
||||
|
||||
<h4>
|
||||
<a name="field">
|
||||
field (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The field containing a response code If this field is an
|
||||
array, only the first value will be used.</p>
|
||||
|
||||
<h4>
|
||||
<a name="override">
|
||||
override
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>In case dstination field already exists should we skip translation(default) or override it with new translation</p>
|
||||
|
||||
<h4>
|
||||
<a name="regex">
|
||||
regex
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>treat dictionary keys as regular expressions to match against, used only then @exact enabled.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
translate {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
translate {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/translate.rb">lib/logstash/filters/translate.rb</a>
|
||||
|
|
@ -1,220 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/urldecode
|
||||
layout: content_right
|
||||
---
|
||||
<h2>urldecode</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>The urldecode filter is for decoding fields that are urlencoded.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
urldecode {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#all_fields">all_fields</a> => ... # boolean (optional), default: false
|
||||
<a href="#field">field</a> => ... # string (optional), default: "message"
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
urldecode {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
urldecode {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="all_fields">
|
||||
all_fields
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Urldecode all fields</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="field">
|
||||
field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "message" </li>
|
||||
</ul>
|
||||
|
||||
<p>The field which value is urldecoded</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
urldecode {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
urldecode {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/urldecode.rb">lib/logstash/filters/urldecode.rb</a>
|
||||
|
|
@ -1,266 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/useragent
|
||||
layout: content_right
|
||||
---
|
||||
<h2>useragent</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Parse user agent strings into structured data based on BrowserScope data</p>
|
||||
|
||||
<p>UserAgent filter, adds information about user agent like family, operating
|
||||
system, version, and device</p>
|
||||
|
||||
<p>Logstash releases ship with the regexes.yaml database made available from
|
||||
ua-parser with an Apache 2.0 license. For more details on ua-parser, see
|
||||
<a href="https://github.com/tobie/ua-parser/">https://github.com/tobie/ua-parser/</a>.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
useragent {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#prefix">prefix</a> => ... # string (optional), default: ""
|
||||
<a href="#regexes">regexes</a> => ... # string (optional)
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#source">source</a> => ... # string (required)
|
||||
<a href="#target">target</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
useragent {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
useragent {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="prefix">
|
||||
prefix
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>A string to prepend to all of the extracted keys</p>
|
||||
|
||||
<h4>
|
||||
<a name="regexes">
|
||||
regexes
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>regexes.yaml file to use</p>
|
||||
|
||||
<p>If not specified, this will default to the regexes.yaml that ships
|
||||
with logstash.</p>
|
||||
|
||||
<p>You can find the latest version of this here:
|
||||
<a href="https://github.com/tobie/ua-parser/blob/master/regexes.yaml">https://github.com/tobie/ua-parser/blob/master/regexes.yaml</a></p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
useragent {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
useragent {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The field containing the user agent string. If this field is an
|
||||
array, only the first value will be used.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="target">
|
||||
target
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the field to assign user agent data into.</p>
|
||||
|
||||
<p>If not specified user agent data will be stored in the root of the event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/useragent.rb">lib/logstash/filters/useragent.rb</a>
|
||||
|
|
@ -1,246 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/uuid
|
||||
layout: content_right
|
||||
---
|
||||
<h2>uuid</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>The uuid filter allows you to add a UUID field to messages.
|
||||
This is useful to be able to control the _id messages are indexed into Elasticsearch
|
||||
with, so that you can insert duplicate messages (i.e. the same message multiple times
|
||||
without creating duplicates) - for log pipeline reliability</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
uuid {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#field">field</a> => ... # string (optional)
|
||||
<a href="#overwrite">overwrite</a> => ... # boolean (optional), default: false
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
uuid {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
uuid {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="field">
|
||||
field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a UUID to a field.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
uuid {
|
||||
field => "@uuid"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="overwrite">
|
||||
overwrite
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>If the value in the field currently (if any) should be overridden
|
||||
by the generated UUID. Defaults to false (i.e. if the field is
|
||||
present, with ANY value, it won't be overridden)</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<p> filter {</p>
|
||||
|
||||
<pre><code> uuid {
|
||||
field => "@uuid"
|
||||
overwrite => true
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p> }</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
uuid {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
uuid {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/uuid.rb">lib/logstash/filters/uuid.rb</a>
|
||||
|
|
@ -1,294 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/xml
|
||||
layout: content_right
|
||||
---
|
||||
<h2>xml</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>XML filter. Takes a field that contains XML and expands it into
|
||||
an actual datastructure.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
xml {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#source">source</a> => ... # string (optional)
|
||||
<a href="#store_xml">store_xml</a> => ... # boolean (optional), default: true
|
||||
<a href="#target">target</a> => ... # string (optional)
|
||||
<a href="#xpath">xpath</a> => ... # hash (optional), default: {}
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
xml {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
xml {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
xml {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
xml {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="source">
|
||||
source
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Config for xml to hash is:</p>
|
||||
|
||||
<pre><code>source => source_field
|
||||
</code></pre>
|
||||
|
||||
<p>For example, if you have the whole xml document in your @message field:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
xml {
|
||||
source => "message"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above would parse the xml from the @message field</p>
|
||||
|
||||
<h4>
|
||||
<a name="store_xml">
|
||||
store_xml
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>By default the filter will store the whole parsed xml in the destination
|
||||
field as described above. Setting this to false will prevent that.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="target">
|
||||
target
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Define target for placing the data</p>
|
||||
|
||||
<p>for example if you want the data to be put in the 'doc' field:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
xml {
|
||||
target => "doc"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>XML in the value of the source field will be expanded into a
|
||||
datastructure in the "target" field.
|
||||
Note: if the "target" field already exists, it will be overridden
|
||||
Required</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="xpath">
|
||||
xpath
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>xpath will additionally select string values (.to_s on whatever is selected)
|
||||
from parsed XML (using each source field defined using the method above)
|
||||
and place those values in the destination fields. Configuration:</p>
|
||||
|
||||
<p>xpath => [ "xpath-syntax", "destination-field" ]</p>
|
||||
|
||||
<p>Values returned by XPath parsring from xpath-synatx will be put in the
|
||||
destination field. Multiple values returned will be pushed onto the
|
||||
destination field as an array. As such, multiple matches across
|
||||
multiple source fields will produce duplicate entries in the field</p>
|
||||
|
||||
<p>More on xpath: http://www.w3schools.com/xpath/</p>
|
||||
|
||||
<p>The xpath functions are particularly powerful:
|
||||
http://www.w3schools.com/xpath/xpath_functions.asp</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/xml.rb">lib/logstash/filters/xml.rb</a>
|
||||
|
|
@ -1,280 +0,0 @@
|
|||
---
|
||||
title: logstash docs for filters/zeromq
|
||||
layout: content_right
|
||||
---
|
||||
<h2>zeromq</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>ZeroMQ filter. This is the best way to send an event externally for filtering
|
||||
It works much like an exec filter would by sending the event "offsite"
|
||||
for processing and waiting for a response</p>
|
||||
|
||||
<p>The protocol here is:
|
||||
* REQ sent with JSON-serialized logstash event
|
||||
* REP read expected to be the full JSON 'filtered' event
|
||||
* - if reply read is an empty string, it will cancel the event.</p>
|
||||
|
||||
<p>Note that this is a limited subset of the zeromq functionality in
|
||||
inputs and outputs. The only topology that makes sense here is:
|
||||
REQ/REP.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>filter {
|
||||
zeromq {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_tag">add_tag</a> => ... # array (optional), default: []
|
||||
<a href="#address">address</a> => ... # string (optional), default: "tcp://127.0.0.1:2121"
|
||||
<a href="#field">field</a> => ... # string (optional)
|
||||
<a href="#mode">mode</a> => ... # string, one of ["server", "client"] (optional), default: "client"
|
||||
<a href="#remove_field">remove_field</a> => ... # array (optional), default: []
|
||||
<a href="#remove_tag">remove_tag</a> => ... # array (optional), default: []
|
||||
<a href="#sockopt">sockopt</a> => ... # hash (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add any arbitrary fields to this event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
zeromq {
|
||||
add_field => [ "foo_%{somefield}", "Hello world, from %{source}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add field "foo_hello" if it is present, with the
|
||||
value above and the %{source} piece replaced with that value from the
|
||||
event.</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_tag">
|
||||
add_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, add arbitrary tags to the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
zeromq {
|
||||
add_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would add a tag "foo_hello"</p>
|
||||
|
||||
<h4>
|
||||
<a name="address">
|
||||
address
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "tcp://127.0.0.1:2121" </li>
|
||||
</ul>
|
||||
|
||||
<p>0mq socket address to connect or bind
|
||||
Please note that inproc:// will not work with logstash
|
||||
as we use a context per thread
|
||||
By default, filters connect</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without all/any (controlled by exclude_any config
|
||||
option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="field">
|
||||
field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The field to send off-site for processing
|
||||
If this is unset, the whole event will be sent
|
||||
TODO (lusis)
|
||||
Allow filtering multiple fields</p>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "server", "client" </li>
|
||||
<li> Default value is "client" </li>
|
||||
</ul>
|
||||
|
||||
<p>0mq mode
|
||||
server mode binds/listens
|
||||
client mode connects</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_field">
|
||||
remove_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary fields from this event.
|
||||
Fields names can be dynamic and include parts of the event using the %{field}
|
||||
Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
zeromq {
|
||||
remove_field => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the field with name "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="remove_tag">
|
||||
remove_tag
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>If this filter is successful, remove arbitrary tags from the event.
|
||||
Tags can be dynamic and include parts of the event using the %{field}
|
||||
syntax. Example:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
zeromq {
|
||||
remove_tag => [ "foo_%{somefield}" ]
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>If the event has field "somefield" == "hello" this filter, on success,
|
||||
would remove the tag "foo_hello" if it is present</p>
|
||||
|
||||
<h4>
|
||||
<a name="sockopt">
|
||||
sockopt
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>0mq socket options
|
||||
This exposes zmq_setsockopt
|
||||
for advanced tuning
|
||||
see http://api.zeromq.org/2-1:zmq-setsockopt for details</p>
|
||||
|
||||
<p>This is where you would set values like:
|
||||
ZMQ::HWM - high water mark
|
||||
ZMQ::IDENTITY - named queues
|
||||
ZMQ::SWAP_SIZE - space for disk overflow
|
||||
ZMQ::SUBSCRIBE - topic filters for pubsub</p>
|
||||
|
||||
<p>example: sockopt => ["ZMQ::HWM", 50, "ZMQ::IDENTITY", "my<em>named</em>queue"]</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all/any (controlled by include_any config option) of these tags.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Note that all of the specified routing options (type,tags.exclude<em>tags,include</em>fields,exclude_fields)
|
||||
must be met in order for the event to be handled by the filter.
|
||||
The type to act on. If a type is given, then this filter will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/filters/zeromq.rb">lib/logstash/filters/zeromq.rb</a>
|
||||
|
|
@ -1,55 +0,0 @@
|
|||
---
|
||||
title: Command-line flags - logstash
|
||||
layout: content_right
|
||||
---
|
||||
# Command-line flags
|
||||
|
||||
## Agent
|
||||
|
||||
The logstash agent has the following flags (also try using the '--help' flag)
|
||||
|
||||
<dl>
|
||||
<dt> -f, --config CONFIGFILE </dt>
|
||||
<dd> Load the logstash config from a specific file, directory, or a
|
||||
wildcard. If given a directory or wildcard, config files will be read
|
||||
from the directory in alphabetical order. </dd>
|
||||
<dt> -e CONFIGSTRING </dt>
|
||||
<dd> Use the given string as the configuration data. Same syntax as the
|
||||
config file. If not input is specified, 'stdin { type => stdin }' is
|
||||
default. If no output is specified, 'stdout { debug => true }}' is
|
||||
default. </dd>
|
||||
<dt> -w, --filterworkers COUNT </dt>
|
||||
<dd> Run COUNT filter workers (default: 1) </dd>
|
||||
<dt> --watchdog-timeout TIMEOUT </dt>
|
||||
<dd> Set watchdog timeout value in seconds. Default is 10.</dd>
|
||||
<dt> -l, --log FILE </dt>
|
||||
<dd> Log to a given path. Default is to log to stdout </dd>
|
||||
<dt> -v </dt>
|
||||
<dd> Increase verbosity. There are multiple levels of verbosity available with
|
||||
'-vv' currently being the highest </dd>
|
||||
<dt> --pluginpath PLUGIN_PATH </dt>
|
||||
<dd> A colon-delimted path to find other logstash plugins in </dd>
|
||||
</dl>
|
||||
|
||||
Note: Plugins can provide addition command-line flags, such as the
|
||||
[grok](filters/grok) filter. Plugin-specific flags always start with the plugin
|
||||
name, like --grok-foo.
|
||||
|
||||
## Web UI
|
||||
|
||||
The logstash web interface has the following flags (also try using the '--help'
|
||||
flag)
|
||||
|
||||
<dl>
|
||||
<dt> --log FILE </dt>
|
||||
<dd> Log to a given path. Default is stdout. </dd>
|
||||
<dt> --address ADDRESS </dt>
|
||||
<dd> Address on which to start webserver. Default is 0.0.0.0. </dd>
|
||||
<dt> --port PORT </dt>
|
||||
<dd> Port on which to start webserver. Default is 9292. </dd>
|
||||
<dt> -B, --elasticsearch-bind-host ADDRESS </dt>
|
||||
<dd> Address on which to bind elastic search node. </dd>
|
||||
<dt> -b, --backend URL </dt>
|
||||
<dd>The backend URL to use. Default is elasticsearch:/// (assumes multicast discovery).
|
||||
You can specify elasticsearch://[host][:port]/[clustername]</dd>
|
||||
</dl>
|
||||
|
|
@ -1,28 +0,0 @@
|
|||
#!/usr/bin/env ruby
|
||||
|
||||
require "erb"
|
||||
|
||||
if ARGV.size != 1
|
||||
$stderr.puts "No path given to search for plugin docs"
|
||||
$stderr.puts "Usage: #{$0} plugin_doc_dir"
|
||||
exit 1
|
||||
end
|
||||
|
||||
def plugins(glob)
|
||||
files = Dir.glob(glob)
|
||||
names = files.collect { |f| File.basename(f).gsub(".html", "") }
|
||||
return names.sort
|
||||
end # def plugins
|
||||
|
||||
basedir = ARGV[0]
|
||||
docs = {
|
||||
"inputs" => plugins(File.join(basedir, "inputs/*.html")),
|
||||
"codecs" => plugins(File.join(basedir, "codecs/*.html")),
|
||||
"filters" => plugins(File.join(basedir, "filters/*.html")),
|
||||
"outputs" => plugins(File.join(basedir, "outputs/*.html")),
|
||||
}
|
||||
|
||||
template_path = File.join(File.dirname(__FILE__), "index.html.erb")
|
||||
template = File.new(template_path).read
|
||||
erb = ERB.new(template, nil, "-")
|
||||
puts erb.result(binding)
|
||||
|
|
@ -1,462 +0,0 @@
|
|||
---
|
||||
title: logstash docs index
|
||||
layout: content_right
|
||||
---
|
||||
<div id="doc_index_container">
|
||||
|
||||
<h3> for users </h3>
|
||||
<ul>
|
||||
<li> <a href="https://logstash.objects.dreamhost.com/release/logstash-1.2.0.beta1-flatjar.jar"> download logstash 1.2.0.beta1 </a> </li>
|
||||
<li> <a href="configuration"> configuration file overview </a> </li>
|
||||
<li> <a href="life-of-an-event"> the life of an event in logstash </a> </li>
|
||||
<li> <a href="flags"> command-line flags </a> </li>
|
||||
<li> <a href="conditionals">conditionals</a> </li>
|
||||
<li> <a href="field-references">event field reference</a> </li>
|
||||
<li> <a href="format-strings">text formatting</a> </li>
|
||||
</ul>
|
||||
|
||||
<h3> for developers </h3>
|
||||
<li> <a href="extending"> writing your own plugins </a> </li>
|
||||
</ul>
|
||||
|
||||
<h3> use cases and tutorials </h3>
|
||||
|
||||
<ul>
|
||||
<li> <a href="tutorials/getting-started-simple"> getting started (standalone) </a> </li>
|
||||
<li> <a href="tutorials/getting-started-centralized"> getting started (centralized) </a> </li>
|
||||
<li> <a href="tutorials/10-minute-walkthrough"> 10-minute walkthrough</a> - a simple walkthrough to show you how to configure the logstash agent to process events and even old logs. </li>
|
||||
<li> <a href="tutorials/metrics-from-logs"> Gathering metrics from logs </a> - take metrics from logs and ship them to graphite, ganglia, and more. </li>
|
||||
<li> <a href="tutorials/just-enough-rabbitmq-for-logstash">Just enough RabbitMQ for Logstash </a> - Get a quick primer on RabbitMQ and how to use it in Logstash! </li>
|
||||
</ul>
|
||||
|
||||
<h3> books and articles </h3>
|
||||
|
||||
<ul>
|
||||
<li> <a href="http://www.logstashbook.com">The LogStash Book </a> - An introductory LogStash book. </li>
|
||||
</ul>
|
||||
|
||||
<h3> plugin documentation </h3>
|
||||
<div class="doc_index_section">
|
||||
<h3>inputs</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<a href="inputs/amqp">amqp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/drupal_dblog">drupal_dblog</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/elasticsearch">elasticsearch</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/eventlog">eventlog</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/exec">exec</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/file">file</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/ganglia">ganglia</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/gelf">gelf</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/gemfire">gemfire</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/generator">generator</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/graphite">graphite</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/heroku">heroku</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/imap">imap</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/irc">irc</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/log4j">log4j</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/lumberjack">lumberjack</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/pipe">pipe</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/rabbitmq">rabbitmq</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/redis">redis</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/relp">relp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/s3">s3</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/snmptrap">snmptrap</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/sqlite">sqlite</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/sqs">sqs</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/stdin">stdin</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/stomp">stomp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/syslog">syslog</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/tcp">tcp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/twitter">twitter</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/udp">udp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/unix">unix</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/varnishlog">varnishlog</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/websocket">websocket</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/wmi">wmi</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/xmpp">xmpp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/zenoss">zenoss</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="inputs/zeromq">zeromq</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="doc_index_section">
|
||||
<h3>codecs</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<a href="codecs/dots">dots</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/json">json</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/json_spooler">json_spooler</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/line">line</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/msgpack">msgpack</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/multiline">multiline</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/noop">noop</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/oldlogstashjson">oldlogstashjson</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/plain">plain</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/rubydebug">rubydebug</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="codecs/spool">spool</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="doc_index_section">
|
||||
<h3>filters</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<a href="filters/advisor">advisor</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/alter">alter</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/anonymize">anonymize</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/checksum">checksum</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/cipher">cipher</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/clone">clone</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/csv">csv</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/date">date</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/dns">dns</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/drop">drop</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/environment">environment</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/gelfify">gelfify</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/geoip">geoip</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/grep">grep</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/grok">grok</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/grokdiscovery">grokdiscovery</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/json">json</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/json_encode">json_encode</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/kv">kv</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/metaevent">metaevent</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/metrics">metrics</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/multiline">multiline</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/mutate">mutate</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/noop">noop</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/prune">prune</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/railsparallelrequest">railsparallelrequest</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/range">range</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/ruby">ruby</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/sleep">sleep</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/split">split</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/syslog_pri">syslog_pri</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/translate">translate</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/urldecode">urldecode</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/useragent">useragent</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/uuid">uuid</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/xml">xml</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="filters/zeromq">zeromq</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
<div class="doc_index_section">
|
||||
<h3>outputs</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<a href="outputs/amqp">amqp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/boundary">boundary</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/circonus">circonus</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/cloudwatch">cloudwatch</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/datadog">datadog</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/datadog_metrics">datadog_metrics</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/elasticsearch">elasticsearch</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/elasticsearch_http">elasticsearch_http</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/elasticsearch_river">elasticsearch_river</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/email">email</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/exec">exec</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/file">file</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/ganglia">ganglia</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/gelf">gelf</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/gemfire">gemfire</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/graphite">graphite</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/graphtastic">graphtastic</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/hipchat">hipchat</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/http">http</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/irc">irc</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/juggernaut">juggernaut</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/librato">librato</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/loggly">loggly</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/lumberjack">lumberjack</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/metriccatcher">metriccatcher</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/mongodb">mongodb</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/nagios">nagios</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/nagios_nsca">nagios_nsca</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/null">null</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/opentsdb">opentsdb</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/pagerduty">pagerduty</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/pipe">pipe</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/rabbitmq">rabbitmq</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/redis">redis</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/riak">riak</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/riemann">riemann</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/s3">s3</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/sns">sns</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/sqs">sqs</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/statsd">statsd</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/stdout">stdout</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/stomp">stomp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/syslog">syslog</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/tcp">tcp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/udp">udp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/websocket">websocket</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/xmpp">xmpp</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/zabbix">zabbix</a>
|
||||
</li>
|
||||
<li>
|
||||
<a href="outputs/zeromq">zeromq</a>
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
<div class="clear"></div>
|
||||
|
|
@ -1,53 +0,0 @@
|
|||
---
|
||||
title: logstash docs index
|
||||
layout: content_right
|
||||
---
|
||||
<div id="doc_index_container">
|
||||
|
||||
<h3> for users </h3>
|
||||
<ul>
|
||||
<li> <a href="https://logstash.objects.dreamhost.com/release/logstash-1.2.0.beta1-flatjar.jar"> download logstash 1.2.0.beta1 </a> </li>
|
||||
<li> <a href="configuration"> configuration file overview </a> </li>
|
||||
<li> <a href="life-of-an-event"> the life of an event in logstash </a> </li>
|
||||
<li> <a href="flags"> command-line flags </a> </li>
|
||||
<li> <a href="conditionals">conditionals</a> </li>
|
||||
<li> <a href="field-references">event field reference</a> </li>
|
||||
<li> <a href="format-strings">text formatting</a> </li>
|
||||
</ul>
|
||||
|
||||
<h3> for developers </h3>
|
||||
<li> <a href="extending"> writing your own plugins </a> </li>
|
||||
</ul>
|
||||
|
||||
<h3> use cases and tutorials </h3>
|
||||
|
||||
<ul>
|
||||
<li> <a href="tutorials/getting-started-simple"> getting started (standalone) </a> </li>
|
||||
<li> <a href="tutorials/getting-started-centralized"> getting started (centralized) </a> </li>
|
||||
<li> <a href="tutorials/10-minute-walkthrough"> 10-minute walkthrough</a> - a simple walkthrough to show you how to configure the logstash agent to process events and even old logs. </li>
|
||||
<li> <a href="tutorials/metrics-from-logs"> Gathering metrics from logs </a> - take metrics from logs and ship them to graphite, ganglia, and more. </li>
|
||||
<li> <a href="tutorials/just-enough-rabbitmq-for-logstash">Just enough RabbitMQ for Logstash </a> - Get a quick primer on RabbitMQ and how to use it in Logstash! </li>
|
||||
</ul>
|
||||
|
||||
<h3> books and articles </h3>
|
||||
|
||||
<ul>
|
||||
<li> <a href="http://www.logstashbook.com">The LogStash Book </a> - An introductory LogStash book. </li>
|
||||
</ul>
|
||||
|
||||
<h3> plugin documentation </h3>
|
||||
<% docs.each do |type, paths| -%>
|
||||
<div class="doc_index_section">
|
||||
<h3><%= type %></h3>
|
||||
<ul>
|
||||
<% paths.each do |path| -%>
|
||||
<% name = File.basename(path).gsub(".html", "") -%>
|
||||
<li>
|
||||
<a href="<%= "#{type}/#{name}" %>"><%= name %></a>
|
||||
</li>
|
||||
<% end -%>
|
||||
</ul>
|
||||
</div>
|
||||
<% end -%>
|
||||
</div>
|
||||
<div class="clear"></div>
|
||||
|
|
@ -1,445 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/amqp
|
||||
layout: content_right
|
||||
---
|
||||
<h2>amqp</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
amqp {
|
||||
<a href="#ack">ack</a> => ... # boolean (optional), default: true
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#arguments">arguments</a> => ... # array (optional), default: {}
|
||||
<a href="#auto_delete">auto_delete</a> => ... # boolean (optional), default: true
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#durable">durable</a> => ... # boolean (optional), default: false
|
||||
<a href="#exchange">exchange</a> => ... # string (optional)
|
||||
<a href="#exclusive">exclusive</a> => ... # boolean (optional), default: true
|
||||
<a href="#host">host</a> => ... # string (required)
|
||||
<a href="#key">key</a> => ... # string (optional), default: "logstash"
|
||||
<a href="#passive">passive</a> => ... # boolean (optional), default: false
|
||||
<a href="#password">password</a> => ... # password (optional), default: "guest"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 5672
|
||||
<a href="#prefetch_count">prefetch_count</a> => ... # number (optional), default: 256
|
||||
<a href="#queue">queue</a> => ... # string (optional), default: ""
|
||||
<a href="#ssl">ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#user">user</a> => ... # string (optional), default: "guest"
|
||||
<a href="#verify_ssl">verify_ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#vhost">vhost</a> => ... # string (optional), default: "/"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="ack">
|
||||
ack
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="arguments">
|
||||
arguments
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="auto_delete">
|
||||
auto_delete
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="durable">
|
||||
durable
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="exchange">
|
||||
exchange
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="exclusive">
|
||||
exclusive
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="key">
|
||||
key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="passive">
|
||||
passive
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> Default value is "guest" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5672 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="prefetch_count">
|
||||
prefetch_count
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 256 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="queue">
|
||||
queue
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl">
|
||||
ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "guest" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="verify_ssl">
|
||||
verify_ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="vhost">
|
||||
vhost
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "/" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/amqp.rb">lib/logstash/inputs/amqp.rb</a>
|
||||
|
|
@ -1,250 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/drupal_dblog
|
||||
layout: content_right
|
||||
---
|
||||
<h2>drupal_dblog</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Retrieve watchdog log events from a Drupal installation with DBLog enabled.
|
||||
The events are pulled out directly from the database.
|
||||
The original events are not deleted, and on every consecutive run only new
|
||||
events are pulled.</p>
|
||||
|
||||
<p>The last watchdog event id that was processed is stored in the Drupal
|
||||
variable table with the name "logstash<em>last</em>wid". Delete this variable or
|
||||
set it to 0 if you want to re-import all events.</p>
|
||||
|
||||
<p>More info on DBLog: http://drupal.org/documentation/modules/dblog</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
drupal_dblog {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#add_usernames">add_usernames</a> => ... # boolean (optional), default: false
|
||||
<a href="#bulksize">bulksize</a> => ... # number (optional), default: 5000
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#databases">databases</a> => ... # hash (optional)
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#interval">interval</a> => ... # number (optional), default: 10
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional), default: "watchdog"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_usernames">
|
||||
add_usernames
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>By default, the event only contains the current user id as a field.
|
||||
If you whish to add the username as an additional field, set this to true.</p>
|
||||
|
||||
<h4>
|
||||
<a name="bulksize">
|
||||
bulksize
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5000 </li>
|
||||
</ul>
|
||||
|
||||
<p>The amount of log messages that should be fetched with each query.
|
||||
Bulk fetching is done to prevent querying huge data sets when lots of
|
||||
messages are in the database.</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="databases">
|
||||
databases
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Specify all drupal databases that you whish to import from.
|
||||
This can be as many as you whish.
|
||||
The format is a hash, with a unique site name as the key, and a databse
|
||||
url as the value.</p>
|
||||
|
||||
<p>Example:
|
||||
[
|
||||
"site1", "mysql://user1:password@host1.com/databasename",
|
||||
"other_site", "mysql://user2:password@otherhost.com/databasename",
|
||||
...
|
||||
]</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="interval">
|
||||
interval
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 10 </li>
|
||||
</ul>
|
||||
|
||||
<p>Time between checks in minutes.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "watchdog" </li>
|
||||
</ul>
|
||||
|
||||
<p>Label this input with a type.
|
||||
Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/drupal_dblog.rb">lib/logstash/inputs/drupal_dblog.rb</a>
|
||||
|
|
@ -1,254 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/elasticsearch
|
||||
layout: content_right
|
||||
---
|
||||
<h2>elasticsearch</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read from elasticsearch.</p>
|
||||
|
||||
<p>This is useful for replay testing logs, reindexing, etc.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>input {
|
||||
# Read all documents from elasticsearch matching the given query
|
||||
elasticsearch {
|
||||
host => "localhost"
|
||||
query => "ERROR"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<ul>
|
||||
<li>TODO(sissel): configurable scroll timeout</li>
|
||||
<li>TODO(sissel): Option to keep the index, type, and doc id so we can do reindexing?</li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
elasticsearch {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (required)
|
||||
<a href="#index">index</a> => ... # string (optional), default: "logstash-*"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 9200
|
||||
<a href="#query">query</a> => ... # string (optional), default: "*"
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The address of your elasticsearch server</p>
|
||||
|
||||
<h4>
|
||||
<a name="index">
|
||||
index
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash-*" </li>
|
||||
</ul>
|
||||
|
||||
<p>The index to search</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 9200 </li>
|
||||
</ul>
|
||||
|
||||
<p>The http port of your elasticsearch server's REST interface</p>
|
||||
|
||||
<h4>
|
||||
<a name="query">
|
||||
query
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "*" </li>
|
||||
</ul>
|
||||
|
||||
<p>The query to use</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/elasticsearch.rb">lib/logstash/inputs/elasticsearch.rb</a>
|
||||
|
|
@ -1,200 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/eventlog
|
||||
layout: content_right
|
||||
---
|
||||
<h2>eventlog</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Pull events from a Windows Event Log</p>
|
||||
|
||||
<p>To collect Events from the System Event Log, use a config like:</p>
|
||||
|
||||
<pre><code>input {
|
||||
eventlog {
|
||||
type => 'Win32-EventLog'
|
||||
logfile => 'System'
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
eventlog {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#logfile">logfile</a> => ... # array (optional), default: ["Application", "Security", "System"]
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="logfile">
|
||||
logfile
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["Application", "Security", "System"] </li>
|
||||
</ul>
|
||||
|
||||
<p>Event Log Name</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/eventlog.rb">lib/logstash/inputs/eventlog.rb</a>
|
||||
|
|
@ -1,214 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/exec
|
||||
layout: content_right
|
||||
---
|
||||
<h2>exec</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Run command line tools and capture the whole output as an event.</p>
|
||||
|
||||
<p>Notes:</p>
|
||||
|
||||
<ul>
|
||||
<li>The '@source' of this event will be the command run.</li>
|
||||
<li>The '@message' of this event will be the entire stdout of the command
|
||||
as one event.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
exec {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#command">command</a> => ... # string (required)
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#interval">interval</a> => ... # number (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="command">
|
||||
command (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Command to run. For example, "uptime"</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="interval">
|
||||
interval (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Interval to run the command. Value is in seconds.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/exec.rb">lib/logstash/inputs/exec.rb</a>
|
||||
|
|
@ -1,311 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/file
|
||||
layout: content_right
|
||||
---
|
||||
<h2>file</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Stream events from files.</p>
|
||||
|
||||
<p>By default, each event is assumed to be one line. If you
|
||||
want to join lines, you'll want to use the multiline filter.</p>
|
||||
|
||||
<p>Files are followed in a manner similar to "tail -0F". File rotation
|
||||
is detected and handled by this input.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
file {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#discover_interval">discover_interval</a> => ... # number (optional), default: 15
|
||||
<a href="#exclude">exclude</a> => ... # array (optional)
|
||||
<a href="#path">path</a> => ... # array (required)
|
||||
<a href="#sincedb_path">sincedb_path</a> => ... # string (optional)
|
||||
<a href="#sincedb_write_interval">sincedb_write_interval</a> => ... # number (optional), default: 15
|
||||
<a href="#start_position">start_position</a> => ... # string, one of ["beginning", "end"] (optional), default: "end"
|
||||
<a href="#stat_interval">stat_interval</a> => ... # number (optional), default: 1
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="discover_interval">
|
||||
discover_interval
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 15 </li>
|
||||
</ul>
|
||||
|
||||
<p>How often we expand globs to discover new files to watch.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude">
|
||||
exclude
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Exclusions (matched against the filename, not full path). Globs
|
||||
are valid here, too. For example, if you have</p>
|
||||
|
||||
<pre><code>path => "/var/log/*"
|
||||
</code></pre>
|
||||
|
||||
<p>you might want to exclude gzipped files:</p>
|
||||
|
||||
<pre><code>exclude => "*.gz"
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="path">
|
||||
path (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>TODO(sissel): This should switch to use the 'line' codec by default
|
||||
once file following
|
||||
The path to the file to use as an input.
|
||||
You can use globs here, such as <code>/var/log/*.log</code>
|
||||
Paths must be absolute and cannot be relative.</p>
|
||||
|
||||
<h4>
|
||||
<a name="sincedb_path">
|
||||
sincedb_path
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Where to write the since database (keeps track of the current
|
||||
position of monitored log files). The default will write
|
||||
sincedb files to some path matching "$HOME/.sincedb*"</p>
|
||||
|
||||
<h4>
|
||||
<a name="sincedb_write_interval">
|
||||
sincedb_write_interval
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 15 </li>
|
||||
</ul>
|
||||
|
||||
<p>How often to write a since database with the current position of
|
||||
monitored log files.</p>
|
||||
|
||||
<h4>
|
||||
<a name="start_position">
|
||||
start_position
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "beginning", "end" </li>
|
||||
<li> Default value is "end" </li>
|
||||
</ul>
|
||||
|
||||
<p>Choose where logstash starts initially reading files - at the beginning or
|
||||
at the end. The default behavior treats files like live streams and thus
|
||||
starts at the end. If you have old data you want to import, set this
|
||||
to 'beginning'</p>
|
||||
|
||||
<p>This option only modifieds "first contact" situations where a file is new
|
||||
and not seen before. If a file has already been seen before, this option
|
||||
has no effect.</p>
|
||||
|
||||
<h4>
|
||||
<a name="stat_interval">
|
||||
stat_interval
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>How often we stat files to see if they have been modified. Increasing
|
||||
this interval will decrease the number of system calls we make, but
|
||||
increase the time to detect new log lines.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/file.rb">lib/logstash/inputs/file.rb</a>
|
||||
|
|
@ -1,206 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/ganglia
|
||||
layout: content_right
|
||||
---
|
||||
<h2>ganglia</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read ganglia packets from the network via udp</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
ganglia {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 8649
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>The address to listen on</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 8649 </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to listen on. Remember that ports less than 1024 (privileged
|
||||
ports) may require root to use.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/ganglia.rb">lib/logstash/inputs/ganglia.rb</a>
|
||||
|
|
@ -1,239 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/gelf
|
||||
layout: content_right
|
||||
---
|
||||
<h2>gelf</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Read gelf messages as events over the network.</p>
|
||||
|
||||
<p>This input is a good choice if you already use graylog2 today.</p>
|
||||
|
||||
<p>The main reasoning for this input is to leverage existing GELF
|
||||
logging libraries such as the gelf log4j appender</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
gelf {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 12201
|
||||
<a href="#remap">remap</a> => ... # boolean (optional), default: true
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>The address to listen on</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 12201 </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to listen on. Remember that ports less than 1024 (privileged
|
||||
ports) may require root to use.</p>
|
||||
|
||||
<h4>
|
||||
<a name="remap">
|
||||
remap
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Whether or not to remap the gelf message fields to logstash event fields or
|
||||
leave them intact.</p>
|
||||
|
||||
<p>Default is true</p>
|
||||
|
||||
<p>Remapping converts the following gelf fields to logstash equivalents:</p>
|
||||
|
||||
<ul>
|
||||
<li>event["message"] becomes full<em>message
|
||||
if no full</em>message, use event["message"] becomes short<em>message
|
||||
if no short</em>message, event["message"] is the raw json input</li>
|
||||
<li>host + file to event["source"]</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/gelf.rb">lib/logstash/inputs/gelf.rb</a>
|
||||
|
|
@ -1,306 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/gemfire
|
||||
layout: content_right
|
||||
---
|
||||
<h2>gemfire</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Push events to a GemFire region.</p>
|
||||
|
||||
<p>GemFire is an object database.</p>
|
||||
|
||||
<p>To use this plugin you need to add gemfire.jar to your CLASSPATH.
|
||||
Using format=json requires jackson.jar too; use of continuous
|
||||
queries requires antlr.jar.</p>
|
||||
|
||||
<p>Note: this plugin has only been tested with GemFire 7.0.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
gemfire {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#cache_name">cache_name</a> => ... # string (optional), default: "logstash"
|
||||
<a href="#cache_xml_file">cache_xml_file</a> => ... # string (optional), default: nil
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#interest_regexp">interest_regexp</a> => ... # string (optional), default: ".*"
|
||||
<a href="#query">query</a> => ... # string (optional), default: nil
|
||||
<a href="#region_name">region_name</a> => ... # string (optional), default: "Logstash"
|
||||
<a href="#serialization">serialization</a> => ... # string (optional), default: nil
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="cache_name">
|
||||
cache_name
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash" </li>
|
||||
</ul>
|
||||
|
||||
<p>Your client cache name</p>
|
||||
|
||||
<h4>
|
||||
<a name="cache_xml_file">
|
||||
cache_xml_file
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>The path to a GemFire client cache XML file.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code> <client-cache>
|
||||
<pool name="client-pool" subscription-enabled="true" subscription-redundancy="1">
|
||||
<locator host="localhost" port="31331"/>
|
||||
</pool>
|
||||
<region name="Logstash">
|
||||
<region-attributes refid="CACHING_PROXY" pool-name="client-pool" >
|
||||
</region-attributes>
|
||||
</region>
|
||||
</client-cache>
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="interest_regexp">
|
||||
interest_regexp
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is ".*" </li>
|
||||
</ul>
|
||||
|
||||
<p>A regexp to use when registering interest for cache events.
|
||||
Ignored if a :query is specified.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="query">
|
||||
query
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>A query to run as a GemFire "continuous query"; if specified it takes
|
||||
precedence over :interest_regexp which will be ignore.</p>
|
||||
|
||||
<p>Important: use of continuous queries requires subscriptions to be enabled on the client pool.</p>
|
||||
|
||||
<h4>
|
||||
<a name="region_name">
|
||||
region_name
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "Logstash" </li>
|
||||
</ul>
|
||||
|
||||
<p>The region name</p>
|
||||
|
||||
<h4>
|
||||
<a name="serialization">
|
||||
serialization
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>How the message is serialized in the cache. Can be one of "json" or "plain"; default is plain</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to the number of threads you want this input to spawn.
|
||||
This is the same as declaring the input multiple times</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/gemfire.rb">lib/logstash/inputs/gemfire.rb</a>
|
||||
|
|
@ -1,266 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/generator
|
||||
layout: content_right
|
||||
---
|
||||
<h2>generator</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>Generate random log events.</p>
|
||||
|
||||
<p>The general intention of this is to test performance of plugins.</p>
|
||||
|
||||
<p>An event is generated first</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
generator {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#count">count</a> => ... # number (optional), default: 0
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#lines">lines</a> => ... # array (optional)
|
||||
<a href="#message">message</a> => ... # string (optional), default: "Hello world!"
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="count">
|
||||
count
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 0 </li>
|
||||
</ul>
|
||||
|
||||
<p>Set how many messages should be generated.</p>
|
||||
|
||||
<p>The default, 0, means generate an unlimited number of events.</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="lines">
|
||||
lines
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The lines to emit, in order. This option cannot be used with the 'message'
|
||||
setting.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>input {
|
||||
generator {
|
||||
lines => [
|
||||
"line 1",
|
||||
"line 2",
|
||||
"line 3"
|
||||
]
|
||||
}
|
||||
|
||||
# Emit all lines 3 times.
|
||||
count => 3
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>The above will emit "line 1" then "line 2" then "line", then "line 1", etc...</p>
|
||||
|
||||
<h4>
|
||||
<a name="message">
|
||||
message
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "Hello world!" </li>
|
||||
</ul>
|
||||
|
||||
<p>The message string to use in the event.</p>
|
||||
|
||||
<p>If you set this to 'stdin' then this plugin will read a single line from
|
||||
stdin and use that as the message string for every event.</p>
|
||||
|
||||
<p>Otherwise, this value will be used verbatim as the event message.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to the number of threads you want this input to spawn.
|
||||
This is the same as declaring the input multiple times</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/generator.rb">lib/logstash/inputs/generator.rb</a>
|
||||
|
|
@ -1,325 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/graphite
|
||||
layout: content_right
|
||||
---
|
||||
<h2>graphite</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
graphite {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#data_timeout">data_timeout</a> => ... # number (optional), default: -1
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#mode">mode</a> => ... # string, one of ["server", "client"] (optional), default: "server"
|
||||
<a href="#port">port</a> => ... # number (required)
|
||||
<a href="#ssl_cacert">ssl_cacert</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#ssl_cert">ssl_cert</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#ssl_enable">ssl_enable</a> => ... # boolean (optional), default: false
|
||||
<a href="#ssl_key">ssl_key</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#ssl_key_passphrase">ssl_key_passphrase</a> => ... # password (optional), default: nil
|
||||
<a href="#ssl_verify">ssl_verify</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="data_timeout">
|
||||
data_timeout
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is -1 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "server", "client" </li>
|
||||
<li> Default value is "server" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl_cacert">
|
||||
ssl_cacert
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl_cert">
|
||||
ssl_cert
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl_enable">
|
||||
ssl_enable
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl_key">
|
||||
ssl_key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl_key_passphrase">
|
||||
ssl_key_passphrase
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl_verify">
|
||||
ssl_verify
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/graphite.rb">lib/logstash/inputs/graphite.rb</a>
|
||||
|
|
@ -1,204 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/heroku
|
||||
layout: content_right
|
||||
---
|
||||
<h2>heroku</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Stream events from a heroku app's logs.</p>
|
||||
|
||||
<p>This will read events in a manner similar to how the <code>heroku logs -t</code> command
|
||||
fetches logs.</p>
|
||||
|
||||
<p>Recommended filters:</p>
|
||||
|
||||
<pre><code>filter {
|
||||
grok {
|
||||
pattern => "^%{TIMESTAMP_ISO8601:timestamp} %{WORD:component}\[%{WORD:process}(?:\.%{INT:instance:int})?\]: %{DATA:message}$"
|
||||
}
|
||||
date { timestamp => ISO8601 }
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
heroku {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#app">app</a> => ... # string (required)
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="app">
|
||||
app (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of your heroku application. This is usually the first part of the
|
||||
the domain name 'my-app-name.herokuapp.com'</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/heroku.rb">lib/logstash/inputs/heroku.rb</a>
|
||||
|
|
@ -1,313 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/imap
|
||||
layout: content_right
|
||||
---
|
||||
<h2>imap</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read mail from IMAP servers</p>
|
||||
|
||||
<p>Periodically scans INBOX and moves any read messages
|
||||
to the trash.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
imap {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#check_interval">check_interval</a> => ... # number (optional), default: 300
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#delete">delete</a> => ... # boolean (optional), default: false
|
||||
<a href="#fetch_count">fetch_count</a> => ... # number (optional), default: 50
|
||||
<a href="#host">host</a> => ... # string (required)
|
||||
<a href="#lowercase_headers">lowercase_headers</a> => ... # boolean (optional), default: true
|
||||
<a href="#password">password</a> => ... # password (required)
|
||||
<a href="#port">port</a> => ... # number (optional)
|
||||
<a href="#secure">secure</a> => ... # boolean (optional), default: true
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#user">user</a> => ... # string (required)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="check_interval">
|
||||
check_interval
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 300 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="delete">
|
||||
delete
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="fetch_count">
|
||||
fetch_count
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 50 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="lowercase_headers">
|
||||
lowercase_headers
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="secure">
|
||||
secure
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/imap.rb">lib/logstash/inputs/imap.rb</a>
|
||||
|
|
@ -1,298 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/irc
|
||||
layout: content_right
|
||||
---
|
||||
<h2>irc</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read events from an IRC Server.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
irc {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#channels">channels</a> => ... # array (required)
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (required)
|
||||
<a href="#nick">nick</a> => ... # string (optional), default: "logstash"
|
||||
<a href="#password">password</a> => ... # password (optional)
|
||||
<a href="#port">port</a> => ... # number (optional), default: 6667
|
||||
<a href="#real">real</a> => ... # string (optional), default: "logstash"
|
||||
<a href="#secure">secure</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#user">user</a> => ... # string (optional), default: "logstash"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="channels">
|
||||
channels (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Channels to join and read messages from.</p>
|
||||
|
||||
<p>These should be full channel names including the '#' symbol, such as
|
||||
"#logstash".</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Host of the IRC Server to connect to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="nick">
|
||||
nick
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash" </li>
|
||||
</ul>
|
||||
|
||||
<p>IRC Nickname</p>
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>IRC Server password</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 6667 </li>
|
||||
</ul>
|
||||
|
||||
<p>Port for the IRC Server</p>
|
||||
|
||||
<h4>
|
||||
<a name="real">
|
||||
real
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash" </li>
|
||||
</ul>
|
||||
|
||||
<p>IRC Real name</p>
|
||||
|
||||
<h4>
|
||||
<a name="secure">
|
||||
secure
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable SSL.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash" </li>
|
||||
</ul>
|
||||
|
||||
<p>IRC Username</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/irc.rb">lib/logstash/inputs/irc.rb</a>
|
||||
|
|
@ -1,244 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/log4j
|
||||
layout: content_right
|
||||
---
|
||||
<h2>log4j</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read events over a TCP socket from Log4j SocketAppender.</p>
|
||||
|
||||
<p>Can either accept connections from clients or connect to a server,
|
||||
depending on <code>mode</code>. Depending on mode, you need a matching SocketAppender or SocketHubAppender on the remote side</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
log4j {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#data_timeout">data_timeout</a> => ... # number (optional), default: 5
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#mode">mode</a> => ... # string, one of ["server", "client"] (optional), default: "server"
|
||||
<a href="#port">port</a> => ... # number (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="data_timeout">
|
||||
data_timeout
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5 </li>
|
||||
</ul>
|
||||
|
||||
<p>Read timeout in seconds. If a particular tcp connection is
|
||||
idle for more than this timeout period, we will assume
|
||||
it is dead and close it.
|
||||
If you never want to timeout, use -1.</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>When mode is <code>server</code>, the address to listen on.
|
||||
When mode is <code>client</code>, the address to connect to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "server", "client" </li>
|
||||
<li> Default value is "server" </li>
|
||||
</ul>
|
||||
|
||||
<p>Mode to operate in. <code>server</code> listens for client connections,
|
||||
<code>client</code> connects to a server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>When mode is <code>server</code>, the port to listen on.
|
||||
When mode is <code>client</code>, the port to connect to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/log4j.rb">lib/logstash/inputs/log4j.rb</a>
|
||||
|
|
@ -1,253 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/lumberjack
|
||||
layout: content_right
|
||||
---
|
||||
<h2>lumberjack</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Receive events using the lumberjack protocol.</p>
|
||||
|
||||
<p>This is mainly to receive events shipped with lumberjack,
|
||||
<a href="http://github.com/jordansissel/lumberjack">http://github.com/jordansissel/lumberjack</a></p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
lumberjack {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#port">port</a> => ... # number (required)
|
||||
<a href="#ssl_certificate">ssl_certificate</a> => ... # a valid filesystem path (required)
|
||||
<a href="#ssl_key">ssl_key</a> => ... # a valid filesystem path (required)
|
||||
<a href="#ssl_key_passphrase">ssl_key_passphrase</a> => ... # password (optional)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>the address to listen on.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>the port to listen on.</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_certificate">
|
||||
ssl_certificate (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>ssl certificate to use</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_key">
|
||||
ssl_key (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>ssl key to use</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_key_passphrase">
|
||||
ssl_key_passphrase
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>ssl key passphrase to use</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/lumberjack.rb">lib/logstash/inputs/lumberjack.rb</a>
|
||||
|
|
@ -1,199 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/pipe
|
||||
layout: content_right
|
||||
---
|
||||
<h2>pipe</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Stream events from a long running command pipe.</p>
|
||||
|
||||
<p>By default, each event is assumed to be one line. If you
|
||||
want to join lines, you'll want to use the multiline filter.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
pipe {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#command">command</a> => ... # string (required)
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="command">
|
||||
command (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>TODO(sissel): This should switch to use the 'line' codec by default
|
||||
once we switch away from doing 'readline'
|
||||
Command to run and read events from, one line at a time.</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<p> command => "echo hello world"</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/pipe.rb">lib/logstash/inputs/pipe.rb</a>
|
||||
|
|
@ -1,479 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/rabbitmq
|
||||
layout: content_right
|
||||
---
|
||||
<h2>rabbitmq</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Pull events from a RabbitMQ exchange.</p>
|
||||
|
||||
<p>The default settings will create an entirely transient queue and listen for all messages by default.
|
||||
If you need durability or any other advanced settings, please set the appropriate options</p>
|
||||
|
||||
<p>This has been tested with Bunny 0.9.x, which supports RabbitMQ 2.x and 3.x. You can
|
||||
find links to both here:</p>
|
||||
|
||||
<ul>
|
||||
<li>RabbitMQ - <a href="http://www.rabbitmq.com/">http://www.rabbitmq.com/</a></li>
|
||||
<li>Bunny - <a href="https://github.com/ruby-amqp/bunny">https://github.com/ruby-amqp/bunny</a></li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
rabbitmq {
|
||||
<a href="#ack">ack</a> => ... # boolean (optional), default: true
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#arguments">arguments</a> => ... # array (optional), default: {}
|
||||
<a href="#auto_delete">auto_delete</a> => ... # boolean (optional), default: true
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#durable">durable</a> => ... # boolean (optional), default: false
|
||||
<a href="#exchange">exchange</a> => ... # string (optional)
|
||||
<a href="#exclusive">exclusive</a> => ... # boolean (optional), default: true
|
||||
<a href="#host">host</a> => ... # string (required)
|
||||
<a href="#key">key</a> => ... # string (optional), default: "logstash"
|
||||
<a href="#passive">passive</a> => ... # boolean (optional), default: false
|
||||
<a href="#password">password</a> => ... # password (optional), default: "guest"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 5672
|
||||
<a href="#prefetch_count">prefetch_count</a> => ... # number (optional), default: 256
|
||||
<a href="#queue">queue</a> => ... # string (optional), default: ""
|
||||
<a href="#ssl">ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#user">user</a> => ... # string (optional), default: "guest"
|
||||
<a href="#verify_ssl">verify_ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#vhost">vhost</a> => ... # string (optional), default: "/"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="ack">
|
||||
ack
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Enable message acknowledgement</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="arguments">
|
||||
arguments
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Extra queue arguments as an array.
|
||||
To make a RabbitMQ queue mirrored, use: {"x-ha-policy" => "all"}</p>
|
||||
|
||||
<h4>
|
||||
<a name="auto_delete">
|
||||
auto_delete
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Should the queue be deleted on the broker when the last consumer
|
||||
disconnects? Set this option to 'false' if you want the queue to remain
|
||||
on the broker, queueing up messages until a consumer comes along to
|
||||
consume them.</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Enable or disable logging</p>
|
||||
|
||||
<h4>
|
||||
<a name="durable">
|
||||
durable
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Is this queue durable? (aka; Should it survive a broker restart?)</p>
|
||||
|
||||
<h4>
|
||||
<a name="exchange">
|
||||
exchange
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>(Optional, backwards compatibility) Exchange binding</p>
|
||||
|
||||
<p>Optional.</p>
|
||||
|
||||
<p>The name of the exchange to bind the queue to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclusive">
|
||||
exclusive
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Is the queue exclusive? (aka: Will other clients connect to this named queue?)</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Connection</p>
|
||||
|
||||
<p>RabbitMQ server address</p>
|
||||
|
||||
<h4>
|
||||
<a name="key">
|
||||
key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash" </li>
|
||||
</ul>
|
||||
|
||||
<p>Optional.</p>
|
||||
|
||||
<p>The routing key to use when binding a queue to the exchange.
|
||||
This is only relevant for direct or topic exchanges.</p>
|
||||
|
||||
<ul>
|
||||
<li>Routing keys are ignored on fanout exchanges.</li>
|
||||
<li>Wildcards are not valid on direct exchanges.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="passive">
|
||||
passive
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Passive queue creation? Useful for checking queue existance without modifying server state</p>
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> Default value is "guest" </li>
|
||||
</ul>
|
||||
|
||||
<p>RabbitMQ password</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5672 </li>
|
||||
</ul>
|
||||
|
||||
<p>RabbitMQ port to connect on</p>
|
||||
|
||||
<h4>
|
||||
<a name="prefetch_count">
|
||||
prefetch_count
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 256 </li>
|
||||
</ul>
|
||||
|
||||
<p>Prefetch count. Number of messages to prefetch</p>
|
||||
|
||||
<h4>
|
||||
<a name="queue">
|
||||
queue
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>Queue & Consumer</p>
|
||||
|
||||
<p>The name of the queue Logstash will consume events from.</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl">
|
||||
ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Enable or disable SSL</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to the number of threads you want this input to spawn.
|
||||
This is the same as declaring the input multiple times</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "guest" </li>
|
||||
</ul>
|
||||
|
||||
<p>RabbitMQ username</p>
|
||||
|
||||
<h4>
|
||||
<a name="verify_ssl">
|
||||
verify_ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Validate SSL certificate</p>
|
||||
|
||||
<h4>
|
||||
<a name="vhost">
|
||||
vhost
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "/" </li>
|
||||
</ul>
|
||||
|
||||
<p>The vhost to use. If you don't know what this is, leave the default.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/rabbitmq.rb">lib/logstash/inputs/rabbitmq.rb</a>
|
||||
|
|
@ -1,355 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/redis
|
||||
layout: content_right
|
||||
---
|
||||
<h2>redis</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Read events from a redis. Supports both redis channels and also redis lists
|
||||
(using BLPOP)</p>
|
||||
|
||||
<p>For more information about redis, see <a href="http://redis.io/">http://redis.io/</a></p>
|
||||
|
||||
<h2><code>batch_count</code> note</h2>
|
||||
|
||||
<p>If you use the 'batch_count' setting, you <em>must</em> use a redis version 2.6.0 or
|
||||
newer. Anything older does not support the operations used by batching.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
redis {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#batch_count">batch_count</a> => ... # number (optional), default: 1
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#data_type">data_type</a> => ... # string, one of ["list", "channel", "pattern_channel"] (optional)
|
||||
<a href="#db">db</a> => ... # number (optional), default: 0
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "127.0.0.1"
|
||||
<a href="#key">key</a> => ... # string (optional)
|
||||
<a href="#password">password</a> => ... # password (optional)
|
||||
<a href="#port">port</a> => ... # number (optional), default: 6379
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#timeout">timeout</a> => ... # number (optional), default: 5
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="batch_count">
|
||||
batch_count
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>How many events to return from redis using EVAL</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="data_type">
|
||||
data_type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "list", "channel", "pattern_channel" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Either list or channel. If redis_type is list, then we will BLPOP the
|
||||
key. If redis_type is channel, then we will SUBSCRIBE to the key.
|
||||
If redis_type is pattern_channel, then we will PSUBSCRIBE to the key.
|
||||
TODO: change required to true</p>
|
||||
|
||||
<h4>
|
||||
<a name="db">
|
||||
db
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 0 </li>
|
||||
</ul>
|
||||
|
||||
<p>The redis database number.</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "127.0.0.1" </li>
|
||||
</ul>
|
||||
|
||||
<p>The hostname of your redis server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="key">
|
||||
key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of a redis list or channel.
|
||||
TODO: change required to true</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="name">
|
||||
name
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "default" </li>
|
||||
</ul>
|
||||
|
||||
<p>Name is used for logging in case there are multiple instances.
|
||||
This feature has no real function and will be removed in future versions.</p>
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Password to authenticate with. There is no authentication by default.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 6379 </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to connect on.</p>
|
||||
|
||||
<h4>
|
||||
<a name="queue">
|
||||
queue
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the redis queue (we'll use BLPOP against this).
|
||||
TODO: remove soon.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to the number of threads you want this input to spawn.
|
||||
This is the same as declaring the input multiple times</p>
|
||||
|
||||
<h4>
|
||||
<a name="timeout">
|
||||
timeout
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5 </li>
|
||||
</ul>
|
||||
|
||||
<p>Initial connection timeout in seconds.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/redis.rb">lib/logstash/inputs/redis.rb</a>
|
||||
|
|
@ -1,214 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/relp
|
||||
layout: content_right
|
||||
---
|
||||
<h2>relp</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read RELP events over a TCP socket.</p>
|
||||
|
||||
<p>For more information about RELP, see
|
||||
<a href="http://www.rsyslog.com/doc/imrelp.html">http://www.rsyslog.com/doc/imrelp.html</a></p>
|
||||
|
||||
<p>This protocol implements application-level acknowledgements to help protect
|
||||
against message loss.</p>
|
||||
|
||||
<p>Message acks only function as far as messages being put into the queue for
|
||||
filters; anything lost after that point will not be retransmitted</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
relp {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#port">port</a> => ... # number (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>The address to listen on.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to listen on.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/relp.rb">lib/logstash/inputs/relp.rb</a>
|
||||
|
|
@ -1,322 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/s3
|
||||
layout: content_right
|
||||
---
|
||||
<h2>s3</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Stream events from files from a S3 bucket.</p>
|
||||
|
||||
<p>Each line from each file generates an event.
|
||||
Files ending in '.gz' are handled as gzip'ed files.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
s3 {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#backup_to_bucket">backup_to_bucket</a> => ... # string (optional), default: nil
|
||||
<a href="#backup_to_dir">backup_to_dir</a> => ... # string (optional), default: nil
|
||||
<a href="#bucket">bucket</a> => ... # string (required)
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#credentials">credentials</a> => ... # array (optional), default: nil
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#delete">delete</a> => ... # boolean (optional), default: false
|
||||
<a href="#interval">interval</a> => ... # number (optional), default: 60
|
||||
<a href="#prefix">prefix</a> => ... # string (optional), default: nil
|
||||
<a href="#region">region</a> => ... # string (optional), default: "us-east-1"
|
||||
<a href="#sincedb_path">sincedb_path</a> => ... # string (optional), default: nil
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="backup_to_bucket">
|
||||
backup_to_bucket
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>Name of a S3 bucket to backup processed files to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="backup_to_dir">
|
||||
backup_to_dir
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>Path of a local directory to backup processed files to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="bucket">
|
||||
bucket (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the S3 bucket.</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="credentials">
|
||||
credentials
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>TODO(sissel): refactor to use 'line' codec (requires removing both gzip
|
||||
support and readline usage). Support gzip through a gzip codec! ;)
|
||||
The credentials of the AWS account used to access the bucket.
|
||||
Credentials can be specified:
|
||||
- As an ["id","secret"] array
|
||||
- As a path to a file containing AWS<em>ACCESS</em>KEY<em>ID=... and AWS</em>SECRET<em>ACCESS</em>KEY=...
|
||||
- In the environment (variables AWS<em>ACCESS</em>KEY<em>ID and AWS</em>SECRET<em>ACCESS</em>KEY)</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="delete">
|
||||
delete
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Whether to delete processed files from the original bucket.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="interval">
|
||||
interval
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 60 </li>
|
||||
</ul>
|
||||
|
||||
<p>Interval to wait between to check the file list again after a run is finished.
|
||||
Value is in seconds.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="prefix">
|
||||
prefix
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>If specified, the prefix the filenames in the bucket must match (not a regexp)</p>
|
||||
|
||||
<h4>
|
||||
<a name="region">
|
||||
region
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "us-east-1" </li>
|
||||
</ul>
|
||||
|
||||
<p>The AWS region for your bucket.</p>
|
||||
|
||||
<h4>
|
||||
<a name="sincedb_path">
|
||||
sincedb_path
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>Where to write the since database (keeps track of the date
|
||||
the last handled file was added to S3). The default will write
|
||||
sincedb files to some path matching "$HOME/.sincedb*"</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/s3.rb">lib/logstash/inputs/s3.rb</a>
|
||||
|
|
@ -1,242 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/snmptrap
|
||||
layout: content_right
|
||||
---
|
||||
<h2>snmptrap</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read snmp trap messages as events</p>
|
||||
|
||||
<p>Resulting @message looks like :
|
||||
#<SNMP::SNMPv1<em>Trap:0x6f1a7a4 @varbind</em>list=[#<SNMP::VarBind:0x2d7bcd8f @value="teststring",
|
||||
@name=[1.11.12.13.14.15]>], @timestamp=#<SNMP::TimeTicks:0x1af47e9d @value=55>, @generic<em>trap=6,
|
||||
@enterprise=[1.2.3.4.5.6], @source</em>ip="127.0.0.1", @agent<em>addr=#<SNMP::IpAddress:0x29a4833e @value="\xC0\xC1\xC2\xC3">,
|
||||
@specific</em>trap=99></p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
snmptrap {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#community">community</a> => ... # string (optional), default: "public"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 1062
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#yamlmibdir">yamlmibdir</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="community">
|
||||
community
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "public" </li>
|
||||
</ul>
|
||||
|
||||
<p>SNMP Community String to listen for.</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>The address to listen on</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1062 </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to listen on. Remember that ports less than 1024 (privileged
|
||||
ports) may require root to use. hence the default of 1062.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="yamlmibdir">
|
||||
yamlmibdir
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>directory of YAML MIB maps (same format ruby-snmp uses)</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/snmptrap.rb">lib/logstash/inputs/snmptrap.rb</a>
|
||||
|
|
@ -1,274 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/sqlite
|
||||
layout: content_right
|
||||
---
|
||||
<h2>sqlite</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read rows from an sqlite database.</p>
|
||||
|
||||
<p>This is most useful in cases where you are logging directly to a table.
|
||||
Any tables being watched must have an 'id' column that is monotonically
|
||||
increasing.</p>
|
||||
|
||||
<p>All tables are read by default except:
|
||||
* ones matching 'sqlite<em>%' - these are internal/adminstrative tables for sqlite
|
||||
* 'since</em>table' - this is used by this plugin to track state.</p>
|
||||
|
||||
<h2>Example</h2>
|
||||
|
||||
<pre><code>% sqlite /tmp/example.db
|
||||
sqlite> CREATE TABLE weblogs (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
ip STRING,
|
||||
request STRING,
|
||||
response INTEGER);
|
||||
sqlite> INSERT INTO weblogs (ip, request, response)
|
||||
VALUES ("1.2.3.4", "/index.html", 200);
|
||||
</code></pre>
|
||||
|
||||
<p>Then with this logstash config:</p>
|
||||
|
||||
<pre><code>input {
|
||||
sqlite {
|
||||
path => "/tmp/example.db"
|
||||
type => weblogs
|
||||
}
|
||||
}
|
||||
output {
|
||||
stdout {
|
||||
debug => true
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>Sample output:</p>
|
||||
|
||||
<pre><code>{
|
||||
"@source" => "sqlite://sadness/tmp/x.db",
|
||||
"@tags" => [],
|
||||
"@fields" => {
|
||||
"ip" => "1.2.3.4",
|
||||
"request" => "/index.html",
|
||||
"response" => 200
|
||||
},
|
||||
"@timestamp" => "2013-05-29T06:16:30.850Z",
|
||||
"@source_host" => "sadness",
|
||||
"@source_path" => "/tmp/x.db",
|
||||
"@message" => "",
|
||||
"@type" => "foo"
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
sqlite {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#batch">batch</a> => ... # number (optional), default: 5
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#exclude_tables">exclude_tables</a> => ... # array (optional), default: []
|
||||
<a href="#path">path</a> => ... # string (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="batch">
|
||||
batch
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5 </li>
|
||||
</ul>
|
||||
|
||||
<p>How many rows to fetch at a time from each SELECT call.</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tables">
|
||||
exclude_tables
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Any tables to exclude by name.
|
||||
By default all tables are followed.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="path">
|
||||
path (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The path to the sqlite database file.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/sqlite.rb">lib/logstash/inputs/sqlite.rb</a>
|
||||
|
|
@ -1,396 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/sqs
|
||||
layout: content_right
|
||||
---
|
||||
<h2>sqs</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Pull events from an Amazon Web Services Simple Queue Service (SQS) queue.</p>
|
||||
|
||||
<p>SQS is a simple, scalable queue system that is part of the
|
||||
Amazon Web Services suite of tools.</p>
|
||||
|
||||
<p>Although SQS is similar to other queuing systems like AMQP, it
|
||||
uses a custom API and requires that you have an AWS account.
|
||||
See http://aws.amazon.com/sqs/ for more details on how SQS works,
|
||||
what the pricing schedule looks like and how to setup a queue.</p>
|
||||
|
||||
<p>To use this plugin, you <em>must</em>:</p>
|
||||
|
||||
<ul>
|
||||
<li>Have an AWS account</li>
|
||||
<li>Setup an SQS queue</li>
|
||||
<li>Create an identify that has access to consume messages from the queue.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<p>The "consumer" identity must have the following permissions on the queue:</p>
|
||||
|
||||
<ul>
|
||||
<li>sqs:ChangeMessageVisibility</li>
|
||||
<li>sqs:ChangeMessageVisibilityBatch</li>
|
||||
<li>sqs:DeleteMessage</li>
|
||||
<li>sqs:DeleteMessageBatch</li>
|
||||
<li>sqs:GetQueueAttributes</li>
|
||||
<li>sqs:GetQueueUrl</li>
|
||||
<li>sqs:ListQueues</li>
|
||||
<li>sqs:ReceiveMessage</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<p>Typically, you should setup an IAM policy, create a user and apply the IAM policy to the user.
|
||||
A sample policy is as follows:</p>
|
||||
|
||||
<pre><code>{
|
||||
"Statement": [
|
||||
{
|
||||
"Action": [
|
||||
"sqs:ChangeMessageVisibility",
|
||||
"sqs:ChangeMessageVisibilityBatch",
|
||||
"sqs:GetQueueAttributes",
|
||||
"sqs:GetQueueUrl",
|
||||
"sqs:ListQueues",
|
||||
"sqs:SendMessage",
|
||||
"sqs:SendMessageBatch"
|
||||
],
|
||||
"Effect": "Allow",
|
||||
"Resource": [
|
||||
"arn:aws:sqs:us-east-1:123456789012:Logstash"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<p>See http://aws.amazon.com/iam/ for more details on setting up AWS identities.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
sqs {
|
||||
<a href="#access_key_id">access_key_id</a> => ... # string (optional)
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#aws_credentials_file">aws_credentials_file</a> => ... # string (optional)
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#id_field">id_field</a> => ... # string (optional)
|
||||
<a href="#md5_field">md5_field</a> => ... # string (optional)
|
||||
<a href="#queue">queue</a> => ... # string (required)
|
||||
<a href="#region">region</a> => ... # string, one of ["us-east-1", "us-west-1", "us-west-2", "eu-west-1", "ap-southeast-1", "ap-southeast-2", "ap-northeast-1", "sa-east-1", "us-gov-west-1"] (optional), default: "us-east-1"
|
||||
<a href="#secret_access_key">secret_access_key</a> => ... # string (optional)
|
||||
<a href="#sent_timestamp_field">sent_timestamp_field</a> => ... # string (optional)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#use_ssl">use_ssl</a> => ... # boolean (optional), default: true
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="access_key_id">
|
||||
access_key_id
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>This plugin uses the AWS SDK and supports several ways to get credentials, which will be tried in this order... <br/>
|
||||
1. Static configuration, using <code>access_key_id</code> and <code>secret_access_key</code> params in logstash plugin config <br/>
|
||||
2. External credentials file specified by <code>aws_credentials_file</code> <br/>
|
||||
3. Environment variables <code>AWS_ACCESS_KEY_ID</code> and <code>AWS_SECRET_ACCESS_KEY</code> <br/>
|
||||
4. Environment variables <code>AMAZON_ACCESS_KEY_ID</code> and <code>AMAZON_SECRET_ACCESS_KEY</code> <br/>
|
||||
5. IAM Instance Profile (available when running inside EC2)</p>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="aws_credentials_file">
|
||||
aws_credentials_file
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Path to YAML file containing a hash of AWS credentials. <br/>
|
||||
This file will only be loaded if <code>access_key_id</code> and
|
||||
<code>secret_access_key</code> aren't set. The contents of the
|
||||
file should look like this:</p>
|
||||
|
||||
<pre><code>:access_key_id: "12345"
|
||||
:secret_access_key: "54321"
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="id_field">
|
||||
id_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Name of the event field in which to store the SQS message ID</p>
|
||||
|
||||
<h4>
|
||||
<a name="md5_field">
|
||||
md5_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Name of the event field in which to store the SQS message MD5 checksum</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="queue">
|
||||
queue (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Name of the SQS Queue name to pull messages from. Note that this is just the name of the queue, not the URL or ARN.</p>
|
||||
|
||||
<h4>
|
||||
<a name="region">
|
||||
region
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "us-east-1", "us-west-1", "us-west-2", "eu-west-1", "ap-southeast-1", "ap-southeast-2", "ap-northeast-1", "sa-east-1", "us-gov-west-1" </li>
|
||||
<li> Default value is "us-east-1" </li>
|
||||
</ul>
|
||||
|
||||
<p>The AWS Region</p>
|
||||
|
||||
<h4>
|
||||
<a name="secret_access_key">
|
||||
secret_access_key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The AWS Secret Access Key</p>
|
||||
|
||||
<h4>
|
||||
<a name="sent_timestamp_field">
|
||||
sent_timestamp_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Name of the event field in which to store the SQS message Sent Timestamp</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to the number of threads you want this input to spawn.
|
||||
This is the same as declaring the input multiple times</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="use_ssl">
|
||||
use_ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Should we require (true) or disable (false) using SSL for communicating with the AWS API <br/>
|
||||
The AWS SDK for Ruby defaults to SSL so we preserve that</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/sqs.rb">lib/logstash/inputs/sqs.rb</a>
|
||||
|
|
@ -1,178 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/stdin
|
||||
layout: content_right
|
||||
---
|
||||
<h2>stdin</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">3</a></h3>
|
||||
|
||||
<p>Read events from standard input.</p>
|
||||
|
||||
<p>By default, each event is assumed to be one line. If you
|
||||
want to join lines, you'll want to use the multiline filter.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
stdin {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/stdin.rb">lib/logstash/inputs/stdin.rb</a>
|
||||
|
|
@ -1,267 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/stomp
|
||||
layout: content_right
|
||||
---
|
||||
<h2>stomp</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
stomp {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#destination">destination</a> => ... # string (required)
|
||||
<a href="#host">host</a> => ... # string (required), default: "localhost"
|
||||
<a href="#password">password</a> => ... # password (optional), default: ""
|
||||
<a href="#port">port</a> => ... # number (optional), default: 61613
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#user">user</a> => ... # string (optional), default: ""
|
||||
<a href="#vhost">vhost</a> => ... # string (optional), default: nil
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Enable debugging output?</p>
|
||||
|
||||
<h4>
|
||||
<a name="destination">
|
||||
destination (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The destination to read events from.</p>
|
||||
|
||||
<p>Example: "/topic/logstash"</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "localhost" </li>
|
||||
</ul>
|
||||
|
||||
<p>The address of the STOMP server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>The password to authenticate with.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 61613 </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to connet to on your STOMP server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>The username to authenticate with.</p>
|
||||
|
||||
<h4>
|
||||
<a name="vhost">
|
||||
vhost
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>The vhost to use</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/stomp.rb">lib/logstash/inputs/stomp.rb</a>
|
||||
|
|
@ -1,265 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/syslog
|
||||
layout: content_right
|
||||
---
|
||||
<h2>syslog</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read syslog messages as events over the network.</p>
|
||||
|
||||
<p>This input is a good choice if you already use syslog today.
|
||||
It is also a good choice if you want to receive logs from
|
||||
appliances and network devices where you cannot run your own
|
||||
log collector.</p>
|
||||
|
||||
<p>Of course, 'syslog' is a very muddy term. This input only supports RFC3164
|
||||
syslog with some small modifications. The date format is allowed to be
|
||||
RFC3164 style or ISO8601. Otherwise the rest of the RFC3164 must be obeyed.
|
||||
If you do not use RFC3164, do not use this input.</p>
|
||||
|
||||
<p>Note: this input will start listeners on both TCP and UDP</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
syslog {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#facility_labels">facility_labels</a> => ... # array (optional), default: ["kernel", "user-level", "mail", "system", "security/authorization", "syslogd", "line printer", "network news", "UUCP", "clock", "security/authorization", "FTP", "NTP", "log audit", "log alert", "clock", "local0", "local1", "local2", "local3", "local4", "local5", "local6", "local7"]
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 514
|
||||
<a href="#severity_labels">severity_labels</a> => ... # array (optional), default: ["Emergency", "Alert", "Critical", "Error", "Warning", "Notice", "Informational", "Debug"]
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#use_labels">use_labels</a> => ... # boolean (optional), default: true
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="facility_labels">
|
||||
facility_labels
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["kernel", "user-level", "mail", "system", "security/authorization", "syslogd", "line printer", "network news", "UUCP", "clock", "security/authorization", "FTP", "NTP", "log audit", "log alert", "clock", "local0", "local1", "local2", "local3", "local4", "local5", "local6", "local7"] </li>
|
||||
</ul>
|
||||
|
||||
<p>Labels for facility levels
|
||||
This comes from RFC3164.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>The address to listen on</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 514 </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to listen on. Remember that ports less than 1024 (privileged
|
||||
ports) may require root to use.</p>
|
||||
|
||||
<h4>
|
||||
<a name="severity_labels">
|
||||
severity_labels
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["Emergency", "Alert", "Critical", "Error", "Warning", "Notice", "Informational", "Debug"] </li>
|
||||
</ul>
|
||||
|
||||
<p>Labels for severity levels
|
||||
This comes from RFC3164.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="use_labels">
|
||||
use_labels
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Use label parsing for severity and facility levels</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/syslog.rb">lib/logstash/inputs/syslog.rb</a>
|
||||
|
|
@ -1,338 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/tcp
|
||||
layout: content_right
|
||||
---
|
||||
<h2>tcp</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Read events over a TCP socket.</p>
|
||||
|
||||
<p>Like stdin and file inputs, each event is assumed to be one line of text.</p>
|
||||
|
||||
<p>Can either accept connections from clients or connect to a server,
|
||||
depending on <code>mode</code>.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
tcp {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#data_timeout">data_timeout</a> => ... # number (optional), default: -1
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#mode">mode</a> => ... # string, one of ["server", "client"] (optional), default: "server"
|
||||
<a href="#port">port</a> => ... # number (required)
|
||||
<a href="#ssl_cacert">ssl_cacert</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#ssl_cert">ssl_cert</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#ssl_enable">ssl_enable</a> => ... # boolean (optional), default: false
|
||||
<a href="#ssl_key">ssl_key</a> => ... # a valid filesystem path (optional)
|
||||
<a href="#ssl_key_passphrase">ssl_key_passphrase</a> => ... # password (optional), default: nil
|
||||
<a href="#ssl_verify">ssl_verify</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="data_timeout">
|
||||
data_timeout
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is -1 </li>
|
||||
</ul>
|
||||
|
||||
<p>The 'read' timeout in seconds. If a particular tcp connection is idle for
|
||||
more than this timeout period, we will assume it is dead and close it.</p>
|
||||
|
||||
<p>If you never want to timeout, use -1.</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>When mode is <code>server</code>, the address to listen on.
|
||||
When mode is <code>client</code>, the address to connect to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "server", "client" </li>
|
||||
<li> Default value is "server" </li>
|
||||
</ul>
|
||||
|
||||
<p>Mode to operate in. <code>server</code> listens for client connections,
|
||||
<code>client</code> connects to a server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>When mode is <code>server</code>, the port to listen on.
|
||||
When mode is <code>client</code>, the port to connect to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_cacert">
|
||||
ssl_cacert
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>ssl CA certificate, chainfile or CA path
|
||||
The system CA path is automatically included</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_cert">
|
||||
ssl_cert
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>ssl certificate</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_enable">
|
||||
ssl_enable
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Enable ssl (must be set for other <code>ssl_</code> options to take effect)</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_key">
|
||||
ssl_key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#path">path</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>ssl key</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_key_passphrase">
|
||||
ssl_key_passphrase
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> Default value is nil </li>
|
||||
</ul>
|
||||
|
||||
<p>ssl key passphrase</p>
|
||||
|
||||
<h4>
|
||||
<a name="ssl_verify">
|
||||
ssl_verify
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Verify the identity of the other end of the ssl connection against the CA
|
||||
For input, sets the <code>@field.sslsubject</code> to that of the client certificate</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/tcp.rb">lib/logstash/inputs/tcp.rb</a>
|
||||
|
|
@ -1,273 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/twitter
|
||||
layout: content_right
|
||||
---
|
||||
<h2>twitter</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read events from the twitter streaming api.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
twitter {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#consumer_key">consumer_key</a> => ... # string (required)
|
||||
<a href="#consumer_secret">consumer_secret</a> => ... # password (required)
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#keywords">keywords</a> => ... # array (required)
|
||||
<a href="#oauth_token">oauth_token</a> => ... # string (required)
|
||||
<a href="#oauth_token_secret">oauth_token_secret</a> => ... # password (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="consumer_key">
|
||||
consumer_key (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Your twitter app's consumer key</p>
|
||||
|
||||
<p>Don't know what this is? You need to create an "application"
|
||||
on twitter, see this url: <a href="https://dev.twitter.com/apps/new">https://dev.twitter.com/apps/new</a></p>
|
||||
|
||||
<h4>
|
||||
<a name="consumer_secret">
|
||||
consumer_secret (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Your twitter app's consumer secret</p>
|
||||
|
||||
<p>If you don't have one of these, you can create one by
|
||||
registering a new application with twitter:
|
||||
<a href="https://dev.twitter.com/apps/new">https://dev.twitter.com/apps/new</a></p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="keywords">
|
||||
keywords (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Any keywords to track in the twitter stream</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="oauth_token">
|
||||
oauth_token (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Your oauth token.</p>
|
||||
|
||||
<p>To get this, login to twitter with whatever account you want,
|
||||
then visit <a href="https://dev.twitter.com/apps">https://dev.twitter.com/apps</a></p>
|
||||
|
||||
<p>Click on your app (used with the consumer<em>key and consumer</em>secret settings)
|
||||
Then at the bottom of the page, click 'Create my access token' which
|
||||
will create an oauth token and secret bound to your account and that
|
||||
application.</p>
|
||||
|
||||
<h4>
|
||||
<a name="oauth_token_secret">
|
||||
oauth_token_secret (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Your oauth token secret.</p>
|
||||
|
||||
<p>To get this, login to twitter with whatever account you want,
|
||||
then visit <a href="https://dev.twitter.com/apps">https://dev.twitter.com/apps</a></p>
|
||||
|
||||
<p>Click on your app (used with the consumer<em>key and consumer</em>secret settings)
|
||||
Then at the bottom of the page, click 'Create my access token' which
|
||||
will create an oauth token and secret bound to your account and that
|
||||
application.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/twitter.rb">lib/logstash/inputs/twitter.rb</a>
|
||||
|
|
@ -1,221 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/udp
|
||||
layout: content_right
|
||||
---
|
||||
<h2>udp</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Read messages as events over the network via udp.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
udp {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#buffer_size">buffer_size</a> => ... # number (optional), default: 8192
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional), default: "0.0.0.0"
|
||||
<a href="#port">port</a> => ... # number (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="buffer_size">
|
||||
buffer_size
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 8192 </li>
|
||||
</ul>
|
||||
|
||||
<p>Buffer size</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>The address to listen on</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The port to listen on. Remember that ports less than 1024 (privileged
|
||||
ports) may require root or elevated privileges to use.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/udp.rb">lib/logstash/inputs/udp.rb</a>
|
||||
|
|
@ -1,245 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/unix
|
||||
layout: content_right
|
||||
---
|
||||
<h2>unix</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Read events over a UNIX socket.</p>
|
||||
|
||||
<p>Like stdin and file inputs, each event is assumed to be one line of text.</p>
|
||||
|
||||
<p>Can either accept connections from clients or connect to a server,
|
||||
depending on <code>mode</code>.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
unix {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#data_timeout">data_timeout</a> => ... # number (optional), default: -1
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#force_unlink">force_unlink</a> => ... # boolean (optional), default: false
|
||||
<a href="#mode">mode</a> => ... # string, one of ["server", "client"] (optional), default: "server"
|
||||
<a href="#path">path</a> => ... # string (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="data_timeout">
|
||||
data_timeout
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is -1 </li>
|
||||
</ul>
|
||||
|
||||
<p>The 'read' timeout in seconds. If a particular connection is idle for
|
||||
more than this timeout period, we will assume it is dead and close it.</p>
|
||||
|
||||
<p>If you never want to timeout, use -1.</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="force_unlink">
|
||||
force_unlink
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Remove socket file in case of EADDRINUSE failure</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "server", "client" </li>
|
||||
<li> Default value is "server" </li>
|
||||
</ul>
|
||||
|
||||
<p>Mode to operate in. <code>server</code> listens for client connections,
|
||||
<code>client</code> connects to a server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="path">
|
||||
path (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>When mode is <code>server</code>, the path to listen on.
|
||||
When mode is <code>client</code>, the path to connect to.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/unix.rb">lib/logstash/inputs/unix.rb</a>
|
||||
|
|
@ -1,191 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/varnishlog
|
||||
layout: content_right
|
||||
---
|
||||
<h2>varnishlog</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read from varnish cache's shared memory log</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
varnishlog {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to the number of threads you want this input to spawn.
|
||||
This is the same as declaring the input multiple times</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/varnishlog.rb">lib/logstash/inputs/varnishlog.rb</a>
|
||||
|
|
@ -1,212 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/websocket
|
||||
layout: content_right
|
||||
---
|
||||
<h2>websocket</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Read events over the websocket protocol.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
websocket {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#mode">mode</a> => ... # string, one of ["server", "client"] (optional), default: "client"
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#url">url</a> => ... # string (optional), default: "0.0.0.0"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "server", "client" </li>
|
||||
<li> Default value is "client" </li>
|
||||
</ul>
|
||||
|
||||
<p>Operate as a client or a server.</p>
|
||||
|
||||
<p>Client mode causes this plugin to connect as a websocket client
|
||||
to the URL given. It expects to receive events as websocket messages.</p>
|
||||
|
||||
<p>(NOT IMPLEMENTED YET) Server mode causes this plugin to listen on
|
||||
the given URL for websocket clients. It expects to receive events
|
||||
as websocket messages from these clients.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="url">
|
||||
url
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "0.0.0.0" </li>
|
||||
</ul>
|
||||
|
||||
<p>The url to connect to or serve from</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/websocket.rb">lib/logstash/inputs/websocket.rb</a>
|
||||
|
|
@ -1,221 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/wmi
|
||||
layout: content_right
|
||||
---
|
||||
<h2>wmi</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>Collect data from WMI query</p>
|
||||
|
||||
<p>This is useful for collecting performance metrics and other data
|
||||
which is accessible via WMI on a Windows host</p>
|
||||
|
||||
<p>Example:</p>
|
||||
|
||||
<pre><code>input {
|
||||
wmi {
|
||||
query => "select * from Win32_Process"
|
||||
interval => 10
|
||||
}
|
||||
wmi {
|
||||
query => "select PercentProcessorTime from Win32_PerfFormattedData_PerfOS_Processor where name = '_Total'"
|
||||
}
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
wmi {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#interval">interval</a> => ... # number (optional), default: 10
|
||||
<a href="#query">query</a> => ... # string (required)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="interval">
|
||||
interval
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 10 </li>
|
||||
</ul>
|
||||
|
||||
<p>Polling interval</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="query">
|
||||
query (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>WMI query</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/wmi.rb">lib/logstash/inputs/wmi.rb</a>
|
||||
|
|
@ -1,242 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/xmpp
|
||||
layout: content_right
|
||||
---
|
||||
<h2>xmpp</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>This input allows you to receive events over XMPP/Jabber.</p>
|
||||
|
||||
<p>This plugin can be used for accepting events from humans or applications
|
||||
XMPP, or you can use it for PubSub or general message passing for logstash to
|
||||
logstash.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
xmpp {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#host">host</a> => ... # string (optional)
|
||||
<a href="#password">password</a> => ... # password (required)
|
||||
<a href="#rooms">rooms</a> => ... # array (optional)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#user">user</a> => ... # string (required)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set to true to enable greater debugging in XMPP. Useful for debugging
|
||||
network/authentication erros.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The xmpp server to connect to. This is optional. If you omit this setting,
|
||||
the host on the user/identity is used. (foo.com for user@foo.com)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The xmpp password for the user/identity.</p>
|
||||
|
||||
<h4>
|
||||
<a name="rooms">
|
||||
rooms
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>if muc/multi-user-chat required, give the name of the room that
|
||||
you want to join: room@conference.domain/nick</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The user or resource ID, like foo@example.com.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/xmpp.rb">lib/logstash/inputs/xmpp.rb</a>
|
||||
|
|
@ -1,452 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/zenoss
|
||||
layout: content_right
|
||||
---
|
||||
<h2>zenoss</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
zenoss {
|
||||
<a href="#ack">ack</a> => ... # boolean (optional), default: true
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#arguments">arguments</a> => ... # array (optional), default: {}
|
||||
<a href="#auto_delete">auto_delete</a> => ... # boolean (optional), default: true
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#durable">durable</a> => ... # boolean (optional), default: false
|
||||
<a href="#exchange">exchange</a> => ... # string (optional), default: "zenoss.zenevents"
|
||||
<a href="#exclusive">exclusive</a> => ... # boolean (optional), default: true
|
||||
<a href="#host">host</a> => ... # string (optional), default: "localhost"
|
||||
<a href="#key">key</a> => ... # string (optional), default: "zenoss.zenevent.#"
|
||||
<a href="#passive">passive</a> => ... # boolean (optional), default: false
|
||||
<a href="#password">password</a> => ... # password (optional), default: "zenoss"
|
||||
<a href="#port">port</a> => ... # number (optional), default: 5672
|
||||
<a href="#prefetch_count">prefetch_count</a> => ... # number (optional), default: 256
|
||||
<a href="#queue">queue</a> => ... # string (optional), default: ""
|
||||
<a href="#ssl">ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#threads">threads</a> => ... # number (optional), default: 1
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
<a href="#user">user</a> => ... # string (optional), default: "zenoss"
|
||||
<a href="#verify_ssl">verify_ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#vhost">vhost</a> => ... # string (optional), default: "/zenoss"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="ack">
|
||||
ack
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="arguments">
|
||||
arguments
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="auto_delete">
|
||||
auto_delete
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="durable">
|
||||
durable
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="exchange">
|
||||
exchange
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "zenoss.zenevents" </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the exchange to bind the queue. This is analogous to the 'rabbitmq
|
||||
output' <a href="../outputs/rabbitmq">config 'name'</a></p>
|
||||
|
||||
<h4>
|
||||
<a name="exclusive">
|
||||
exclusive
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "localhost" </li>
|
||||
</ul>
|
||||
|
||||
<p>Your rabbitmq server address</p>
|
||||
|
||||
<h4>
|
||||
<a name="key">
|
||||
key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "zenoss.zenevent.#" </li>
|
||||
</ul>
|
||||
|
||||
<p>The routing key to use. This is only valid for direct or fanout exchanges</p>
|
||||
|
||||
<ul>
|
||||
<li>Routing keys are ignored on topic exchanges.</li>
|
||||
<li>Wildcards are not valid on direct exchanges.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="passive">
|
||||
passive
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> Default value is "zenoss" </li>
|
||||
</ul>
|
||||
|
||||
<p>Your rabbitmq password</p>
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5672 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="prefetch_count">
|
||||
prefetch_count
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 256 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="queue">
|
||||
queue
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl">
|
||||
ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="threads">
|
||||
threads
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 1 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "zenoss" </li>
|
||||
</ul>
|
||||
|
||||
<p>Your rabbitmq username</p>
|
||||
|
||||
<h4>
|
||||
<a name="verify_ssl">
|
||||
verify_ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="vhost">
|
||||
vhost
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "/zenoss" </li>
|
||||
</ul>
|
||||
|
||||
<p>The vhost to use. If you don't know what this is, leave the default.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/zenoss.rb">lib/logstash/inputs/zenoss.rb</a>
|
||||
|
|
@ -1,305 +0,0 @@
|
|||
---
|
||||
title: logstash docs for inputs/zeromq
|
||||
layout: content_right
|
||||
---
|
||||
<h2>zeromq</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
<p>Read events over a 0MQ SUB socket.</p>
|
||||
|
||||
<p>You need to have the 0mq 2.1.x library installed to be able to use
|
||||
this input plugin.</p>
|
||||
|
||||
<p>The default settings will create a subscriber binding to tcp://127.0.0.1:2120
|
||||
waiting for connecting publishers.</p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>input {
|
||||
zeromq {
|
||||
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
||||
<a href="#address">address</a> => ... # array (optional), default: ["tcp://*:2120"]
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#mode">mode</a> => ... # string, one of ["server", "client"] (optional), default: "server"
|
||||
<a href="#sender">sender</a> => ... # string (optional)
|
||||
<a href="#sockopt">sockopt</a> => ... # hash (optional)
|
||||
<a href="#tags">tags</a> => ... # array (optional)
|
||||
<a href="#topic">topic</a> => ... # array (optional)
|
||||
<a href="#topology">topology</a> => ... # string, one of ["pushpull", "pubsub", "pair"] (required)
|
||||
<a href="#type">type</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="add_field">
|
||||
add_field
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a field to an event</p>
|
||||
|
||||
<h4>
|
||||
<a name="address">
|
||||
address
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is ["tcp://*:2120"] </li>
|
||||
</ul>
|
||||
|
||||
<p>0mq socket address to connect or bind
|
||||
Please note that <code>inproc://</code> will not work with logstash
|
||||
as each we use a context per thread.
|
||||
By default, inputs bind/listen
|
||||
and outputs connect</p>
|
||||
|
||||
<h4>
|
||||
<a name="charset">
|
||||
charset
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The character encoding used in this input. Examples include "UTF-8"
|
||||
and "cp1252"</p>
|
||||
|
||||
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
||||
or in another character set other than UTF-8.</p>
|
||||
|
||||
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for input data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Set this to true to enable debugging on an input.</p>
|
||||
|
||||
<h4>
|
||||
<a name="format">
|
||||
format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The format of input data (plain, json, json_event)</p>
|
||||
|
||||
<h4>
|
||||
<a name="message_format">
|
||||
message_format
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>If format is "json", an event sprintf string to build what
|
||||
the display @message should be given (defaults to the raw JSON).
|
||||
sprintf format strings look like %{fieldname}</p>
|
||||
|
||||
<p>If format is "json_event", ALL fields except for @type
|
||||
are expected to be present. Not receiving all fields
|
||||
will cause unexpected results.</p>
|
||||
|
||||
<h4>
|
||||
<a name="mode">
|
||||
mode
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "server", "client" </li>
|
||||
<li> Default value is "server" </li>
|
||||
</ul>
|
||||
|
||||
<p>mode
|
||||
server mode binds/listens
|
||||
client mode connects</p>
|
||||
|
||||
<h4>
|
||||
<a name="sender">
|
||||
sender
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>sender
|
||||
overrides the sender to
|
||||
set the source of the event
|
||||
default is "zmq+topology://type/"</p>
|
||||
|
||||
<h4>
|
||||
<a name="sockopt">
|
||||
sockopt
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>0mq socket options
|
||||
This exposes zmq_setsockopt
|
||||
for advanced tuning
|
||||
see http://api.zeromq.org/2-1:zmq-setsockopt for details</p>
|
||||
|
||||
<p>This is where you would set values like:
|
||||
ZMQ::HWM - high water mark
|
||||
ZMQ::IDENTITY - named queues
|
||||
ZMQ::SWAP_SIZE - space for disk overflow</p>
|
||||
|
||||
<p>example: sockopt => ["ZMQ::HWM", 50, "ZMQ::IDENTITY", "my<em>named</em>queue"]</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add any number of arbitrary tags to your event.</p>
|
||||
|
||||
<p>This can help with processing later.</p>
|
||||
|
||||
<h4>
|
||||
<a name="topic">
|
||||
topic
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>0mq topic
|
||||
This is used for the 'pubsub' topology only
|
||||
On inputs, this allows you to filter messages by topic
|
||||
On outputs, this allows you to tag a message for routing
|
||||
NOTE: ZeroMQ does subscriber side filtering.
|
||||
NOTE: All topics have an implicit wildcard at the end
|
||||
You can specify multiple topics here</p>
|
||||
|
||||
<h4>
|
||||
<a name="topology">
|
||||
topology (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "pushpull", "pubsub", "pair" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>0mq topology
|
||||
The default logstash topologies work as follows:
|
||||
* pushpull - inputs are pull, outputs are push
|
||||
* pubsub - inputs are subscribers, outputs are publishers
|
||||
* pair - inputs are clients, inputs are servers</p>
|
||||
|
||||
<p>If the predefined topology flows don't work for you,
|
||||
you can change the 'mode' setting
|
||||
TODO (lusis) add req/rep MAYBE
|
||||
TODO (lusis) add router/dealer</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Add a 'type' field to all events handled by this input.</p>
|
||||
|
||||
<p>Types are used mainly for filter activation.</p>
|
||||
|
||||
<p>If you create an input with type "foobar", then only filters
|
||||
which also have type "foobar" will act on them.</p>
|
||||
|
||||
<p>The type is also stored as part of the event itself, so you
|
||||
can also use the type to search for in the web interface.</p>
|
||||
|
||||
<p>If you try to set a type on an event that already has one (for
|
||||
example when you send an event from a shipper to an indexer) then
|
||||
a new input will not override the existing type. A type set at
|
||||
the shipper stays with that event for its life even
|
||||
when sent to another LogStash server.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/zeromq.rb">lib/logstash/inputs/zeromq.rb</a>
|
||||
|
|
@ -1,46 +0,0 @@
|
|||
---
|
||||
title: Learn - logstash
|
||||
layout: content_right
|
||||
---
|
||||
# What is logstash?
|
||||
|
||||
logstash is a tool for managing your logs.
|
||||
|
||||
It helps you take logs and other event data from your systems and move it into
|
||||
a central place. logstash is open source and completely free. You can find
|
||||
support on the mailing list and on IRC.
|
||||
|
||||
For an overview of logstash and why you would use it, you should watch the
|
||||
presentation I gave at CarolinaCon 2011:
|
||||
[video here](http://carolinacon.blip.tv/file/5105901/). This presentation covers
|
||||
logstash, how you can use it, some alternatives, logging best practices,
|
||||
parsing tools, etc. Video also below:
|
||||
|
||||
<!--
|
||||
<embed src="http://blip.tv/play/gvE9grjcdQI" type="application/x-shockwave-flash" width="480" height="296" allowscriptaccess="always" allowfullscreen="true"></embed>
|
||||
|
||||
The slides are available online here: [slides](http://goo.gl/68c62). The slides
|
||||
include speaker notes (click 'actions' then 'speaker notes').
|
||||
-->
|
||||
<iframe width="480" height="296" src="http://www.youtube.com/embed/RuUFnog29M4" frameborder="0" allowfullscreen="allowfullscreen"></iframe>
|
||||
|
||||
The slides are available online here: [slides](http://semicomplete.com/presentations/logstash-puppetconf-2012/).
|
||||
|
||||
## Getting Help
|
||||
|
||||
There's [documentation](.) here on this site. If that isn't sufficient, you can
|
||||
email the mailing list (logstash-users@googlegroups.com). Further, there is also
|
||||
an IRC channel - #logstash on irc.freenode.org.
|
||||
|
||||
If you find a bug or have a feature request, file them
|
||||
on <http://logstash.jira.com/>. (Honestly though, if you prefer email or irc
|
||||
for such things, that works for me, too.)
|
||||
|
||||
## Download It
|
||||
|
||||
[Download logstash-1.2.0.beta1](https://logstash.objects.dreamhost.com/release/logstash-1.2.0.beta1-flatjar.jar)
|
||||
|
||||
## What's next?
|
||||
|
||||
Try the [standalone logstash guide](tutorials/getting-started-simple) for a simple
|
||||
real-world example getting started using logstash.
|
||||
|
|
@ -1,81 +0,0 @@
|
|||
---
|
||||
title: the life of an event - logstash
|
||||
layout: content_right
|
||||
---
|
||||
# the life of an event
|
||||
|
||||
The logstash agent is an event pipeline.
|
||||
|
||||
The logstash agent is 3 parts: inputs -> filters -> outputs. Inputs generate
|
||||
events, filters modify them, outputs ship them elsewhere.
|
||||
|
||||
Internal to logstash, events are passed from each phase using internal queues.
|
||||
It is implemented with a 'SizedQueue' in Ruby. SizedQueue allows a bounded
|
||||
maximum of items in the queue such that any writes to the queue will block if
|
||||
the queue is full at maximum capacity.
|
||||
|
||||
Logstash sets each queue size to 20. This means only 20 events can be pending
|
||||
into the next phase - this helps reduce any data loss and in general avoids
|
||||
logstash trying to act as a data storage system. These internal queues are not
|
||||
for storing messages long-term.
|
||||
|
||||
Starting at outputs, here's what happens with a queue fills up.
|
||||
|
||||
If an output is failing, the output thread will wait until this output is
|
||||
healthy again and able to successfully send the message. Therefore, the output
|
||||
queue will stop being read from by this output and will eventually fill up with
|
||||
events and cause write blocks.
|
||||
|
||||
A full output queue means filters will block trying to write to the output
|
||||
queue. Because filters will be stuck, blocked writing to the output queue, they
|
||||
will stop reading from the filter queue which will eventually cause the filter
|
||||
queue (input -> filter) to fill up.
|
||||
|
||||
A full filter queue will cause inputs to block when writing to the filters.
|
||||
This will cause each input to block, causing each input to stop processing new
|
||||
data from wherever that input is getting new events.
|
||||
|
||||
In ideal circumstances, this will behave similarly to when the tcp window
|
||||
closes to 0, no new data is sent because the receiver hasn't finished
|
||||
processing the current queue of data.
|
||||
|
||||
## Thread Model
|
||||
|
||||
The thread model in logstash is currently:
|
||||
|
||||
input threads | filter threads | output threads
|
||||
|
||||
Filters are optional, so you will have this model if you have no filters defined:
|
||||
|
||||
input threads | output threads
|
||||
|
||||
Each input runs in a thread by itself. This allows busier inputs to not be
|
||||
blocked by slower ones, etc. It also allows for easier containment of scope
|
||||
because each input has a thread.
|
||||
|
||||
The filter thread model is a 'worker' model where each worker receives an event
|
||||
and applies all filters, in order, before emitting that to the output queue.
|
||||
This allows scalability across CPUs because many filters are CPU intensive
|
||||
(permitting that we have thread safety). Currently, logstash forces the number
|
||||
of filter worker threads to be 1, but this will be tunable in the future once
|
||||
we analyze the thread safety of each filter.
|
||||
|
||||
The output thread model one thread per output. Each output has its own queue
|
||||
receiving events. This is implemented in logstash with LogStash::MultiQueue.
|
||||
|
||||
## Consequences and Expectations
|
||||
|
||||
Small queue sizes mean that logstash simply blocks and stalls safely during
|
||||
times of load or other temporary pipeline problems. The alternative is
|
||||
unlimited queues which grow unbounded and eventually exceed memory causing a
|
||||
crash which loses all of those messages.
|
||||
|
||||
At a minum, logstash will have probably 3 threads (2 if you have no filters).
|
||||
One input, one filter worker, and one output thread each.
|
||||
|
||||
If you see logstash using multiple CPUs, this is likely why. If you want to
|
||||
know more about what each thread is doing, you should read this:
|
||||
<http://www.semicomplete.com/blog/geekery/debugging-java-performance.html>.
|
||||
|
||||
Threads in java have names, and you can use jstack and top to figure out who is
|
||||
using what resources. The URL above will help you learn how to do this.
|
||||
|
|
@ -1,60 +0,0 @@
|
|||
---
|
||||
title: Logging tools comparisons - logstash
|
||||
layout: content_right
|
||||
---
|
||||
# Logging tools comparison
|
||||
|
||||
The information below is provided as "best effort" and is not strictly intended
|
||||
as a complete source of truth. If the information below is unclear or incorrect, please
|
||||
email the logstash-users list (or send a pull request with the fix) :)
|
||||
|
||||
Where feasible, this document will also provide information on how you can use
|
||||
logstash with these other projects.
|
||||
|
||||
# logstash
|
||||
|
||||
Primary goal: Make log/event data and analytics accessible.
|
||||
|
||||
Overview: Where your logs come from, how you store them, or what you do with
|
||||
them is up to you. Logstash exists to help make such actions easier and faster.
|
||||
|
||||
It provides you a simple event pipeline for taking events and logs from any
|
||||
input, manipulating them with filters, and sending them to any output. Inputs
|
||||
can be files, network, message brokers, etc. Filters are date and string
|
||||
parsers, grep-like, etc. Outputs are data stores (elasticsearch, mongodb, etc),
|
||||
message systems (rabbitmq, stomp, etc), network (tcp, syslog), etc.
|
||||
|
||||
It also provides a web interface for doing search and analytics on your
|
||||
logs.
|
||||
|
||||
# graylog2
|
||||
|
||||
[http://graylog2.org/](http://graylog2.org)
|
||||
|
||||
_Overview to be written_
|
||||
|
||||
You can use graylog2 with logstash by using the 'gelf' output to send logstash
|
||||
events to a graylog2 server. This gives you logstash's excellent input and
|
||||
filter features while still being able to use the graylog2 web interface.
|
||||
|
||||
# whoops
|
||||
|
||||
[whoops site](http://www.whoopsapp.com/)
|
||||
|
||||
_Overview to be written_
|
||||
|
||||
A logstash output to whoops is coming soon - <https://logstash.jira.com/browse/LOGSTASH-133>
|
||||
|
||||
# flume
|
||||
|
||||
[flume site](https://github.com/cloudera/flume/wiki)
|
||||
|
||||
Flume is primarily a transport system aimed at reliably copying logs from
|
||||
application servers to HDFS.
|
||||
|
||||
You can use it with logstash by having a syslog sink configured to shoot logs
|
||||
at a logstash syslog input.
|
||||
|
||||
# scribe
|
||||
|
||||
_Overview to be written_
|
||||
|
|
@ -1,287 +0,0 @@
|
|||
---
|
||||
title: logstash docs for outputs/amqp
|
||||
layout: content_right
|
||||
---
|
||||
<h2>amqp</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">2</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>output {
|
||||
amqp {
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
||||
<a href="#durable">durable</a> => ... # boolean (optional), default: true
|
||||
<a href="#exchange">exchange</a> => ... # string (required)
|
||||
<a href="#exchange_type">exchange_type</a> => ... # string, one of ["fanout", "direct", "topic"] (required)
|
||||
<a href="#host">host</a> => ... # string (required)
|
||||
<a href="#key">key</a> => ... # string (optional), default: "logstash"
|
||||
<a href="#password">password</a> => ... # password (optional), default: "guest"
|
||||
<a href="#persistent">persistent</a> => ... # boolean (optional), default: true
|
||||
<a href="#port">port</a> => ... # number (optional), default: 5672
|
||||
<a href="#ssl">ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#user">user</a> => ... # string (optional), default: "guest"
|
||||
<a href="#verify_ssl">verify_ssl</a> => ... # boolean (optional), default: false
|
||||
<a href="#vhost">vhost</a> => ... # string (optional), default: "/"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for output data</p>
|
||||
|
||||
<h4>
|
||||
<a name="debug">
|
||||
debug
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="durable">
|
||||
durable
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="exchange">
|
||||
exchange (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="exchange_type">
|
||||
exchange_type (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "fanout", "direct", "topic" </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without any of these tags. Note this check is additional to type and tags.</p>
|
||||
|
||||
<h4>
|
||||
<a name="host">
|
||||
host (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="key">
|
||||
key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "logstash" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="password">
|
||||
password
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#password">password</a> </li>
|
||||
<li> Default value is "guest" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="persistent">
|
||||
persistent
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="port">
|
||||
port
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 5672 </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="ssl">
|
||||
ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all of these tags. Note that if you specify
|
||||
a type, the event must also match that type.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>The type to act on. If a type is given, then this output will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="user">
|
||||
user
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "guest" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="verify_ssl">
|
||||
verify_ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
<h4>
|
||||
<a name="vhost">
|
||||
vhost
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "/" </li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/outputs/amqp.rb">lib/logstash/outputs/amqp.rb</a>
|
||||
|
|
@ -1,235 +0,0 @@
|
|||
---
|
||||
title: logstash docs for outputs/boundary
|
||||
layout: content_right
|
||||
---
|
||||
<h2>boundary</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>output {
|
||||
boundary {
|
||||
<a href="#api_key">api_key</a> => ... # string (required)
|
||||
<a href="#auto">auto</a> => ... # boolean (optional), default: false
|
||||
<a href="#bsubtype">bsubtype</a> => ... # string (optional)
|
||||
<a href="#btags">btags</a> => ... # array (optional)
|
||||
<a href="#btype">btype</a> => ... # string (optional)
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#end_time">end_time</a> => ... # string (optional)
|
||||
<a href="#org_id">org_id</a> => ... # string (required)
|
||||
<a href="#start_time">start_time</a> => ... # string (optional)
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="api_key">
|
||||
api_key (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>This output lets you send annotations to
|
||||
Boundary based on Logstash events</p>
|
||||
|
||||
<p>Note that since Logstash maintains no state
|
||||
these will be one-shot events</p>
|
||||
|
||||
<p>By default the start and stop time will be
|
||||
the event timestamp</p>
|
||||
|
||||
<p>Your Boundary API key</p>
|
||||
|
||||
<h4>
|
||||
<a name="auto">
|
||||
auto
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is false </li>
|
||||
</ul>
|
||||
|
||||
<p>Auto
|
||||
If set to true, logstash will try to pull boundary fields out
|
||||
of the event. Any field explicitly set by config options will
|
||||
override these.
|
||||
['type', 'subtype', 'creation<em>time', 'end</em>time', 'links', 'tags', 'loc']</p>
|
||||
|
||||
<h4>
|
||||
<a name="bsubtype">
|
||||
bsubtype
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Sub-Type</p>
|
||||
|
||||
<h4>
|
||||
<a name="btags">
|
||||
btags
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Tags
|
||||
Set any custom tags for this event
|
||||
Default are the Logstash tags if any</p>
|
||||
|
||||
<h4>
|
||||
<a name="btype">
|
||||
btype
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Type</p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for output data</p>
|
||||
|
||||
<h4>
|
||||
<a name="end_time">
|
||||
end_time
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>End time
|
||||
Override the stop time
|
||||
Note that Boundary requires this to be seconds since epoch
|
||||
If overriding, it is your responsibility to type this correctly
|
||||
By default this is set to <code>event.unix_timestamp.to_i</code></p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without any of these tags. Note this check is additional to type and tags.</p>
|
||||
|
||||
<h4>
|
||||
<a name="org_id">
|
||||
org_id (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Your Boundary Org ID</p>
|
||||
|
||||
<h4>
|
||||
<a name="start_time">
|
||||
start_time
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Start time
|
||||
Override the start time
|
||||
Note that Boundary requires this to be seconds since epoch
|
||||
If overriding, it is your responsibility to type this correctly
|
||||
By default this is set to <code>event.unix_timestamp.to_i</code></p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all of these tags. Note that if you specify
|
||||
a type, the event must also match that type.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>The type to act on. If a type is given, then this output will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/outputs/boundary.rb">lib/logstash/outputs/boundary.rb</a>
|
||||
|
|
@ -1,155 +0,0 @@
|
|||
---
|
||||
title: logstash docs for outputs/circonus
|
||||
layout: content_right
|
||||
---
|
||||
<h2>circonus</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>output {
|
||||
circonus {
|
||||
<a href="#annotation">annotation</a> => ... # hash (required), default: {}
|
||||
<a href="#api_token">api_token</a> => ... # string (required)
|
||||
<a href="#app_name">app_name</a> => ... # string (required)
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="annotation">
|
||||
annotation (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> Default value is {} </li>
|
||||
</ul>
|
||||
|
||||
<p>Annotations
|
||||
Registers an annotation with Circonus
|
||||
The only required field is <code>title</code> and <code>description</code>.
|
||||
<code>start</code> and <code>stop</code> will be set to <code>event.unix_timestamp</code>
|
||||
You can add any other optional annotation values as well.
|
||||
All values will be passed through <code>event.sprintf</code></p>
|
||||
|
||||
<p>Example:
|
||||
["title":"Logstash event", "description":"Logstash event for %{host}"]
|
||||
or
|
||||
["title":"Logstash event", "description":"Logstash event for %{host}", "parent_id", "1"]</p>
|
||||
|
||||
<h4>
|
||||
<a name="api_token">
|
||||
api_token (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>This output lets you send annotations to
|
||||
Circonus based on Logstash events</p>
|
||||
|
||||
<p>Your Circonus API Token</p>
|
||||
|
||||
<h4>
|
||||
<a name="app_name">
|
||||
app_name (required setting)
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Your Circonus App name
|
||||
This will be passed through <code>event.sprintf</code>
|
||||
so variables are allowed here:</p>
|
||||
|
||||
<p>Example:
|
||||
<code>app_name => "%{myappname}"</code></p>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for output data</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without any of these tags. Note this check is additional to type and tags.</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all of these tags. Note that if you specify
|
||||
a type, the event must also match that type.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>The type to act on. If a type is given, then this output will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/outputs/circonus.rb">lib/logstash/outputs/circonus.rb</a>
|
||||
|
|
@ -1,457 +0,0 @@
|
|||
---
|
||||
title: logstash docs for outputs/cloudwatch
|
||||
layout: content_right
|
||||
---
|
||||
<h2>cloudwatch</h2>
|
||||
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
||||
|
||||
<p>This output lets you aggregate and send metric data to AWS CloudWatch</p>
|
||||
|
||||
<h4>Summary:</h4>
|
||||
|
||||
<p>This plugin is intended to be used on a logstash indexer agent (but that
|
||||
is not the only way, see below.) In the intended scenario, one cloudwatch
|
||||
output plugin is configured, on the logstash indexer node, with just AWS API
|
||||
credentials, and possibly a region and/or a namespace. The output looks
|
||||
for fields present in events, and when it finds them, it uses them to
|
||||
calculate aggregate statistics. If the <code>metricname</code> option is set in this
|
||||
output, then any events which pass through it will be aggregated & sent to
|
||||
CloudWatch, but that is not recommended. The intended use is to NOT set the
|
||||
metricname option here, and instead to add a <code>CW_metricname</code> field (and other
|
||||
fields) to only the events you want sent to CloudWatch.</p>
|
||||
|
||||
<p>When events pass through this output they are queued for background
|
||||
aggregation and sending, which happens every minute by default. The
|
||||
queue has a maximum size, and when it is full aggregated statistics will be
|
||||
sent to CloudWatch ahead of schedule. Whenever this happens a warning
|
||||
message is written to logstash's log. If you see this you should increase
|
||||
the <code>queue_size</code> configuration option to avoid the extra API calls. The queue
|
||||
is emptied every time we send data to CloudWatch.</p>
|
||||
|
||||
<p>Note: when logstash is stopped the queue is destroyed before it can be processed.
|
||||
This is a known limitation of logstash and will hopefully be addressed in a
|
||||
future version.</p>
|
||||
|
||||
<h4>Details:</h4>
|
||||
|
||||
<p>There are two ways to configure this plugin, and they can be used in
|
||||
combination: event fields & per-output defaults</p>
|
||||
|
||||
<p>Event Field configuration...
|
||||
You add fields to your events in inputs & filters and this output reads
|
||||
those fields to aggregate events. The names of the fields read are
|
||||
configurable via the <code>field_*</code> options.</p>
|
||||
|
||||
<p>Per-output defaults...
|
||||
You set universal defaults in this output plugin's configuration, and
|
||||
if an event does not have a field for that option then the default is
|
||||
used.</p>
|
||||
|
||||
<p>Notice, the event fields take precedence over the per-output defaults.</p>
|
||||
|
||||
<p>At a minimum events must have a "metric name" to be sent to CloudWatch.
|
||||
This can be achieved either by providing a default here OR by adding a
|
||||
<code>CW_metricname</code> field. By default, if no other configuration is provided
|
||||
besides a metric name, then events will be counted (Unit: Count, Value: 1)
|
||||
by their metric name (either a default or from their <code>CW_metricname</code> field)</p>
|
||||
|
||||
<p>Other fields which can be added to events to modify the behavior of this
|
||||
plugin are, <code>CW_namespace</code>, <code>CW_unit</code>, <code>CW_value</code>, and
|
||||
<code>CW_dimensions</code>. All of these field names are configurable in
|
||||
this output. You can also set per-output defaults for any of them.
|
||||
See below for details.</p>
|
||||
|
||||
<p>Read more about <a href="http://aws.amazon.com/cloudwatch/">AWS CloudWatch</a>,
|
||||
and the specific of API endpoint this output uses,
|
||||
<a href="http://docs.amazonwebservices.com/AmazonCloudWatch/latest/APIReference/API_PutMetricData.html">PutMetricData</a></p>
|
||||
|
||||
|
||||
<h3> Synopsis </h3>
|
||||
|
||||
This is what it might look like in your config file:
|
||||
|
||||
<pre><code>output {
|
||||
cloudwatch {
|
||||
<a href="#access_key_id">access_key_id</a> => ... # string (optional)
|
||||
<a href="#aws_credentials_file">aws_credentials_file</a> => ... # string (optional)
|
||||
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
||||
<a href="#dimensions">dimensions</a> => ... # hash (optional)
|
||||
<a href="#field_dimensions">field_dimensions</a> => ... # string (optional), default: "CW_dimensions"
|
||||
<a href="#field_metricname">field_metricname</a> => ... # string (optional), default: "CW_metricname"
|
||||
<a href="#field_namespace">field_namespace</a> => ... # string (optional), default: "CW_namespace"
|
||||
<a href="#field_unit">field_unit</a> => ... # string (optional), default: "CW_unit"
|
||||
<a href="#field_value">field_value</a> => ... # string (optional), default: "CW_value"
|
||||
<a href="#metricname">metricname</a> => ... # string (optional)
|
||||
<a href="#namespace">namespace</a> => ... # string (optional), default: "Logstash"
|
||||
<a href="#queue_size">queue_size</a> => ... # number (optional), default: 10000
|
||||
<a href="#region">region</a> => ... # string, one of ["us-east-1", "us-west-1", "us-west-2", "eu-west-1", "ap-southeast-1", "ap-southeast-2", "ap-northeast-1", "sa-east-1", "us-gov-west-1"] (optional), default: "us-east-1"
|
||||
<a href="#secret_access_key">secret_access_key</a> => ... # string (optional)
|
||||
<a href="#timeframe">timeframe</a> => ... # string (optional), default: "1m"
|
||||
<a href="#unit">unit</a> => ... # string, one of ["Seconds", "Microseconds", "Milliseconds", "Bytes", "Kilobytes", "Megabytes", "Gigabytes", "Terabytes", "Bits", "Kilobits", "Megabits", "Gigabits", "Terabits", "Percent", "Count", "Bytes/Second", "Kilobytes/Second", "Megabytes/Second", "Gigabytes/Second", "Terabytes/Second", "Bits/Second", "Kilobits/Second", "Megabits/Second", "Gigabits/Second", "Terabits/Second", "Count/Second", "None"] (optional), default: "Count"
|
||||
<a href="#use_ssl">use_ssl</a> => ... # boolean (optional), default: true
|
||||
<a href="#value">value</a> => ... # string (optional), default: "1"
|
||||
}
|
||||
|
||||
}
|
||||
</code></pre>
|
||||
|
||||
<h3> Details </h3>
|
||||
|
||||
<h4>
|
||||
<a name="access_key_id">
|
||||
access_key_id
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>This plugin uses the AWS SDK and supports several ways to get credentials, which will be tried in this order... <br/>
|
||||
1. Static configuration, using <code>access_key_id</code> and <code>secret_access_key</code> params in logstash plugin config <br/>
|
||||
2. External credentials file specified by <code>aws_credentials_file</code> <br/>
|
||||
3. Environment variables <code>AWS_ACCESS_KEY_ID</code> and <code>AWS_SECRET_ACCESS_KEY</code> <br/>
|
||||
4. Environment variables <code>AMAZON_ACCESS_KEY_ID</code> and <code>AMAZON_SECRET_ACCESS_KEY</code> <br/>
|
||||
5. IAM Instance Profile (available when running inside EC2)</p>
|
||||
|
||||
<h4>
|
||||
<a name="aws_credentials_file">
|
||||
aws_credentials_file
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>Path to YAML file containing a hash of AWS credentials. <br/>
|
||||
This file will only be loaded if <code>access_key_id</code> and
|
||||
<code>secret_access_key</code> aren't set. The contents of the
|
||||
file should look like this:</p>
|
||||
|
||||
<pre><code>:access_key_id: "12345"
|
||||
:secret_access_key: "54321"
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="codec">
|
||||
codec
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
||||
<li> Default value is "plain" </li>
|
||||
</ul>
|
||||
|
||||
<p>The codec used for output data</p>
|
||||
|
||||
<h4>
|
||||
<a name="dimensions">
|
||||
dimensions
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The default dimensions [ name, value, ... ] to use for events which do not have a <code>CW_dimensions</code> field</p>
|
||||
|
||||
<h4>
|
||||
<a name="exclude_tags">
|
||||
exclude_tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events without any of these tags. Note this check is additional to type and tags.</p>
|
||||
|
||||
<h4>
|
||||
<a name="field_dimensions">
|
||||
field_dimensions
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "CW_dimensions" </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the field used to set the dimensions on an event metric <br/>
|
||||
The field named here, if present in an event, must have an array of
|
||||
one or more key & value pairs, for example...</p>
|
||||
|
||||
<pre><code>add_field => [ "CW_dimensions", "Environment", "CW_dimensions", "prod" ]
|
||||
</code></pre>
|
||||
|
||||
<p>or, equivalently...</p>
|
||||
|
||||
<pre><code>add_field => [ "CW_dimensions", "Environment" ]
|
||||
add_field => [ "CW_dimensions", "prod" ]
|
||||
</code></pre>
|
||||
|
||||
<h4>
|
||||
<a name="field_metricname">
|
||||
field_metricname
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "CW_metricname" </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the field used to set the metric name on an event <br/>
|
||||
The author of this plugin recommends adding this field to events in inputs &
|
||||
filters rather than using the per-output default setting so that one output
|
||||
plugin on your logstash indexer can serve all events (which of course had
|
||||
fields set on your logstash shippers.)</p>
|
||||
|
||||
<h4>
|
||||
<a name="field_namespace">
|
||||
field_namespace
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "CW_namespace" </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the field used to set a different namespace per event <br/>
|
||||
Note: Only one namespace can be sent to CloudWatch per API call
|
||||
so setting different namespaces will increase the number of API calls
|
||||
and those cost money.</p>
|
||||
|
||||
<h4>
|
||||
<a name="field_unit">
|
||||
field_unit
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "CW_unit" </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the field used to set the unit on an event metric</p>
|
||||
|
||||
<h4>
|
||||
<a name="field_value">
|
||||
field_value
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "CW_value" </li>
|
||||
</ul>
|
||||
|
||||
<p>The name of the field used to set the value (float) on an event metric</p>
|
||||
|
||||
<h4>
|
||||
<a name="metricname">
|
||||
metricname
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The default metric name to use for events which do not have a <code>CW_metricname</code> field. <br/>
|
||||
Beware: If this is provided then all events which pass through this output will be aggregated and
|
||||
sent to CloudWatch, so use this carefully. Furthermore, when providing this option, you
|
||||
will probably want to also restrict events from passing through this output using event
|
||||
type, tag, and field matching</p>
|
||||
|
||||
<h4>
|
||||
<a name="namespace">
|
||||
namespace
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "Logstash" </li>
|
||||
</ul>
|
||||
|
||||
<p>The default namespace to use for events which do not have a <code>CW_namespace</code> field</p>
|
||||
|
||||
<h4>
|
||||
<a name="queue_size">
|
||||
queue_size
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#number">number</a> </li>
|
||||
<li> Default value is 10000 </li>
|
||||
</ul>
|
||||
|
||||
<p>How many events to queue before forcing a call to the CloudWatch API ahead of <code>timeframe</code> schedule <br/>
|
||||
Set this to the number of events-per-timeframe you will be sending to CloudWatch to avoid extra API calls</p>
|
||||
|
||||
<h4>
|
||||
<a name="region">
|
||||
region
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "us-east-1", "us-west-1", "us-west-2", "eu-west-1", "ap-southeast-1", "ap-southeast-2", "ap-northeast-1", "sa-east-1", "us-gov-west-1" </li>
|
||||
<li> Default value is "us-east-1" </li>
|
||||
</ul>
|
||||
|
||||
<p>The AWS Region</p>
|
||||
|
||||
<h4>
|
||||
<a name="secret_access_key">
|
||||
secret_access_key
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> There is no default value for this setting. </li>
|
||||
</ul>
|
||||
|
||||
<p>The AWS Secret Access Key</p>
|
||||
|
||||
<h4>
|
||||
<a name="tags">
|
||||
tags
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#array">array</a> </li>
|
||||
<li> Default value is [] </li>
|
||||
</ul>
|
||||
|
||||
<p>Only handle events with all of these tags. Note that if you specify
|
||||
a type, the event must also match that type.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="timeframe">
|
||||
timeframe
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "1m" </li>
|
||||
</ul>
|
||||
|
||||
<p>Constants
|
||||
aggregate_key members
|
||||
Units
|
||||
How often to send data to CloudWatch <br/>
|
||||
This does not affect the event timestamps, events will always have their
|
||||
actual timestamp (to-the-minute) sent to CloudWatch.</p>
|
||||
|
||||
<p>We only call the API if there is data to send.</p>
|
||||
|
||||
<p>See the Rufus Scheduler docs for an <a href="https://github.com/jmettraux/rufus-scheduler#the-time-strings-understood-by-rufus-scheduler">explanation of allowed values</a></p>
|
||||
|
||||
<h4>
|
||||
<a name="type">
|
||||
type
|
||||
<strong>DEPRECATED</strong>
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "" </li>
|
||||
</ul>
|
||||
|
||||
<p>The type to act on. If a type is given, then this output will only
|
||||
act on messages with the same type. See any input plugin's "type"
|
||||
attribute for more.
|
||||
Optional.</p>
|
||||
|
||||
<h4>
|
||||
<a name="unit">
|
||||
unit
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value can be any of: "Seconds", "Microseconds", "Milliseconds", "Bytes", "Kilobytes", "Megabytes", "Gigabytes", "Terabytes", "Bits", "Kilobits", "Megabits", "Gigabits", "Terabits", "Percent", "Count", "Bytes/Second", "Kilobytes/Second", "Megabytes/Second", "Gigabytes/Second", "Terabytes/Second", "Bits/Second", "Kilobits/Second", "Megabits/Second", "Gigabits/Second", "Terabits/Second", "Count/Second", "None" </li>
|
||||
<li> Default value is "Count" </li>
|
||||
</ul>
|
||||
|
||||
<p>The default unit to use for events which do not have a <code>CW_unit</code> field <br/>
|
||||
If you set this option you should probably set the "value" option along with it</p>
|
||||
|
||||
<h4>
|
||||
<a name="use_ssl">
|
||||
use_ssl
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
||||
<li> Default value is true </li>
|
||||
</ul>
|
||||
|
||||
<p>Should we require (true) or disable (false) using SSL for communicating with the AWS API <br/>
|
||||
The AWS SDK for Ruby defaults to SSL so we preserve that</p>
|
||||
|
||||
<h4>
|
||||
<a name="value">
|
||||
value
|
||||
|
||||
</a>
|
||||
</h4>
|
||||
|
||||
<ul>
|
||||
<li> Value type is <a href="../configuration#string">string</a> </li>
|
||||
<li> Default value is "1" </li>
|
||||
</ul>
|
||||
|
||||
<p>The default value to use for events which do not have a <code>CW_value</code> field <br/>
|
||||
If provided, this must be a string which can be converted to a float, for example...</p>
|
||||
|
||||
<pre><code>"1", "2.34", ".5", and "0.67"
|
||||
</code></pre>
|
||||
|
||||
<p>If you set this option you should probably set the <code>unit</code> option along with it</p>
|
||||
|
||||
|
||||
<hr>
|
||||
|
||||
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/outputs/cloudwatch.rb">lib/logstash/outputs/cloudwatch.rb</a>
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue