mirror of
https://github.com/elastic/logstash.git
synced 2025-04-24 14:47:19 -04:00
274 lines
8.3 KiB
HTML
274 lines
8.3 KiB
HTML
---
|
|
title: logstash docs for inputs/sqlite
|
|
layout: content_right
|
|
---
|
|
<h2>sqlite</h2>
|
|
<h3>Milestone: <a href="../plugin-milestones">1</a></h3>
|
|
|
|
<p>Read rows from an sqlite database.</p>
|
|
|
|
<p>This is most useful in cases where you are logging directly to a table.
|
|
Any tables being watched must have an 'id' column that is monotonically
|
|
increasing.</p>
|
|
|
|
<p>All tables are read by default except:
|
|
* ones matching 'sqlite<em>%' - these are internal/adminstrative tables for sqlite
|
|
* 'since</em>table' - this is used by this plugin to track state.</p>
|
|
|
|
<h2>Example</h2>
|
|
|
|
<pre><code>% sqlite /tmp/example.db
|
|
sqlite> CREATE TABLE weblogs (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
ip STRING,
|
|
request STRING,
|
|
response INTEGER);
|
|
sqlite> INSERT INTO weblogs (ip, request, response)
|
|
VALUES ("1.2.3.4", "/index.html", 200);
|
|
</code></pre>
|
|
|
|
<p>Then with this logstash config:</p>
|
|
|
|
<pre><code>input {
|
|
sqlite {
|
|
path => "/tmp/example.db"
|
|
type => weblogs
|
|
}
|
|
}
|
|
output {
|
|
stdout {
|
|
debug => true
|
|
}
|
|
}
|
|
</code></pre>
|
|
|
|
<p>Sample output:</p>
|
|
|
|
<pre><code>{
|
|
"@source" => "sqlite://sadness/tmp/x.db",
|
|
"@tags" => [],
|
|
"@fields" => {
|
|
"ip" => "1.2.3.4",
|
|
"request" => "/index.html",
|
|
"response" => 200
|
|
},
|
|
"@timestamp" => "2013-05-29T06:16:30.850Z",
|
|
"@source_host" => "sadness",
|
|
"@source_path" => "/tmp/x.db",
|
|
"@message" => "",
|
|
"@type" => "foo"
|
|
}
|
|
</code></pre>
|
|
|
|
|
|
<h3> Synopsis </h3>
|
|
|
|
This is what it might look like in your config file:
|
|
|
|
<pre><code>input {
|
|
sqlite {
|
|
<a href="#add_field">add_field</a> => ... # hash (optional), default: {}
|
|
<a href="#batch">batch</a> => ... # number (optional), default: 5
|
|
<a href="#codec">codec</a> => ... # codec (optional), default: "plain"
|
|
<a href="#debug">debug</a> => ... # boolean (optional), default: false
|
|
<a href="#exclude_tables">exclude_tables</a> => ... # array (optional), default: []
|
|
<a href="#path">path</a> => ... # string (required)
|
|
<a href="#tags">tags</a> => ... # array (optional)
|
|
<a href="#type">type</a> => ... # string (optional)
|
|
}
|
|
|
|
}
|
|
</code></pre>
|
|
|
|
<h3> Details </h3>
|
|
|
|
<h4>
|
|
<a name="add_field">
|
|
add_field
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#hash">hash</a> </li>
|
|
<li> Default value is {} </li>
|
|
</ul>
|
|
|
|
<p>Add a field to an event</p>
|
|
|
|
<h4>
|
|
<a name="batch">
|
|
batch
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#number">number</a> </li>
|
|
<li> Default value is 5 </li>
|
|
</ul>
|
|
|
|
<p>How many rows to fetch at a time from each SELECT call.</p>
|
|
|
|
<h4>
|
|
<a name="charset">
|
|
charset
|
|
<strong>DEPRECATED</strong>
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
|
<li> Value can be any of: "ASCII-8BIT", "UTF-8", "US-ASCII", "Big5", "Big5-HKSCS", "Big5-UAO", "CP949", "Emacs-Mule", "EUC-JP", "EUC-KR", "EUC-TW", "GB18030", "GBK", "ISO-8859-1", "ISO-8859-2", "ISO-8859-3", "ISO-8859-4", "ISO-8859-5", "ISO-8859-6", "ISO-8859-7", "ISO-8859-8", "ISO-8859-9", "ISO-8859-10", "ISO-8859-11", "ISO-8859-13", "ISO-8859-14", "ISO-8859-15", "ISO-8859-16", "KOI8-R", "KOI8-U", "Shift_JIS", "UTF-16BE", "UTF-16LE", "UTF-32BE", "UTF-32LE", "Windows-1251", "BINARY", "IBM437", "CP437", "IBM737", "CP737", "IBM775", "CP775", "CP850", "IBM850", "IBM852", "CP852", "IBM855", "CP855", "IBM857", "CP857", "IBM860", "CP860", "IBM861", "CP861", "IBM862", "CP862", "IBM863", "CP863", "IBM864", "CP864", "IBM865", "CP865", "IBM866", "CP866", "IBM869", "CP869", "Windows-1258", "CP1258", "GB1988", "macCentEuro", "macCroatian", "macCyrillic", "macGreek", "macIceland", "macRoman", "macRomania", "macThai", "macTurkish", "macUkraine", "CP950", "Big5-HKSCS:2008", "CP951", "stateless-ISO-2022-JP", "eucJP", "eucJP-ms", "euc-jp-ms", "CP51932", "eucKR", "eucTW", "GB2312", "EUC-CN", "eucCN", "GB12345", "CP936", "ISO-2022-JP", "ISO2022-JP", "ISO-2022-JP-2", "ISO2022-JP2", "CP50220", "CP50221", "ISO8859-1", "Windows-1252", "CP1252", "ISO8859-2", "Windows-1250", "CP1250", "ISO8859-3", "ISO8859-4", "ISO8859-5", "ISO8859-6", "Windows-1256", "CP1256", "ISO8859-7", "Windows-1253", "CP1253", "ISO8859-8", "Windows-1255", "CP1255", "ISO8859-9", "Windows-1254", "CP1254", "ISO8859-10", "ISO8859-11", "TIS-620", "Windows-874", "CP874", "ISO8859-13", "Windows-1257", "CP1257", "ISO8859-14", "ISO8859-15", "ISO8859-16", "CP878", "Windows-31J", "CP932", "csWindows31J", "SJIS", "PCK", "MacJapanese", "MacJapan", "ASCII", "ANSI_X3.4-1968", "646", "UTF-7", "CP65000", "CP65001", "UTF8-MAC", "UTF-8-MAC", "UTF-8-HFS", "UTF-16", "UTF-32", "UCS-2BE", "UCS-4BE", "UCS-4LE", "CP1251", "UTF8-DoCoMo", "SJIS-DoCoMo", "UTF8-KDDI", "SJIS-KDDI", "ISO-2022-JP-KDDI", "stateless-ISO-2022-JP-KDDI", "UTF8-SoftBank", "SJIS-SoftBank", "locale", "external", "filesystem", "internal" </li>
|
|
<li> There is no default value for this setting. </li>
|
|
</ul>
|
|
|
|
<p>The character encoding used in this input. Examples include "UTF-8"
|
|
and "cp1252"</p>
|
|
|
|
<p>This setting is useful if your log files are in Latin-1 (aka cp1252)
|
|
or in another character set other than UTF-8.</p>
|
|
|
|
<p>This only affects "plain" format logs since json is UTF-8 already.</p>
|
|
|
|
<h4>
|
|
<a name="codec">
|
|
codec
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#codec">codec</a> </li>
|
|
<li> Default value is "plain" </li>
|
|
</ul>
|
|
|
|
<p>The codec used for input data</p>
|
|
|
|
<h4>
|
|
<a name="debug">
|
|
debug
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#boolean">boolean</a> </li>
|
|
<li> Default value is false </li>
|
|
</ul>
|
|
|
|
<p>Set this to true to enable debugging on an input.</p>
|
|
|
|
<h4>
|
|
<a name="exclude_tables">
|
|
exclude_tables
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#array">array</a> </li>
|
|
<li> Default value is [] </li>
|
|
</ul>
|
|
|
|
<p>Any tables to exclude by name.
|
|
By default all tables are followed.</p>
|
|
|
|
<h4>
|
|
<a name="format">
|
|
format
|
|
<strong>DEPRECATED</strong>
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
|
<li> Value can be any of: "plain", "json", "json_event", "msgpack_event" </li>
|
|
<li> There is no default value for this setting. </li>
|
|
</ul>
|
|
|
|
<p>The format of input data (plain, json, json_event)</p>
|
|
|
|
<h4>
|
|
<a name="message_format">
|
|
message_format
|
|
<strong>DEPRECATED</strong>
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> DEPRECATED WARNING: This config item is deprecated. It may be removed in a further version. </li>
|
|
<li> Value type is <a href="../configuration#string">string</a> </li>
|
|
<li> There is no default value for this setting. </li>
|
|
</ul>
|
|
|
|
<p>If format is "json", an event sprintf string to build what
|
|
the display @message should be given (defaults to the raw JSON).
|
|
sprintf format strings look like %{fieldname}</p>
|
|
|
|
<p>If format is "json_event", ALL fields except for @type
|
|
are expected to be present. Not receiving all fields
|
|
will cause unexpected results.</p>
|
|
|
|
<h4>
|
|
<a name="path">
|
|
path (required setting)
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#string">string</a> </li>
|
|
<li> There is no default value for this setting. </li>
|
|
</ul>
|
|
|
|
<p>The path to the sqlite database file.</p>
|
|
|
|
<h4>
|
|
<a name="tags">
|
|
tags
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#array">array</a> </li>
|
|
<li> There is no default value for this setting. </li>
|
|
</ul>
|
|
|
|
<p>Add any number of arbitrary tags to your event.</p>
|
|
|
|
<p>This can help with processing later.</p>
|
|
|
|
<h4>
|
|
<a name="type">
|
|
type
|
|
|
|
</a>
|
|
</h4>
|
|
|
|
<ul>
|
|
<li> Value type is <a href="../configuration#string">string</a> </li>
|
|
<li> There is no default value for this setting. </li>
|
|
</ul>
|
|
|
|
<p>Add a 'type' field to all events handled by this input.</p>
|
|
|
|
<p>Types are used mainly for filter activation.</p>
|
|
|
|
<p>If you create an input with type "foobar", then only filters
|
|
which also have type "foobar" will act on them.</p>
|
|
|
|
<p>The type is also stored as part of the event itself, so you
|
|
can also use the type to search for in the web interface.</p>
|
|
|
|
<p>If you try to set a type on an event that already has one (for
|
|
example when you send an event from a shipper to an indexer) then
|
|
a new input will not override the existing type. A type set at
|
|
the shipper stays with that event for its life even
|
|
when sent to another LogStash server.</p>
|
|
|
|
|
|
<hr>
|
|
|
|
This is documentation from <a href="https://github.com/logstash/logstash/blob/v1.2.0.beta1/lib/logstash/inputs/sqlite.rb">lib/logstash/inputs/sqlite.rb</a>
|