github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-24 22:57:16 -04:00
Code Issues Projects Releases Packages Wiki Activity

log files path into /tmp, added missing mutate + replace statements

Browse source
This commit is contained in:
Colin Surprenant 2014-02-24 19:00:54 -05:00
parent 71e8432998
commit 6ed923f0cf
1 changed files with 15 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

24
docs/tutorials/getting-started-with-logstash.asciidoc
View file

@ -260,7 +260,7 @@ Now, let's configure something actually *useful*... apache2 access log files! We
---- ----
input { input {
file { file {
path => "/Users/kurt/logs/access_log" path => "/tmp/access_log"
start_position => beginning start_position => beginning
} }
} }
@ -285,7 +285,7 @@ output {
} }
---- ----
Then, create the file you configured above (in this example, "/Applications/XAMPP/logs/access_log") with the following log lines as contents (or use some from your own webserver): Then, create the file you configured above (in this example, "/tmp/access_log") with the following log lines as contents (or use some from your own webserver):
---- ----
71.141.244.242 - kurt [18/May/2011:01:48:10 -0700] "GET /admin HTTP/1.1" 301 566 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3" 71.141.244.242 - kurt [18/May/2011:01:48:10 -0700] "GET /admin HTTP/1.1" 301 566 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"
@ -304,7 +304,7 @@ In this configuration, Logstash is only watching the apache access_log, but it's
---- ----
input { input {
file { file {
path => "/Applications/XAMPP/logs/*_log" path => "/tmp/*_log"
... ...
---- ----
Now, rerun Logstash, and you will see both the error and access logs processed via Logstash. However, if you inspect your data (using elasticsearch-kopf, perhaps), you will see that the access_log was broken up into discrete fields, but not the error_log. That's because we used a "grok" filter to match the standard combined apache log format and automatically split the data into separate fields. Wouldn't it be nice *if* we could control how a line was parsed, based on its format? Well, we can... Now, rerun Logstash, and you will see both the error and access logs processed via Logstash. However, if you inspect your data (using elasticsearch-kopf, perhaps), you will see that the access_log was broken up into discrete fields, but not the error_log. That's because we used a "grok" filter to match the standard combined apache log format and automatically split the data into separate fields. Wouldn't it be nice *if* we could control how a line was parsed, based on its format? Well, we can...
@ -317,13 +317,13 @@ Now we can build on the previous example, where we introduced the concept of a *
---- ----
input { input {
file { file {
path => "/Applications/XAMPP/logs/*_log" path => "/tmp/*_log"
} }
} }
filter { filter {
if [path] =~ "access" { if [path] =~ "access" {
type => "apache_access" mutate { replace => { type => "apache_access" } }
grok { grok {
match => { "message" => "%{COMBINEDAPACHELOG}" } match => { "message" => "%{COMBINEDAPACHELOG}" }
} }
@ -331,9 +331,9 @@ filter {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ] match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
} }
} else if [path] =~ "error" { } else if [path] =~ "error" {
type => "apache_error" mutate { replace => { type => "apache_error" } }
} else { } else {
type => "random_logs" mutate { replace => { type => "random_logs" } }
} }
} }
@ -395,17 +395,11 @@ You can copy and paste the following lines as samples (feel free to try some of
---- ----
Dec 23 12:11:43 louis postfix/smtpd[31499]: connect from unknown[95.75.93.154] Dec 23 12:11:43 louis postfix/smtpd[31499]: connect from unknown[95.75.93.154]
----
----
Dec 23 14:42:56 louis named[16000]: client 199.48.164.7#64817: query (cache) 'amsterdamboothuren.com/MX/IN' denied Dec 23 14:42:56 louis named[16000]: client 199.48.164.7#64817: query (cache) 'amsterdamboothuren.com/MX/IN' denied
----
----
Dec 23 14:30:01 louis CRON[619]: (www-data) CMD (php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log) Dec 23 14:30:01 louis CRON[619]: (www-data) CMD (php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log)
Dec 22 18:28:06 louis rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="2253" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, type 'lightweight'.
---- ----
----
Dec 22 18:28:06 louis rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="2253" x-info="http://www.rsyslog.com"] rsyslogd was HUPed, ty
pe 'lightweight'.
----
Now you should see the output of Logstash in your original shell as it processes and parses messages! Now you should see the output of Logstash in your original shell as it processes and parses messages!
---- ----