github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-24 06:37:19 -04:00
Code Issues Projects Releases Packages Wiki Activity
10927 commits 258 branches 379 tags 173 MiB
Commit graph

10927 commits

This branch
This branch
All branches
Author SHA1 Message Date
Jordan Sissel
791f15f633 - Add the new refactored agent model. 
An agent can read from any input, apply filters, and pass to any output.
    * Inputs are files, amqp, stomp, http server, syslog server, etc.
    * Outputs are similar.
    * Filters are for manipulating events (parsing, adding data, trimming
      private data, etc)
  Inputs so far: amqp, file.
  Outputs so far: amqp, stdout.
  Filters so far: grok (pattern discovery only)

  A sample custom agent is in examples/test.rb
 2010-10-17 10:27:38 +00:00
Jordan Sissel
1575edcdb1 - Add time wrapper for iso8601 support 2010-10-17 07:27:24 +00:00
Jordan Sissel
d9c8381754 - Add base for unified invocation binary 2010-10-17 07:13:38 +00:00
Jordan Sissel
7a6808418f - Don't use File::Tail anymore 2010-10-17 06:57:01 +00:00
Jordan Sissel
524194d5de 2010-10-17 03:20:05 +00:00
Jordan Sissel
c4c880f51d 2010-10-17 03:19:57 +00:00
Jordan Sissel
d3280be19b - start working on splitting things in to components. I want it to be possible 
to run a single agent that does log reading, parsing, and indexing locally.
 2010-10-17 03:19:45 +00:00
Jordan Sissel
ae21f3d7c0 - Fix eventmachine-tail usage to match 0.2.x api 2010-05-18 08:11:33 +00:00
Jordan Sissel
4aea3bddb3 - Sync grok-patterns from grok svn HEAD. 2010-05-18 08:11:06 +00:00
Jordan Sissel
c1ecbff37c - If no timezone is in the date format, assume localtime and move to UTC time. 2010-05-18 08:10:46 +00:00
Jordan Sissel
6b48e99eaa - Add search form partial 2010-05-04 15:53:27 +00:00
Jordan Sissel
58dc837a14 - Add a title to the graph showing the range (time period) 
- Add a vertical dash on the graph indicating the current time
- Make / (index) show a better view of the search form
 2010-05-04 03:35:33 +00:00
Jordan Sissel
5e955a47ba - Fix output showing result range if total results is less than the query 'limit' 
- Show date stamp of each result.
 2010-05-03 21:36:07 +00:00
Jordan Sissel
9795bb56b5 - Fix time conversion for the graph 
- Fix sort (elasticsearch wants an array of fields, not just a string)
- Allow tunable graph steps (default 3600 seconds)
- Skip empty capture vaules
- add elasticsearch_host to BaseConfig
 2010-05-02 07:47:01 +00:00
Jordan Sissel
5b9e83ce13 - fix version generation (major.svnrev), current major is 0. 
- Add missing files to spec
 2010-04-27 08:36:54 +00:00
Jordan Sissel
4c95afa52c - Add files to spec 2010-04-27 08:26:31 +00:00
Jordan Sissel
b4865b76b0 - Move elasticsearch support into LogStash::Net::Clients::ElasticSearch 2010-04-26 09:26:06 +00:00
Jordan Sissel
877e527d91 - Add elasticsearch_host config option 
- Make LogStash::Log index to elasticsearch
 2010-04-26 09:24:09 +00:00
Jordan Sissel
2c299fae25 - Fix search's graph 2010-04-25 20:56:39 +00:00
Jordan Sissel
1cb56a94d4 - Use elasticsearch again for a indexing backend. 
- Update logstash web to hit elasticsearch (hardcoded host for now)
 2010-04-25 09:12:45 +00:00
Jordan Sissel
32ca7fbaea - copy grok patterns again 2010-04-14 09:19:21 +00:00
Jordan Sissel
5c8bef85db - just print matching lines 2010-04-14 09:06:36 +00:00
Jordan Sissel
418efeb2fd - Try full-text search, limit 10 results 2010-04-14 09:03:42 +00:00
Jordan Sissel
5c9922be05 - Add new indexing that uses TokyoCabinet table db for storage. 2010-04-14 07:44:15 +00:00
Jordan Sissel
cbfa08081d - Copy new patterns from grok 2010-04-14 07:42:33 +00:00
Jordan Sissel
6cd4f63cb4 2010-04-13 08:12:04 +00:00
Jordan Sissel
9cc9b2b952 2010-04-13 08:11:57 +00:00
Jordan Sissel
8965fa6c49 - Rewrite Agent for local-running only. The goal here is mainly to make the 
agent EventMachine-safe. Later I'll pull the index/parse functionality into a module
  that can be included into the agent for optional local-only or network modes.
 2010-04-13 07:03:36 +00:00
Jordan Sissel
7f66dcc2ea - use event machine timers 2010-04-11 15:51:36 +00:00
Jordan Sissel
e020e8487e - use elasticsearch instead of an indexer 2010-03-04 06:33:59 +00:00
Jordan Sissel
d84f742554 - can't include a class, only a module. Fix later. 2010-02-19 07:50:53 +00:00
Jordan Sissel
056a1d6920 - Move to use ElasticSearch's REST api for indexing instead of our own custom mqrpc+ferret instance. 2010-02-19 07:47:10 +00:00
Jordan Sissel
44d8eb4143 - Grab the pidfile before we daemonize so we can report pid lock errors 
promptly.
 2010-02-19 06:58:47 +00:00
Jordan Sissel
f079e8e2be - fix class usage. InvalidArgument isn't valid, we want ArgumentError 2010-01-26 09:05:29 +00:00
Pete Fritchman
4b3460491a - s/info/debug/ for "Sending" messages 2009-12-16 23:09:00 +00:00
Pete Fritchman
5143d3b85b - bump to 0.3.6 2009-11-30 22:12:23 +00:00
Pete Fritchman
468ae95ca2 - create a new thread for watching logs so we can properly handle globs. 
We don't just expand the glob on start, we constantly re-scan for any
  new log files.
 2009-11-15 05:35:26 +00:00
Pete Fritchman
c5448cf479 - detect when an index is corrupt (index.flush throws an exception), 
log a fatal error, and kill the entire logstashd process.
 2009-11-08 07:20:38 +00:00
Pete Fritchman
8b7b1b4623 - use proper variable name for termination callback block 2009-11-08 06:31:37 +00:00
Jordan Sissel
3a02fb6abd - migrate up some code 2009-11-08 06:14:46 +00:00
Jordan Sissel
1c5c6b8c03 - don't require a file we don't have naymore 2009-11-08 06:10:15 +00:00
Pete Fritchman
2a4e947c34 - don't start up a File::Tail::Since until the file exists 
- if File::Tail::Since does exit, we want to know about it
 2009-11-08 05:18:48 +00:00
Pete Fritchman
fff7413057 - gracefully handle errors from log parsers 
+ they can throw a LogStash::Log::LogParseError now
- handle JSON parse exceptions
 2009-11-08 05:07:12 +00:00
Jordan Sissel
e838dfaee4 - Remove old SlidingWindowSet code that MQRPC replaced with SizedThreadSafeHash 2009-11-07 23:43:39 +00:00
Jordan Sissel
8c22b1f314 - Style 2009-11-07 22:37:38 +00:00
Jordan Sissel
445eada3e2 - use Util::collapse 2009-11-07 22:36:39 +00:00
Jordan Sissel
b22506fa17 - style fix 2009-11-07 22:34:30 +00:00
Jordan Sissel
50eb137e1e - Use mqrpc's new 'delayable' message header 2009-11-07 21:58:40 +00:00
Pete Fritchman
f4b0fafc00 - bump to 0.3.5 (mqrpc split-out) 
- include wiki docs in rpm
 2009-11-07 21:37:29 +00:00
Jordan Sissel
c234e01f38 - Messages sent from parser -> indexer that came from an IndexEventRequest now 
get pushed into a queue that is managed by a separate thread. This is
  necessary to prevent sendmsg() from blocking the main AMQP reader thread when
  sendmsg() might block due to sliding window closure.

  This queue length is unchecked, however, the correct fix is to unsubscribe
  from the input (the AMQP queue) and only resubscribe once our queue has
  cleared a bit.
 2009-11-06 10:16:44 +00:00