github-mirrors/logstash
1
0
Fork 0
mirror of https://github.com/elastic/logstash.git synced 2025-04-19 04:15:23 -04:00
Code Issues Projects Releases Packages Wiki Activity
11063 commits 255 branches 379 tags 173 MiB
Commit graph

11063 commits

This branch
This branch
All branches
Author SHA1 Message Date
Pete Fritchman
71c6cb138f - all reflected in wiki/Design now 2009-08-09 18:43:00 +00:00
Jordan Sissel
c0fd574685 - Read larger chunks into our buffer when reading messages 
- Make the client read from a file and dump 20 messages at a time to the server.
 2009-08-09 11:07:48 +00:00
Jordan Sissel
c79ed129cf - MessageStream should keep a count of its messages 
- Add MessageStream.clear for wiping messages in the stream.
- Make socket reading faster and more reliable (since read(N) may return elss
  than N bytes)

  Fun stats, dumping lines of apache logs into sandbox/srv.rb 
  from sandbox/client.rb:
    1421000 finished @ 163/sec => 8680.2 secs
    1422000 finished @ 163/sec => 8678.5 secs
    1423000 finished @ 163/sec => 8679.1 secs

  8500 lines per second? That's about 2MB/sec. Not bad?
 2009-08-09 10:45:12 +00:00
Jordan Sissel
45663365a5 - Start work on network layer. Messages are automagically decoded off the wire 
to the correct message instance (an IndexEvent request becomes an
  IndexEventRequest instance, etc)
- We use some metaprogramming tricks to bind Message JSON fields to
  functions, see BindToHash and hashbind in net/message.rb
- Protocol versioning is poor right now, but it is present.
- The server code is not well-written, yet. I just wanted real client/server
  encode/decode testing.
- Add some play code to sandbox/
 2009-08-09 09:40:57 +00:00
Pete Fritchman
e7a6f57274 - respect LOGSTASH_HOME 2009-08-09 03:30:17 +00:00
Pete Fritchman
a6f282c4d7 - let ferret index Arrays, it does the right thing. 2009-08-09 02:28:51 +00:00
Pete Fritchman
3d9cf8e58f - add profiling (temporary, but very useful for now) 2009-08-07 00:52:17 +00:00
Jordan Sissel
127394a84b - Use RubyGrok instead of IO.popen("grok ...") 
Requires installed: http://semicomplete.googlecode.com/svn/cgrok/ruby/
- use $HOME for logstash index directory.
- Have import use Time.now.to_f to get higher-precision time values.
 2009-08-06 08:39:05 +00:00
Pete Fritchman
b92be62794 - remove the grok teardown stuff, since we're going to be using cgrok's 
new ruby api
- ignore fields not explicitly named in grok (i.e. %{FOO} is ignored,
  but %{FOO:bar} is imported as key=bar)
 2009-08-06 05:50:52 +00:00
Pete Fritchman
002c080abb - display import rate 2009-08-06 05:38:20 +00:00
Pete Fritchman
944939a7ee - search by @DATE, by default 2009-08-06 05:37:59 +00:00
Pete Fritchman
94944089d9 - actually create an index from the default FieldInfos. now that we're 
using these defaults, importing is faster and there is a much better
  log:index size ratio.
 2009-08-06 05:37:23 +00:00
Pete Fritchman
82fdc11f2e - early early early-stage logstash prototype 2009-08-05 01:01:23 +00:00