Commit e62536a introduced a complication: there are times when one
wants to match against zero as well as the positive integers (such
as in the LINUXTTY pattern). For these times, NONNEGINT can be used.
Existing users of POSINT might continue to expect zero to match, so
this change should probably be mentioned in the release notes (on the
other hand, some could be using POSINT without wanting it to match
zero, as happened to me).
Ref: Paragraph 3 of http://en.wikipedia.org/wiki/Natural_number
RFC 3986 (the URI specification) describes the , ; and =
characters used for including parameters in path segments.
Typically these are seen only on the final segment, just before
any query parameters, i.e.
http://www.site.com/path1/path2;jsessionid=OI24B9ASD7BSSD
Adding ; and = to the regex, as , is already included
Description :
Usual syslog message :
<85>Jun 14 15:19:47 localhost sudo: root : TTY=pts/1 ; PWD=/opt/logstash ; USER=root ; COMMAND=/bin/bash
Cisco typical message :
<166> Jun 14 15:30:00 10.100.252.52 %ASA-6-302021: Teardown ICMP connection for faddr 10.100.120.120/0 gaddr 10.100.252.1/0 laddr 10.100.252.1/0
----> program name start with a %
Can be reproduced sending a manual syslog message with python script :
import logging
from logging.handlers import SysLogHandler
#message='Jun 14 15:19:47 localhost sudo: root : TTY=pts/1 ; PWD=/opt/logstash ; USER=root ; COMMAND=/bin/bash'
message=' Jun 15 09:47:36 10.100.252.1 %ASA-6-111116: Teardown UDP connection 6201992 for internet:192.168.1.1/1026 to interne:10.100.120.120/427 duration 0:02:04 bytes 588'
logger = logging.getLogger()
logger.setLevel(logging.INFO)
syslog = SysLogHandler(address=('localhost',5544))
#syslog = SysLogHandler(address='/dev/log')
#formatter = logging.Formatter('%(name)s: %(levelname)s %(message)s')
#syslog.setFormatter(formatter)
logger.addHandler(syslog)
logger.warning(message)
Leading to a "NOT SYSLOG" message in the logs and no @fields{} values
With this change the fields are OK and "NOT SYSLOG" message is gone. I still have a "@tags":["_grokparsefailure"], error though...
In some cases, Onigiruma gets confused about negative matches, so
previously a pattern of '%{QS} something', if false match, would
cause Oniguruma to loop frantickly. I haven't yet dug into
the part of Oni that does this, but it's common that some regexp
engines have this behavior. Easy fix moving to non-backtracking
matches..
