Actually, I'd rather people sent replies with sample data to the list. That way anyone who is working on any kind of parser can take advantage of it. On Sun, 19 Aug 2001, Ryan Russell wrote: > Date: Sun, 19 Aug 2001 23:59:23 -0600 (MDT) > From: Ryan Russell <ryanat_private> > To: loganalysisat_private > Subject: [loganalysis] SIDS 0.20 > > http://www.internettradecraft.com/sids/ > > OK, so I mentioned this last week, and said I'd have something over the > weekend... which would have been last weekend. Hey, a week late for > something I do in my spare time isn't too bad for me. :) > > Briefly, SIDS is an anomaly detection/log reduction tool. So, I figure > this would be a good mailing list for it. A present, what it does is go > through an HTTP log file (Roxen right now... see below) and pulls out the > less common entries. It does this in the most brain-dead way possible, by > counting. The general idea is that if something has been seen 100 times, > then it is probably "normal", and can be considered a candidate for no > longer alerting on. Over a large enough amount of logs, this eventually > ends up spitting out only the lines it hasn't seen before. > > Please understand that at present, SIDS is incredibly crude, hence the .2 > version number. Probably the main things that keeps it from being useful > at the moment is the output format, and the lack of ability to track > "safe" items between session. Those will both be addressed in the next > couple revs. > > What I would like from the subscribers of this list is some help with log > file formats. What I'm looking for is samples from different HTTP log > files, so that I can make compatible filters for them, so that less > technical people who want to use it will hopefully not have to write their > own. So, if you're interested, send me a few lines of your web logs > (sanitized is fine, doesn't matter), and I'll write the appropriate config > file for your web server. Please tell me what web server you're using > too, so I can name it appropriately. Then, I can sent you the config > file, you can try it on your full logs, and then tell me how useless SIDS > is at present. :) (Actually, what that will do for you is have your log > file format ready to go when SIDS is actually useful.) > > For the future, I'm looking to extend SIDS beyond HTTP logs, so I will > also want to look at other log types. Anything that has a very regular > single-line format is an immediate candidate. Others I will look at, so I > can get ideas on how to handle them. This plays directly into the recent > discussion here about how to eliminate unintersting log entries with > swatch, and similar ideas. > > I would tend to think that Tina would want the replies off-list. I can't > think what good the samples would do anyone else, but that is ultimately > up to the moderator, of course. > > Oh.. and it's open-source, etc... haven't gotten around to picking a > license yet.. but it is hardly important (take a look at how short the > code is.) > > Ryan > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: loganalysis-unsubscribeat_private > For additional commands, e-mail: loganalysis-helpat_private > VPN: http://kubarb.phsx.ukans.edu/~tbird/vpn.html life: http://kubarb.phsx.ukans.edu/~tbird work: http://www.counterpane.com --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Mon Aug 20 2001 - 10:15:30 PDT