We need to have all these contributions online in the SecurityFocus Server. Is that possible? Todd ----- Original Message ----- From: "Ryan Russell" <ryanat_private> To: <loganalysisat_private> Sent: Sunday, August 19, 2001 10:59 PM Subject: [loganalysis] SIDS 0.20 > http://www.internettradecraft.com/sids/ > > OK, so I mentioned this last week, and said I'd have something over the > weekend... which would have been last weekend. Hey, a week late for > something I do in my spare time isn't too bad for me. :) > > Briefly, SIDS is an anomaly detection/log reduction tool. So, I figure > this would be a good mailing list for it. A present, what it does is go > through an HTTP log file (Roxen right now... see below) and pulls out the > less common entries. It does this in the most brain-dead way possible, by > counting. The general idea is that if something has been seen 100 times, > then it is probably "normal", and can be considered a candidate for no > longer alerting on. Over a large enough amount of logs, this eventually > ends up spitting out only the lines it hasn't seen before. > > Please understand that at present, SIDS is incredibly crude, hence the .2 > version number. Probably the main things that keeps it from being useful > at the moment is the output format, and the lack of ability to track > "safe" items between session. Those will both be addressed in the next > couple revs. > > What I would like from the subscribers of this list is some help with log > file formats. What I'm looking for is samples from different HTTP log > files, so that I can make compatible filters for them, so that less > technical people who want to use it will hopefully not have to write their > own. So, if you're interested, send me a few lines of your web logs > (sanitized is fine, doesn't matter), and I'll write the appropriate config > file for your web server. Please tell me what web server you're using > too, so I can name it appropriately. Then, I can sent you the config > file, you can try it on your full logs, and then tell me how useless SIDS > is at present. :) (Actually, what that will do for you is have your log > file format ready to go when SIDS is actually useful.) > > For the future, I'm looking to extend SIDS beyond HTTP logs, so I will > also want to look at other log types. Anything that has a very regular > single-line format is an immediate candidate. Others I will look at, so I > can get ideas on how to handle them. This plays directly into the recent > discussion here about how to eliminate unintersting log entries with > swatch, and similar ideas. > > I would tend to think that Tina would want the replies off-list. I can't > think what good the samples would do anyone else, but that is ultimately > up to the moderator, of course. > > Oh.. and it's open-source, etc... haven't gotten around to picking a > license yet.. but it is hardly important (take a look at how short the > code is.) > > Ryan > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: loganalysis-unsubscribeat_private > For additional commands, e-mail: loganalysis-helpat_private > --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Mon Aug 20 2001 - 10:14:22 PDT