[logs] a tool for detecting patterns from logfiles

Previous message: Rainer Gerhards: "[logs] Configuring devices to emit syslog"
Next in thread: Pete Finnigan: "[logs] 65 Oracle security papers, articles and presentations"
Reply: Pete Finnigan: "[logs] 65 Oracle security papers, articles and presentations"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

risto.vaarandiat_private

hi all,

I have created an experimental tool for mining patterns from logfiles. I 
have been using it for a month now and it has worked quite well in my 
environment, so I decided to put it on the web.

The tool uses clustering algorithm to divide the logfile into clusters, 
so that each cluster corresponds to a certain pattern that occurs 
frequently enough. Lines that do not belong to any detected clusters (so 
called outliers) are written to a separate file. This allows one to 
detect rare lines that are possibly anomalous. Also, one can run the 
analysis process iteratively by reapplying the tool for the outliers 
file for several times.

The tool is available at http://kodu.neti.ee/~risto/slct/, and any 
feedback is welcome.

br,
risto

_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis