Re: [logs] Re: Generic Log Message Parsing Tool

From: Sweth Chandramouli (loganalysisat_private)
Date: Tue Jun 04 2002 - 18:03:49 PDT

  • Next message: Marcus J. Ranum: "Re: [logs] Re: Generic Log Message Parsing Tool"

    On Wed, Jun 05, 2002 at 11:50:58AM +1200, Russell Fulton wrote:
    > Hmmm... One way to do this in perl is to use an OOP approach where we
    > create a Logs::Parse module with all the generic fuctionality we need
    > (interfaces to databases, sorting, filtering and a generic data
    > structures).  We then have a bunch of specific modules
    > Logs::Parse::Syslog, Logs::Parse::Pix, Logs::Parse::FW1, etc. which
    > inherit all the generic methods and data structures from the parent
    > module and define the actual parsing rules in native perl.  This is not
    > such an elegant solution as Parse::Recurse but should run much faster.
    > 
    > Hmmmm... a more general solution would be to have a heirarchy rooted at
    > Logs::ZLogs (for want of a better name).
    > 
    > Logs:ZLogs.pm  (generic data structures)
    >   Logs:ZLogs::Parse.pm (generic parsing module etc.)
    >      Logs:ZLogs::Parse.Syslog.pm (handle input from file, tcp and 
    >                                    udp streams)
    	That kind of hierarchy is my eventual goal, but I think
    an initial step would be just getting down the syslog parsing portion;
    thanks to the profusion of implementations out there, there are literally
    hundreds of possible formats to deal with just to parse a syslog message.
    Work could proceed in parallel on the different types of parsers, of
    course, and some of the parsers ("application-level" parsers) would have
    to be able to accept "parsed message" objects from other parsers
    ("infrastructure-level" parsers, like the syslog parser) as their input,
    to deal with, for example, a webserver that has been configured to send
    its logs to syslog.
    
    	Any help in terms of student labor that you could provide
    would always be welcome, of course.  :)
    
    	-- Sweth.
    
    -- 
    Sweth Chandramouli      Idiopathic Systems Consulting
    svcat_private      http://www.idiopathic.net/
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    For additional commands, e-mail: loganalysis-helpat_private
    



    This archive was generated by hypermail 2b30 : Tue Jun 04 2002 - 19:48:55 PDT