RE: [logs] Re: Generic Log Message Parsing Tool

From: yehuda (yehudaat_private)
Date: Wed Jun 05 2002 - 08:33:25 PDT

  • Next message: Tina Bird: "Re: [logs] Re: Generic Log Message Parsing Tool"

    How about having the parsed log output in XML?
    
    > -----Original Message-----
    > From:	Sweth Chandramouli [SMTP:loganalysisat_private]
    > Sent:	Tuesday, June 04, 2002 9:04 PM
    > To:	loganalysisat_private
    > Subject:	Re: [logs] Re: Generic Log Message Parsing Tool
    > 
    > On Wed, Jun 05, 2002 at 11:50:58AM +1200, Russell Fulton wrote:
    > > Hmmm... One way to do this in perl is to use an OOP approach where we
    > > create a Logs::Parse module with all the generic fuctionality we need
    > > (interfaces to databases, sorting, filtering and a generic data
    > > structures).  We then have a bunch of specific modules
    > > Logs::Parse::Syslog, Logs::Parse::Pix, Logs::Parse::FW1, etc. which
    > > inherit all the generic methods and data structures from the parent
    > > module and define the actual parsing rules in native perl.  This is not
    > > such an elegant solution as Parse::Recurse but should run much faster.
    > > 
    > > Hmmmm... a more general solution would be to have a heirarchy rooted at
    > > Logs::ZLogs (for want of a better name).
    > > 
    > > Logs:ZLogs.pm  (generic data structures)
    > >   Logs:ZLogs::Parse.pm (generic parsing module etc.)
    > >      Logs:ZLogs::Parse.Syslog.pm (handle input from file, tcp and 
    > >                                    udp streams)
    > 	That kind of hierarchy is my eventual goal, but I think
    > an initial step would be just getting down the syslog parsing portion;
    > thanks to the profusion of implementations out there, there are literally
    > hundreds of possible formats to deal with just to parse a syslog message.
    > Work could proceed in parallel on the different types of parsers, of
    > course, and some of the parsers ("application-level" parsers) would have
    > to be able to accept "parsed message" objects from other parsers
    > ("infrastructure-level" parsers, like the syslog parser) as their input,
    > to deal with, for example, a webserver that has been configured to send
    > its logs to syslog.
    > 
    > 	Any help in terms of student labor that you could provide
    > would always be welcome, of course.  :)
    > 
    > 	-- Sweth.
    > 
    > -- 
    > Sweth Chandramouli      Idiopathic Systems Consulting
    > svcat_private      http://www.idiopathic.net/
    > 
    > ---------------------------------------------------------------------
    > To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    > For additional commands, e-mail: loganalysis-helpat_private
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    For additional commands, e-mail: loganalysis-helpat_private
    



    This archive was generated by hypermail 2b30 : Wed Jun 05 2002 - 10:57:12 PDT