Re: [logs] Re: Generic Log Message Parsing Tool

From: Tom Perrine (tepat_private)
Date: Wed Jun 05 2002 - 13:07:04 PDT

  • Next message: Steffen Kluge: "Re: [logs] Re: Generic Log Message Parsing Tool"

    >>>>> On Wed, 5 Jun 2002 13:57:01 -0400, Sweth Chandramouli <loganalysisat_private> said:
    
        Sweth> On Wed, Jun 05, 2002 at 11:33:25AM -0400, yehuda wrote:
        >> How about having the parsed log output in XML?
        Sweth> 	Physical format is almost irrelevant; once we've got an
        Sweth> agreed-upon data structure syntax, people are free to write output
        Sweth> modules for XML, the relational or OO db of their choice, or whatever.
    
        Sweth> 	-- Sweth, whose gut response to XML suggestions is similar
        Sweth> to mjr's response to regex suggestions, because he's seen too many
        Sweth> people assume that XML==portable data when the real portability comes
        Sweth> from a well-defined data structure that the XML document can then
        Sweth> describe.
    
    I'll let a kitten out of the bag:
    
    We're building a next-gen clean-sheet design syslog.  Had our first
    message pass through a minimal system last night, so we're at version
    0.01.  Will do all the relevant current RFCs.  One thing we see is
    wanting to make message routing decisions based on log message
    content.  And the format for existing syslog messages is random, at
    best.
    
    We only want to parse the poorly-formatted, legacy messages once, into
    some canonical form.  Once they are canonicalized, we plan to forward
    them through all the relays, into the final sink, all in the canonical
    form.  The final sink formats we know we need to write are legacy
    syslog, "easily parsable", and various database backends.
    
    In other words, we know we will have to define our own canonical form
    for transmission/switching, and hope that an RFC will get others
    behind it.
    
    XML *is* a possibility, but I agree with Sweth on this one.  I've seen
    so many silver bullets for various problems over the years that I'm
    glad I'm not a were-person.  Remember "A.I.", and "data PBXes", and
    raft of other over-hyped, one-toy-fixes-all super-technologies?
    
    And I'd like to take advantage of all the stuff learned by the people
    who made some IPv6 header changes to support faster switching, too.
    WHy should every syslog relay have to re-parse the same
    pseudo-random-format messages?  Log formats may be one thing the
    Apache people did well.
    
    --tep
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    For additional commands, e-mail: loganalysis-helpat_private
    



    This archive was generated by hypermail 2b30 : Wed Jun 05 2002 - 13:22:03 PDT