RE: [logs] Syslog payload format

From: Frank O'Dwyer (fodat_private)
Date: Sun Dec 22 2002 - 16:53:45 PST

  • Next message: Darren Reed: "Re: [logs] Year value in timestamps"

    Mikael Olsson wrote:
    [...]
    > Suffice to say that pretty
    > much any unit emitting log events will have to go beyond some
    > small set of "standard" events, which again means that any
    > form of automated log parser must know about these names.
    
    There is no getting away from the fact that if you want to do anything
    intelligent with the events produced by an app or a unit, then you need some
    understanding of what the app or unit does, the kinds of events it may
    produce, and what each one means.
    
    But currently you cannot even automatically look at a log entry and tell
    what *kind* of event it is, without a great deal of bailing wire and chewing
    gum. In some cases, even that is not enough, you practically need AI. So too
    bad if the parser otherwise has the requisite understanding, it's sometimes
    screwed before it can even identify what processor to hand the event off to.
    
    Not only that, but even with some of the better formats out there you need
    to alter the parser anytime there is a new rev of the unit or program you
    are looking at. This gets very ugly very fast. Sometimes it is not even easy
    to know which unit or program logged the event in the first place, never
    mind which version of it. How many different variants of ftp will identify
    themselves as 'ftpd', yet produce a totally different set of events, for
    example?
    
    Multiply these problems across all the interesting devices and applications
    out there, and trying to keep 50 corks underwater in a bathtub would be easy
    by comparison.
    
    [...]
    > So, um, anyway, to sum up my ramblings above: what is the goal of
    > standardizing syslog payloads?  I can certainly understand why
    > anyone doing log analysis would want applications to emit some sort
    > of sane output that is actually machine parseable rather than just
    > random written english statements, but I fail to see the point in
    > enforcing the same set of rules to things as disparate as /bin/login
    > and a router (?)
    
    I don't think that "one size fits all" is the intention really. It's just a
    matter of getting the basic building blocks in place so that any structure
    in an event source is maintained end-to-end, giving a parser a foothold in
    interpreting the event. This doesn't remove the need to tailor parsers for
    any source whose events they need to handle in an intelligent and context
    dependent manner, it just enables doing that.
    
    Cheers,
    Frank.
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Tue Dec 24 2002 - 09:09:59 PST