RE: [logs] Syslog payload format

From: Frank O'Dwyer (fodat_private)
Date: Sun Dec 22 2002 - 10:20:52 PST

  • Next message: Frank O'Dwyer: "RE: [logs] Syslog payload format"

    Darren Reed wrote:
    > In some mail from Frank O'Dwyer, sie said:
    > [...]
    > > which, by the way, can also be expressed in pseudo-XML as:
    > >
    > >    <entry host="foo.bar.com" prog="trader" etc... />
    [...]
    > The above line is actually VALID XML, where host, prog, etc are
    > ATTRIBUTES of "entry".
    
    Well, indeed. I didn't come up with it by fluke. :) It's "pseudo" only in
    that it's a compatible subset of XML rather than allowing everything XML
    would allow.
    
    The point I was trying to make is that all the formats we have seen proposed
    for this (mjr tags, idsa, rfc3195-cooked, any others?) are fundamentally
    equivalent. You can easily express any of them in terms of the other, indeed
    you can trivially automate that translation with your favorite scripting
    language or with XSL style sheet processing.
    
    In particular all of these things are equivalent to rfc3195-cooked if you
    are allowed to switch or simply ignore its DTD:
    
    mjr-tags:
       <REC>
       <QUEUEID>foo</QUEUEID>
       <TO>mjrat_private</TO>
       <TO>lanceat_private</TO>
       <TO>infoat_private</TO>
       </REC>
    
    idsa:
        queueid="foo" to="mjrat_private,lanceat_private,infoat_private"
    
    pseudo-rfc3195-cooked:
        <entry queueid="foo"
    to="mjrat_private,lanceat_private,infoat_private"/>
    
    Similarly the suggested APIs are all fundamentally equivalent - you can
    implement any of them in terms of the other, and expose/hide any interface
    you like/dislike.
    
    Given that this is the case, the actual tags seem more interesting to me. It
    looks like it's not controversial that something like the following are
    mandatory and MUST be present:
      DATE, HOSTNAME, RAWMSG.
    
    Something like the following additional are more or less current practice
    and SHOULD be present:
      PROGRAM, PID, FACILITY, PRIORITY
    
    In my opinion, other candidates for SHOULD include the following:
      - a MSGID (as unique as possible and identifying a particular event
    instance, e.g. 1231312.200at_private)
    
      - an EVENTID (something as unique as possible that tells you what *kind*
    of event this is, this is tricky but e.g. "sendmail.2", "sendmail(v8.19).2",
    "ftpd(wu,v2.6.2)").
    
      - Or instead of the above you could break that down into components and
    let the analyzer try to build an EVENTID from any of these that are present:
    PROGRAM, EVENTNUM (at least), and PROGVER and/or VENDOR if necessary to
    disambiguate.
    
    MSGID lets you refer to an event instance and gives you a database key,
    EVENTID lets you filter and route events.
    
    Anything else MAY be present and is either drawn from a suggested shopping
    list such as mjr's logging map, or is up to the application.
    
    It also looks to me like all the values are strings (ASCII? UTF-8?) by
    default, with some few tags needing to be typed (e.g. DATE should be
    standard timestamp, PRI should be a number?).
    
    Does anyone violently disagree with any of that, or think it is way off
    course?
    
    Cheers,
    Frank
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Tue Dec 24 2002 - 01:36:13 PST