RE: [logs] Syslog payload format

From: Ogle Ron (Rennes) (ron.ogleat_private)
Date: Tue Jan 14 2003 - 06:12:04 PST

  • Next message: Frank O'Dwyer: "RE: [logs] Syslog payload format"

    > -----Original Message-----
    > From: Bennett Todd [mailto:betat_private]
    > Sent: Monday, January 13, 2003 07:02 PM
    > To: Ogle Ron (Rennes)
    > Cc: 'Marcus J. Ranum'; Frank O'Dwyer; Rainer Gerhards;
    > loganalysisat_private; tcleary2at_private
    > Subject: Re: [logs] Syslog payload format
    > An interesting comment. Have you been following the prior
    > discussion, over the last six months or so, that has led us to this
    > point?
    Oh yes, way back when Tina started the original list and then moved it and
    then changed jobs.  Or was the job first and then the list change?
    > I'll try to sum it up.
    > The current practice of logging uncontrolled free-format strings
    > works to a point, and another design project on this list (SELP) is
    > trying to tweak RFC 3164 to the least extent possible to correct its
    > gross defects.
    > But there's one big downside to the current approach: it makes no
    > attempt to (a) precisely and unambiguously preserve as much
    > information as possible from the logging source, and (b) attempt to
    > offer developers a rich and comprehensive lexicon for classifying
    > log events.
    I know there are some problems with syslog (timestamp and udp), but you guys
    are throwing the baby out with the bath water, and I had to say something.
    There is nothing in the current syslog that prevents me from being precise
    or ambiguous.  I also understand on trying to formalize some higher level
    constructs, but the price is simplicity and ease of use.
    > These two defects are important and costly ways that current syslog,
    > mostly in the API, is inadequate; it's not capturing all the
    > knowlege that's available at the point of logging, and not all of
    > what it doesn't capture can be reconstructed robustly. It's lossy at
    > the logging API as well as the transport.
    No matter if it's the OS or the application, a developer has to write the
    calls to put the data out there.  If he/she isn't doing it with the current
    syslog, do you really believe he/she will do it when they have to look up
    all of this data to know which appropriate log event to throw out?  The
    lossiness is due to the fact that the developer didn't care about giving any
    more details.
    The point is that you guys are wasting a lot of time if the developer
    doesn't use it or the OS manufacturer doesn't incorporate it.  Logging like
    security is an after thought.  If you can't make it easy to use and low code
    and resource overhead, then they may decide to use it.  Again, syslog has
    been out there for how many years, and how many people (developers) use it
    to the best possible way?
    > After some discussion we came up with a convention for a tagged
    > transport format (suitable for carrying in the MSG of an RFC 3164
    > syslog, or SELP, or RFC 3195 syslog-in-BEEP, or ...); and we're
    > working out a concept for how a global lexicon of known tags and
    > perhaps in some cases values can be maintained over time, as it will
    > certainly have to grow.
    > There are still occasional rumbles of "tagged -vs- XML", from the
    > fragment you quoted perhaps that's not completely settled, but I
    > think there may be a majority of the long-term participants on this
    > thread advocating tagged in preference to XML, mostly on the grounds
    > that it's easier to specify a simple format, barely adequate to
    > express the semantics that we require, than it is to specify exactly
    > _what_ subset of XML is to be used --- and we do _Not_ want people
    > using full-powered XML parsers to grovel logged data, that's a
    > certain road to security problems. Don't provide strangers easy ways
    > to feed powerful parsers.
    > -Bennett
    This is the biggest beef.  Tagged or XML doesn't make any difference.
    You're bloating the data so that a network sniffer can easily read your
    logs.  This wastes a lot of resources on the client.  Get the data out of
    the client as efficiently as possible.  Like I said before, it would be
    amazing if Ethernet, IP, TCP, UDP, etc. would have survived you guys.  You
    can have all kinds of tools that can take this nice efficient log and show
    you pretty pictures later on the log server.
    With this tagging/xml, you are putting a layer of complexity that will kill
    it.  Every language will have their own tags to describe the same thing.
    It's an amazing sight to see that no matter which country a tool comes from
    that does network analysis of an IP packet, it can still show you IP
    addresses, ports, and flags.  Of course, my tool of choice always puts it in
    the language that I'm most comfortable with using.
    IMNSHO, these standards should ideally be started under the ISO instead of
    the IETF.  That way we know that they'll be dead on arrival.
    Ron Ogle
    Rennes, France
    LogAnalysis mailing list

    This archive was generated by hypermail 2b30 : Tue Jan 14 2003 - 08:25:02 PST