RE: [logs] Syslog payload format

ron.ogleat_private

> -----Original Message-----
> From: Bennett Todd [mailto:betat_private]
> Sent: Monday, January 13, 2003 07:02 PM
> To: Ogle Ron (Rennes)
> Cc: 'Marcus J. Ranum'; Frank O'Dwyer; Rainer Gerhards;
> loganalysisat_private; tcleary2at_private
> Subject: Re: [logs] Syslog payload format
> 
> 
> An interesting comment. Have you been following the prior
> discussion, over the last six months or so, that has led us to this
> point?

Oh yes, way back when Tina started the original list and then moved it and
then changed jobs.  Or was the job first and then the list change?

> 
> I'll try to sum it up.
> 
> The current practice of logging uncontrolled free-format strings
> works to a point, and another design project on this list (SELP) is
> trying to tweak RFC 3164 to the least extent possible to correct its
> gross defects.
> 
> But there's one big downside to the current approach: it makes no
> attempt to (a) precisely and unambiguously preserve as much
> information as possible from the logging source, and (b) attempt to
> offer developers a rich and comprehensive lexicon for classifying
> log events.

I know there are some problems with syslog (timestamp and udp), but you guys
are throwing the baby out with the bath water, and I had to say something.
There is nothing in the current syslog that prevents me from being precise
or ambiguous.  I also understand on trying to formalize some higher level
constructs, but the price is simplicity and ease of use.

> 
> These two defects are important and costly ways that current syslog,
> mostly in the API, is inadequate; it's not capturing all the
> knowlege that's available at the point of logging, and not all of
> what it doesn't capture can be reconstructed robustly. It's lossy at
> the logging API as well as the transport.

No matter if it's the OS or the application, a developer has to write the
calls to put the data out there.  If he/she isn't doing it with the current
syslog, do you really believe he/she will do it when they have to look up
all of this data to know which appropriate log event to throw out?  The
lossiness is due to the fact that the developer didn't care about giving any
more details.

The point is that you guys are wasting a lot of time if the developer
doesn't use it or the OS manufacturer doesn't incorporate it.  Logging like
security is an after thought.  If you can't make it easy to use and low code
and resource overhead, then they may decide to use it.  Again, syslog has
been out there for how many years, and how many people (developers) use it
to the best possible way?

> 
> After some discussion we came up with a convention for a tagged
> transport format (suitable for carrying in the MSG of an RFC 3164
> syslog, or SELP, or RFC 3195 syslog-in-BEEP, or ...); and we're
> working out a concept for how a global lexicon of known tags and
> perhaps in some cases values can be maintained over time, as it will
> certainly have to grow.
> 
> There are still occasional rumbles of "tagged -vs- XML", from the
> fragment you quoted perhaps that's not completely settled, but I
> think there may be a majority of the long-term participants on this
> thread advocating tagged in preference to XML, mostly on the grounds
> that it's easier to specify a simple format, barely adequate to
> express the semantics that we require, than it is to specify exactly
> _what_ subset of XML is to be used --- and we do _Not_ want people
> using full-powered XML parsers to grovel logged data, that's a
> certain road to security problems. Don't provide strangers easy ways
> to feed powerful parsers.
> 
> -Bennett

This is the biggest beef.  Tagged or XML doesn't make any difference.
You're bloating the data so that a network sniffer can easily read your
logs.  This wastes a lot of resources on the client.  Get the data out of
the client as efficiently as possible.  Like I said before, it would be
amazing if Ethernet, IP, TCP, UDP, etc. would have survived you guys.  You
can have all kinds of tools that can take this nice efficient log and show
you pretty pictures later on the log server.

With this tagging/xml, you are putting a layer of complexity that will kill
it.  Every language will have their own tags to describe the same thing.
It's an amazing sight to see that no matter which country a tool comes from
that does network analysis of an IP packet, it can still show you IP
addresses, ports, and flags.  Of course, my tool of choice always puts it in
the language that I'm most comfortable with using.

IMNSHO, these standards should ideally be started under the ISO instead of
the IETF.  That way we know that they'll be dead on arrival.

Ron Ogle
Rennes, France
_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis