RE: [logs] Syslog payload format

rgerhardsat_private

Bennett,

Reading your mail, I looks like the recent syslog RFC's should never
have hit the IETF - no running code an the like...

I thought a lot about your post. To me, it sound a little bit like give
up on the payload, there will never be a standard... Wouldn't it be an
option for at least some implementors to define something they can work
with and do that? Is it really so hopeless?

Regarding the taxonomy, I see your point. It is just that I think it
should not be a closed set, but dynamically extensible. Yes, of course,
that makes it hard for existing analysers to work with new sources but
wouldn't that be better than to restrict a not-yet-existing new kind of
device or software to a not matching taxonomy? Also, is it that bad an
idea to start with something that is already widely accepted like WELF
(except, of course there are some IPR issues....)?

Rainer

> -----Original Message-----
> From: Bennett Todd [mailto:betat_private] 
> Sent: Tuesday, December 17, 2002 7:00 PM
> To: Rainer Gerhards
> Cc: loganalysisat_private
> Subject: Re: [logs] Syslog payload format
> 
> 
> 2002-12-17-05:42:55 Rainer Gerhards:
> > I would like to focus on the payload with this
> > posting. Interestingly, there never has been any standard for the 
> > exact format and also interestingly nobody at the IETF 
> seems to care 
> > (right now) about it.
> 
> The impression I've gotten is that any time discussion of 
> payload starts up, it stalls out in looping arguments --- 
> everyone keeps saying the same thing, over and over again.
> 
> For myself, I'm keen to see the idiot syslog timestamp format:
> 
> 	Dec 17 12:33:54
> 
> replaced with ISO 8601 / RFC 3339:
> 
> 	2002-12-17T12:33:54-0500
> 
> Beyond that, the next two whitespace-delimited fields are 
> pretty widely agreed on; the second is the hostname, and the 
> third is program[pid]:. After that comes free text.
> 
> I've yet to see a proposal for a tagged format that struck me 
> as convincing. If we could work out really valuable semantics 
> for such a format --- a good convincing taxonomy of loggable 
> events --- then maybe it'd motivate going to the trouble of 
> introducing a tagged format. Until we see one, though, I'll 
> stick with the current format, fixing only the broken 
> timestamp format, and of course tacking the unreliable and 
> restrictive transport.
> 
> Back to the payload, the way to guide the transition is to 
> first define your taxonomy.  Then write a converter that 
> reads current syslog data, together with a rules file 
> describing known patterns and their associated classification 
> tags. Extend this rulebase to cover an interesting range of 
> real log types. Build tools that work with it. Demonstrate 
> their value. You'll need that converter almost indefinitely 
> anyway, until the very last embedded-OS gizmo gets converted 
> to the New Way Of Logging.
> 
> Once you've demonstrated the real utility of the taxonomy, 
> then create a New Logging over-the-wire protocol, a client 
> API, and a companion server. Remember you'll need to retain 
> some kind of backwards-compatibility interop with Syslog 
> Classic more or less of forever.
> 
> Contribute code to some prominent free software that does 
> important logging (e.g. the main MTAs, packet filters and 
> firewall proxies, web servers, security components like sudo, 
> ...) adding compile-time-optional logging using the New 
> library API. Build a varient of your favourite open source OS 
> distribution in which every invocation of syslog(3) is 
> replaced with good invocations including appropriate 
> classification info in the new logging API. Get people to do 
> this for all major open source OS distributions. Let people 
> build experience that the new way of doing things is really 
> convincingly better, over a wide range of scales. Present how 
> much better it is, with real big interesting case examples, 
> at a Usenix. When you've gotten a big enough popular base 
> that the complete kit is shipped with the major distributions 
> of popular OSes, together with apps built to work with it, 
> then go beating on IETF's door:-).
> 
> -Bennett
> 
_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis