RE: [logs] Syslog payload format

fodat_private

Darren Reed wrote:
> In some mail from Frank O'Dwyer, sie said:
> [...]
> > which, by the way, can also be expressed in pseudo-XML as:
> >
> >    <entry host="foo.bar.com" prog="trader" etc... />
[...]
> The above line is actually VALID XML, where host, prog, etc are
> ATTRIBUTES of "entry".

Well, indeed. I didn't come up with it by fluke. :) It's "pseudo" only in
that it's a compatible subset of XML rather than allowing everything XML
would allow.

The point I was trying to make is that all the formats we have seen proposed
for this (mjr tags, idsa, rfc3195-cooked, any others?) are fundamentally
equivalent. You can easily express any of them in terms of the other, indeed
you can trivially automate that translation with your favorite scripting
language or with XSL style sheet processing.

In particular all of these things are equivalent to rfc3195-cooked if you
are allowed to switch or simply ignore its DTD:

mjr-tags:
   <REC>
   <QUEUEID>foo</QUEUEID>
   <TO>mjrat_private</TO>
   <TO>lanceat_private</TO>
   <TO>infoat_private</TO>
   </REC>

idsa:
    queueid="foo" to="mjrat_private,lanceat_private,infoat_private"

pseudo-rfc3195-cooked:
    <entry queueid="foo"
to="mjrat_private,lanceat_private,infoat_private"/>

Similarly the suggested APIs are all fundamentally equivalent - you can
implement any of them in terms of the other, and expose/hide any interface
you like/dislike.

Given that this is the case, the actual tags seem more interesting to me. It
looks like it's not controversial that something like the following are
mandatory and MUST be present:
  DATE, HOSTNAME, RAWMSG.

Something like the following additional are more or less current practice
and SHOULD be present:
  PROGRAM, PID, FACILITY, PRIORITY

In my opinion, other candidates for SHOULD include the following:
  - a MSGID (as unique as possible and identifying a particular event
instance, e.g. 1231312.200at_private)

  - an EVENTID (something as unique as possible that tells you what *kind*
of event this is, this is tricky but e.g. "sendmail.2", "sendmail(v8.19).2",
"ftpd(wu,v2.6.2)").

  - Or instead of the above you could break that down into components and
let the analyzer try to build an EVENTID from any of these that are present:
PROGRAM, EVENTNUM (at least), and PROGVER and/or VENDOR if necessary to
disambiguate.

MSGID lets you refer to an event instance and gives you a database key,
EVENTID lets you filter and route events.

Anything else MAY be present and is either drawn from a suggested shopping
list such as mjr's logging map, or is up to the application.

It also looks to me like all the values are strings (ASCII? UTF-8?) by
default, with some few tags needing to be typed (e.g. DATE should be
standard timestamp, PRI should be a number?).

Does anyone violently disagree with any of that, or think it is way off
course?

Cheers,
Frank

_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis