RE: [logs] Syslog payload format

mjrat_private

Frank O'Dwyer wrote:
>  log(event);
>
>Where 'event' is an event object, or something like a 'struct' if you're a
>'C' programmer.

Well, what we do now is bad and wrong - typical calls to
syslog() look like:

{
        char    buf[BUFSIZ];

        /* never mind overflows, this is an EXAMPLE... */
        sprintf(buf,"%s - file %s size %d",host,file,size);
        syslog(LOG_NOTICE,buf);
}

Well, that's just EVIL because here we already have it all
parsed out already and we're unparsing it into a string, then
handing that to syslog and EVENTUALLY someone tries to parse
the results out of syslog. Can you think of anything stupider
except using ice made out of distilled water to cool your tea?

So what I think we don't do is store it in a structure; it
needs to be open-ended.

RULE #1:
        - An event record is an arbitrary collection of tagged values
RULE #2:
        - Some tags are "known" public tags, some tags are application,
        site, or event specific - cannot know them all in advance
        therefore there must be some flexibility in letting app
        designers pick their own
RULE #3:
        - the API for tossing event records should be EASIER to use than
        the current syslog API and should inherently discourage
        implementation of buffer overruns
RULE #4:
        - defaults should be used wherever possible

so how about an event logging API that vaguely resembles
something like:

#include <eventlog.h>

eventlog_open("my program");
/* note this sets up a few defaults which may be
overridden by eventlog_setdefault() calls. i.e.:
EVENTLOG_PID is set to the results of getpid()
automatically */

eventlog_setdefault(EVENTLOG_PROTO,"ftp");
/* EVENTLOG_PROTO is a #define from eventlog.h and is the tag
for event log facility - i.e.: "PROTO"
 From now on unless an event record SPECIFIES a facility
it'll get assigned to whatever we defaulted it to "ftp"
because we're doing FTP stuff... */

EventRec        *newev;

newev = eventlog_new();
eventlog_addvalue(newev,EVENTLOG_PATH,pathname);
eventlog_addvalue(newev,EVENTLOG_PRIO,4);
eventlog_send(newev); /* note time gets added when SENT */
eventlog_free(newev);

Also have some basic routines that work on a statically
allocated default event record...
eventlog_clearstd();
eventlog_addstd(EVENTLOG_PATH,pathname);
eventlog_sendstd();

Note that this way you _could_ allocate an event record
and if you were logging (say) URLs you could just go in a
while(1) {
        eventlog_addvalue(newev,EVENTLOG_PATH,url);
        eventlog_send(newev);
}

The stuff that gets burped out the back would simply be
an representation of the tags with meta-character translation
applied. I.e.:
<REC>
<PATH>pathname</PATH>
<PRIO>4</PRIO>
<PROTO>ftp</PROTO>
<DATE>standard date</DATE>
</REC>

This is easy. We're looking at, what, a couple hours of coding?
Toss in some options that let you do useful things like tell the
subsystem to generate unique event-IDs automatically, etc, etc.,
and you're in good shape.

>So the first question is what does an event consist of? Things like
>generation time, event id, priority, source, human readable message,
>forwarding trail, maybe some application-specific payload, what else? It's
>easy to come up with a long shopping list, but what are the basics that
>people could agree on, and which are optional and which are mandatory?

You want to make the number of options small and the core stuff
should ALL be defaulted if not provided.

date/timestamp (standard)

>Once you know the content of the object/struct, you can then worry about
>getting it from A to B, and safely tucked into some log or other.

Yah, that's easy. :)

>[As an aside, anything you come up with will likely have an obvious XML
>equivalent, but that doesn't mean XML needs to be the on-the-wire format
>(although that or an XML subset is an obvious choice to consider).]

A subset of XML is so easy (see above) it's not even funny.
Where XML gets silly is when you get into complex nested
stuff and DTDs and all that houghmagandy.

mjr. 
---
Marcus J. Ranum				http://www.ranum.com
Computer and Communications Security	mjrat_private

_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis