Re: [logs] Syslog payload format

bazsiat_private

On Tue, Dec 31, 2002 at 02:28:14PM +1100, Darren Reed wrote:
> In some mail from Balazs Scheidler, sie said:
> > 
> > Hi,
> [...]
> > By using the ';' as a separator in the format string. No macros would be
> > allowed before the ';'.
> 
> Why not ?  And why ';' ?
> Are you saying ';' cannot be used in a log message, now ?
> 
> > Maybe this separation could be strictened by using
> > two separate arguments:
> > 
> > xnewsyslog(LOG_DAEMON | LOG_INFO, 
> >            "User logged in", 
> >            "%(user)s %(tty)s %(host)s",
> >            "marcus", "ttyp6", host);
> 
> So how do I include random bits of printf-like formatting in the
> event message ?  It looks like I should be able to!
> 
> > My original intention was to clearly mark the separation between human
> > readable description and variable part. In my original suggestion this was
> > the ';' though it may not have been emphasized enough.
> 
> It wasn't.

Sorry for that. Using the ';' may have been a bad choice, that's why we are
discussing right now. See my following function definition where the
constant part and the tags part were separated.

> 
> > > > 1) provide a clean API for sending tagged messages
> > > > 2) provide a not-so-clean but easier to use interface based on the first
> > > 
> > > I think you need a (3) as well:
> > > 
> > > 3) provide a replacement for the current syslog(3) API that produces
> > >    tagged messages.
> > 
> > I think 2) and 3) is the same. A completely syslog(3) compatible function
> > could not generate tags, as there is no tagging in its arguments.
> 
> No, they're not because that's not the point of doing (3).
> 
> The point of doing (3) is to get syslog(3) to produce compatible output
> with the desired formatting.  FWIW, there are implicit tags - pid,
> hostname, program name, time - that can be marked up.
> 
> Doing (3) gets you a minimal update to a marked up output without
> having to change *ANY* applications, today.

ok. 

> Sorry, that doesn't let me log raw (binary) data structures.
> 
> For that matter, using %(blah)foo just makes processing harder, not
> easier and definately not "obvious".
> 
> If I see "%(user)s %(host)s" in a string somewhere, I would expect it
> to be output somewhere as a whole, not in parts.  Since you're going
> down vararg lane (or stdargs), you may as well use proper value pairing,
> such as was suggested earlier - log(..., USER|STRING, username, ...)

Hmm... Is USER and STRING above a numeric constant, that what the operator
'|' makes me believe so. We have to solve two problems:

1) identifying the tag
2) create a handy interface for the programmer to format values

I mixed these two in my previous posts, using tag names and sprintf like
formatting in a single argument:

xnewsyslog(..., "User logged in", "%(srcuser)s %(srcdev)s", "marcus", "ttyp0")

(this time using Marcus's tag names as described in
 http://www.ranum.com/logging/logging-data-map.html)

Maybe formatting and tag names should be separated:

xnewsyslog(..., "User logged in", 
           "srcuser", "%s", "marcus", 
           "srcdev", "%s", "ttyp0")

(of course the tag names "srcuser" and "srcdev" could be specified as macros)

The problem here is that you don't know where the arguments of the first
format string ends. So creating a function like above is not possible
portably (unless you create your own sprintf-like function)

Something like this would be possible though:

xnewsyslog(..., "User logged in",
	   xnewsyslogtag("srcuser", "%s", "marcus"),
	   xnewsyslogtag("srcdev", "%s", "ttyp0"),
           ...,
           NULL);

where xnewsyslogtag() would return either an allocated string (formatted
like an XML attribute value) or a structure describing the tag/value, and
xnewsyslog would take care about freeing its arguments. This would result in
a lot of small allocated memory chunks though, which might not be the best
idea to do.

Of course the whole idea applies only if we agree on the basic terms, e.g:
"event type + tag/value pairs", and event type is a string by itself without
variable parts.

-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis