On Fri, Aug 23, 2002 at 04:29:08PM -0700, Kyle R. Hofmann wrote:
> On Fri, 23 Aug 2002 00:29:22 -0700, Chris Adams wrote:
[ snip ]
>
> The first interface is between the program and /dev/log. This is a library
> call. Since we're proposing to change the standard for log messages to
> something, either this will have to be replaced with a call that provides
> more structure, or we will have to rely on programmers to always use the
> right structure all the time. As I see it, our options are:
>
> 0) Leave syslog(3) unchanged and give up.
>
> 1) Leave syslog(3) unchanged and add a reformatter between /dev/log and
> syslogd.
(1) is a loosing idea. Currently the CONTENT field of a syslog message
has most of the interesting information in it. The CONTENT field is
also free form text, which is just too difficult to parse. And without
understanding the what is in the CONTENT field the other fields don't
really help all that much. It does no good to know that ftpd on host
foo logged a message at 13:00, which is all that is available without
looking at the CONTENT field.
>
> 2) Leave syslog(3) unchanged and trust the programmer to use only "new-style"
> log messages.
>
> 3) Replace syslog(3), but do not significantly change the interface; trust the
> programmer to use only "new-style" log messages on the new library call.
(2) and (3) probably won't do much good either.
>
> 4) Replace syslog(3) with
>
> int newsyslog(struct logmsg *logmsg, void *logdata);
>
> where struct logmsg has fields like facility, priority, and a type for
> logdata. logmsg->type would determine what sort of struct * to cast
> logdata to. This ensures that the data will contain exactly the structures
> that we want it to, but is inextensible: Adding a new format requires
> recompiling the syslog daemon and has portability problems.
>
> 5) Replace syslog(3) with
>
> int newsyslog(unsigned int logfieldcount, struct field *logfields, unsigned int varfieldcount, struct field *varfields);
>
> where logfields and varfields are arrays of structures and logfieldcount
> and varfield count determine the number of fields in logfields and
> varfields. struct field would be
>
> struct field {
> unsigned char *name;
> unsigned char *value;
> unsigned int name_length;
> unsigned int value_length;
> };
>
> This does not ensure that we have all the data we need or want, but is
> very extensible. (Note that doing without struct field is a bad idea. We
> can't safely assume that there will never be any \0's in name and value.)
>
> 6) Replace syslog(3) with
>
> int newsyslog(xml_t xml_logmsg);
>
> where xml_logmsg is some representation of a log message as XML. This
> requires developing a set of XML library routines to manipulate log
> messages and using them in every program that wants to log.
>
(5) or (6) is probably the best answer, but as Chris pointed out the
problem will be convincing programmers to use the new format. Also it
doesn't help all that much unless we have some standard fields and know
what the field mean. I suggest that tools should be written that will
automatically generate some helper functions from a set of field descriptions.
This will make it eaiser to use the logging functions, and it means that
field descriptions have been written, which can be published along with
the software.
[snip]
> The sixth interface is between syslogd and a remote syslog host. If syslogd
> speaks the new format, and that format is unintelligible to the remote host,
> our logs might get dropped on the floor. Thus syslogd either requires one
> of:
>
> 0) Incompatibility,
> 1) Careful configuration and the ability to speak a little bit of the old
> format,
> 2) A new format which is backwards compatible with the old, or
> 3) A new format which is always sent and received over the network in a
> format which is backwards compatible with the old, but which is different
> the rest of the time.
>
> (2) feels like it might be needlessly limiting, and (3) is ugly. I like (1).
(1) is a good option for logging to a remote host that only speaks UDP.
However, a trivial (and backwards compatible) extension to
syslog-reliable (RFC 3195) is possbile. Currently syslog-reliable wraps
the syslog message with
<entry facility='' priority='' hostname='' time='' ...>CONTENT</entry>
And CONTENT is just free form text. CONTENT could be replaced with an
xml message that contains the new information we want. i.e.
<entry facility='' ...>
<name>value</name>
...
or whatever the format ends up looking like.
</entry>
A log host which doesn't speak the extension would just log everything
between <entry></entry> as literal text.
Between the <entry></entry> the new format almost has to be XML because
that way it will be distinguishable from a normal syslog-reliable
message. Also an XML parser is already needed.
--
Devin Kowatch
devinkat_private
_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis
This archive was generated by hypermail 2b30 : Mon Aug 26 2002 - 13:58:19 PDT