RE: [logs] Syslog payload format

From: Rainer Gerhards (rgerhardsat_private)
Date: Fri Jan 03 2003 - 07:49:49 PST

  • Next message: Ed Schmollinger: "Re: [logs] SWATCH configuration"

    > It exists in my brain only, but I'm thinking about a 
    > multi-layer API, where the lowest level would be something 
    > like your IDSA API (or Marcus's
    > eventlog_* suggestion).
    > 
    > I am thinking along the following lines:
    > 
    > 1) low level API, something resembling to Marcus's and your API
    > 2) a convinience API with format strings and varargs, calls 
    > 1) functions
    > 3) a syslog compatible wrapper (probably in an 
    > LD_PRELOAD-able shared object)
    > 
    > As an event record is created it is processed by two 
    > orthogonal modules:
    > 
    > 1) formatting, format the description+tag/value pairs into a message,
    >    possible to inject into syslog. The method used to map a 
    > message is user
    >    selectable:
    >    - a simple non-XML format
    >    - a couple of different XML format we were discussing 
    > (syslog tags as
    >      attributes, syslog tags as XML tags, etc.)
    > 
    >    These formatting options would be linked into the library, 
    > to extend the
    >    set of supported mappings one would need to add the code 
    > and recompile the
    >    library. I don't want to introduce dynamic loading here 
    > for simplicity.
    
    Using static linking is a good idea...
    
    > 
    > 2) output, send the formatted message to 
    >    - local syslogd
    >    - RFC3164 via UDP
    >    - RFC3164 via TCP
    
    Hey, are you anticipating a standard on 3164 over TCP ;-). Well, I'd
    agree to this as long as there can be consensus reached on how to do
    this. Andrew's suggestion might be a good thing to go... Honestly, I
    don't care if it'll be an RFC or not, but I would like to see at least 4
    implementors promising they will implement it this year (yes, this now
    again leaves us much time ;)).
    
    >    - RFC3195 via TCP
    > 
    >    Output should support flow-control where possible (e.g. 
    > block the calling
    >    program when the message cannot be sent), but this is optional.
    
    It must - we shouldn't try to implement any flow control in syslog/udp.
    But with TCP, I would like to see a notification to the caller (return
    value) when the loghost could not be contacted. Then leave it to the
    caller...
    
    > 
    >    Output modules should again be linked into the library 
    > statically, though
    >    the set would be extendable.
    > 
    > Output format and output protocol should be controlled by the 
    > administrator, thus I might introduce a configuration file in 
    > /etc. Alternatively environment variables could also be used.
    
    Please keep the Win32 world in mind ;) So let's say we have a helper
    class to obtain the config settings - how about this. That helper could
    then go to /etc, environment or those new sexy XLM files Microsoft
    introduced which resemble the .INI files Microsoft called evil in
    1990...
    
    > 
    > I'm thinking about a blocking API, but in the long run I also 
    > want to support non-blocking operation (in a non-intrusive 
    > way, probably like in OpenSSL).
    
    Non-intrusive is a key word here. For the first release, I would
    definitely stick with blocking. Keep it damn simple, let app developers
    use it...
    
    > 
    > I do not want to rely on anything except libc, and I also 
    > don't want to be intrusive to the main program.
    
    Excellent. I again stress the keep it simple point. *NIX und Win are
    powerful platform. Especially as syslog is involved, we need to keep
    things in mind like low-powered routers...
    
    Rainer
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Fri Jan 03 2003 - 09:09:43 PST