RE: [logs] Syslog payload format

From: Andrew Ross (andrewat_private)
Date: Tue Jan 07 2003 - 02:45:40 PST

  • Next message: Andrew Ross: "RE: [logs] Syslog payload format"

    I would be all for a simple extension to the 3164 format, as long as it
    will handle all our requirements. But currently it falls short in many
    ways.
    
    <SOAPBOX>
    
    Since there is no standard or widely used implementation over TCP, we
    have the chance to do it right the first time, rather than carrying the
    weight of an old implementation on our shoulders.
    
    RFC3164 simply documents the existing status quo, that doesn't
    necessarily mean that it is the right way to do things.
    
    Doing syslog over TCP means writing new code anyway, so why not
    establish some mandatory fields that tell us what we want to know. The
    new timestamp is a great start, but why not add in the "RealAddress:"
    tag as well (Actual host that sent the message, not the peer address of
    the collector)?
    
    We could either use a length field or a delimiter to break the incoming
    TCP stream into messages, both options have their merits and drawbacks.
    If we use a delimiter of CR/LF, can we be sure that this won't be
    contained in the message text? What about binary data or DBCS? How do we
    represent that without using a length field at the beginning? (I see
    CR/LF chrs in the middle of UDP syslog messages all the time.)
    
    What other fields are required?
    
    Priority - No problem here. 0 to 191 is fine by me.
    Host name - Should this be fully qualified with the domain name, so it
    is better identified when sending to remote sites?
    Host IP address - Should we include hostname only, or would the address
    be useful too?
    Process ID - Can we ensure that the Process ID doesn't contain a space,
    this would ensure easier parsing?
    
    The problem with 3164 is if a message passes through a collector the IP
    address of the original sender is lost. Syslog-ng, WinSyslog and Kiwi
    Syslog all handle this in different ways by adding an additional tag.
    Since it is obviously a practical requirement, shouldn't we add this in
    to the spec?
    
    What about the length limit of 1024 chrs? As Rainer has pointed out
    earlier, in practice this is too small for windows event log dumps and
    is actually only 512 chrs when using DBCS. Should we not make the limit
    something like 65500? It is not too large to handle and would cover a
    lot of bases. It can also be easily represented in 4 bytes of ASCII Hex
    (FFFF) in a length field.
    
    I think running with a slightly modified version of 3164 would be an
    easy option, but since we won't be using any of the old 3164 code for
    TCP, why not do it right while we have the chance?
    
    </SOAPBOX>
    
    Cheers
    
    Andrew
    
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Wed Jan 08 2003 - 08:48:52 PST