RE: [logs] syslog/tcp (selp)

From: Rainer Gerhards (rgerhardsat_private)
Date: Thu Jan 09 2003 - 13:10:20 PST

  • Next message: Bennett Todd: "Re: [logs] syslog/tcp (selp)"

    Hi list,
    First of all, I would like to thank all of those that so far sent
    feedback. Thanks for the encouraging words and special thanks for the
    many great ideas.
    I will definitely go ahead with the project as it looks like there is a
    need for it. I will revise the document and I fear this will take me at
    least over the weekend, maybe a bit longer (depending on what else is
    becoming pressing ;)). And, yes, this time I will post the nroff source
    togeher with the generated document.
    Based on the discussion, I am currently thinking about one BIG change in
    the protocol. Discussion has show that non-ANSI characters are something
    we should have in there but has also shown that there are severe issues
    with the CRLF trailer as soon as we allow this. The main reason is that
    e.g. in DBCS charsets CRLF can be part of the "normal" message - not the
    terminator. This introduced the idea of a byte count header for the
    message, before the PRI part. I, too, think this is a great idea, I just
    fear (as always) about the "syslog-likeness" of the protocol, which we
    more and more depart from. I think I now have an acceptable solution. I
    will very quickly draft it here, leaving out many details and samples. I
    do so in the hope to receive feedback before I re-draft the document. So
    here it is:
    SELP will come in two flavours or profiles as RFC3195 calls them. Hold
    on, it'll won't get that compilcated, please read on...
    There is "syslog classic" format and a "syslog extended" format. "syslog
    classic" has most of the restrictions RFC3164 puts on us, that is
    - US-ASCII charset, printable characters, only
    - 1024 overall size limit
    However, it extends RFC3164 in the following ways:
    - TCP instead of UDP
    - CRLF TRAILER to allow multiple message in a single stream
    - RFC3339 timestamp
    - fully qualified host names allowed
    The goal of the "syslog classic" (sub-) format is to allow for extremely
    easy interop with classic and RFC-series syslog. It should be very easy
    for a relay to convert to/from this format. But it still has the plus of
    The "syslog extended" (sub-)format still uses the spirit of syslog, but
    it is harder to interop with classic or RFC syslog. Here we have the
    following additions on top of the "classic syslog":
    - up to 65000 chars allowed per message
    - 8 bit encoding in the MSG part of the message (so DBCS is fine)
    - a leading byte count so that messages are NOT split by the CRLF
    - byte count expressed in printable chars (either in hex or dec - I
    still think
      about decimal, because it is used in so many Internet protocols and
    easier during
      debug - but wastes one byte...)
    - for consitency, the CRLF TRAILER is still there, but now more or less
    a tool to
      check for framing correctness (if we don't drop it altogether)
    This are the main points. Of course, others could be added (and I may
    have overlooked something that came up during the discussion). But I
    would still opt to keep it as simple as possible, otherwise we don't
    fill a gap but could already go for RFC3195.
    And how to differentiate between "syslog classic" and "syslog extended"?
    Fortunately a simple shot. Let's use the header. "syslog classic" must
    start with the PRI field. So the message will look like
    <121>rest of message (not of interest here)CRLF
    "syslog extended" starts with the fixed size byte count. So the same
    message would look as follows (I use a dec representation here):
    00100<121>rest of message (not of interest here)CRLF
    (the 100 is just a sample - I don't care here about anything but the
    first parts, thus also no timestamp and the like)
    As all message MUST be well-formed (or treated as errors), the receiver
    just needs to check the initial byte and then can decide if it needs to
    parse a "syslog classic" or "syslog extended" message. I can even
    envison that both message formats are intemixed in a single stream, but
    have not yet made my mind up if that is desirable.
    A receiver could also decide (or be configured to decide) to NOT accept
    "syslog extended" if it is concerned about the security issues that
    might arise out of 8 bit encoding. Of course, that could spawn off a new
    discussion on notifying the sender of this decision which would in turn
    introduce the need for acknowledgment (an issue that is already lurking
    under the hood).
    However, I would NOT like to take that path - this will definitely bring
    us in a direction very close to where RFC3195 already is. Keep in mind:
    we are looking for the second best solution (some may even say the third
    best ;)) with the sole advantage of being very easy to implement. We all
    know that we could do much better and in fact I think RFC3195 is a good
    standard if we would like to have a full-blow, most concerns addressed
    standard. Our approach will always be less sophisiticated as this
    less-sophisticatedness is a design goal ;)
    I would appreciate feedback on these thoughts very much. I'll sit back
    and relax in the mean time. Or, more precisely, I'll hold the revision
    of major parts of the draft a little in the hope to save some editing
    work. Please be as critical with the above suggestion as possible. The
    reason is I am myself not 100% sure that it is really the route to take.
    Maybe it would be much better to just have a "syslog extended" like
    format and the two formats simply screw up the intended simplicity...
    > -----Original Message-----
    > From: Rainer Gerhards 
    > Sent: Wednesday, January 08, 2003 9:47 PM
    > To: loganalysisat_private
    > Cc: Mikael Olsson; Andrew Ross; Balazs Scheidler; 
    > avalonat_private
    > Subject: [logs] syslog/tcp (selp)
    > Hi all,
    > I have tried to create a draft spec for the syslog over tcp 
    > protocol discussed on the list. It is a very (very!) rough 
    > first draft, primarily intended to raise some discussion here 
    > on the list. I have decided to create a draft because 
    > although the extensions to RFC3164 are only minimal, I would 
    > like to have a spec to be used when implementing it. Also, I 
    > would like to make sure that those of us that will indeed 
    > implement it do so from a common understanding. As we all 
    > know, the details cause the pain...
    > I do not expect this draft to become an Internet Draft or a 
    > RFC - there is so much violent opposition in the IETF syslog 
    > WG against any syslog over tcp that is not BEEP, that I don't 
    > think we can become it accepted with a reasonable amount of 
    > work (and there are some good reasons for it to be so - but 
    > that's an other story). Anyhow, I have tried to format it 
    > RFCish, as this format is well known among implementors. It 
    > is my first try in doing so. Please bear with me a little ;)
    > Oh, one more thing: I have (silently) decided to NOT call it 
    > "syslog over tcp" but "simple event log protocol" (being 
    > inspired by the eventlog libraries popping up for the 
    > payload). I have the impression that syslog over tcp could be 
    > misleading, especially when RFC3195 (syslog/tcp via beep) 
    > becomes more popular over time. Of course, this is not a real 
    > decison but rather a suggestion.
    > The draft so far is incomplete and probably inconsistent in 
    > some areas. If it were software, I would call it 0.1 ;) My 
    > primary goal was to have a quick shot to raise these questions:
    > - does the list find such work valuable? Does it make sense 
    > to continue?
    > - does a name other than "syslog over tcp" make sense? is the 
    > one I have
    >   choosen ok? Any other suggestions?
    > - does it sum up what has been discussed regarding the transport 
    >   (I won't cover the payload - at least not now)?
    > - are there any things to add?
    > - who would be willing to implement?
    > The draft is available at
    > Looking forward to all kinds of feedback ;)
    > Rainer
    > _______________________________________________
    > LogAnalysis mailing list
    > LogAnalysisat_private 
    >> lysis
    LogAnalysis mailing list

    This archive was generated by hypermail 2b30 : Thu Jan 09 2003 - 15:25:28 PST