Re: [logs] High Network Load

From: Marcus J. Ranum (photonerdat_private)
Date: Fri Sep 19 2003 - 13:18:04 PDT

  • Next message: Marcus J. Ranum: "RE: [logs] High Network Load"

    Paul Robertson wrote:
    >> But what if the OS kernel drops the message?  Does anyone here have any 
    >> experience with the OS losing messages before they get to the syslogd 
    >> process?  How can this be monitored and overcome?
    On UNIX boxen most client sides write to /dev/log, which behaves
    differently (since it's a pseudo-device) than an actual UDP send.
    I've never measured the kernel dropping log messages between
    the application, /dev/log, and the kernel. I have, however, measured
    some horrible log traffic loss...  (see below)
    >With the right volume, the OS won't even get the message, it'll be dropped 
    >at the router if its buffers get full...
    With the right volume, it'll never leave the machine. I did some
    testing (and posted it to loganalysis ages ago) and discovered
    that the UDP output queue is maintained per network interface
    and is relatively "shallow" -- lots of outgoing UDP packets results
    in them simply being tossed before they leave the box. I tested
    this by running tcpdump on one machine while I syslogged in a
    tight loop on the other. I counted something like 10,000 packets
    sent as a result of 1,000,000 syslog() calls. Syslog over TCP
    will, of course, not exhibit this issue. It'll suck in other ways.
    Syslog is very, very badly designed.
    LogAnalysis mailing list

    This archive was generated by hypermail 2b30 : Fri Sep 19 2003 - 17:42:49 PDT