Re: [logs] High Network Load

From: Marcus J. Ranum (photonerdat_private)
Date: Fri Sep 19 2003 - 13:18:04 PDT

  • Next message: Marcus J. Ranum: "RE: [logs] High Network Load"

    Paul Robertson wrote:
    >> But what if the OS kernel drops the message?  Does anyone here have any 
    >> experience with the OS losing messages before they get to the syslogd 
    >> process?  How can this be monitored and overcome?
    
    On UNIX boxen most client sides write to /dev/log, which behaves
    differently (since it's a pseudo-device) than an actual UDP send.
    I've never measured the kernel dropping log messages between
    the application, /dev/log, and the kernel. I have, however, measured
    some horrible log traffic loss...  (see below)
    
    >With the right volume, the OS won't even get the message, it'll be dropped 
    >at the router if its buffers get full...
    
    With the right volume, it'll never leave the machine. I did some
    testing (and posted it to loganalysis ages ago) and discovered
    that the UDP output queue is maintained per network interface
    and is relatively "shallow" -- lots of outgoing UDP packets results
    in them simply being tossed before they leave the box. I tested
    this by running tcpdump on one machine while I syslogged in a
    tight loop on the other. I counted something like 10,000 packets
    sent as a result of 1,000,000 syslog() calls. Syslog over TCP
    will, of course, not exhibit this issue. It'll suck in other ways.
    Syslog is very, very badly designed.
    
    mjr. 
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Fri Sep 19 2003 - 17:42:49 PDT