RE: [logs] High Network Load

From: Brown, James (Jim) (JBrownat_private)
Date: Fri Sep 19 2003 - 07:32:45 PDT

  • Next message: Marcus J. Ranum: "Re: [logs] High Network Load"

    Hi,
    
    Your concerns about packet flow and whether packets are being 
    dropped in transit or on the host OS can be answered with
    SNMP monitoring.  All modern routers and switches keep SNMP statistics
    for full buffers, packet discard rates and other similar conditions.
    Most OS's keep similar data.  
    
    Because SNMP monitoring can itself be traffic intensive, if you can 
    devise an out-of-band solution for collection and monitoring you
    will be better off in the long run.
    
    Best Regards,
    Jim Brown, ThruPoint
    
    
    -----Original Message-----
    From: Philip Webster
    To: loganalysisat_private
    Sent: 9/18/03 7:59 PM
    Subject: [logs] High Network Load
    
    Hello,
    
    I'm implementing a central log server over a large class B network, and 
    have chosen syslog-ng as the server.  One of syslog-ng's features is 
    that it can report the number of messages dropped internally, usually 
    through either the receive or write buffers not being large enough. 
    These values can be tweaked, and at least you know when there is 
    something going wrong.
    
    But what if the OS kernel drops the message?  Does anyone here have any 
    experience with the OS losing messages before they get to the syslogd 
    process?  How can this be monitored and overcome?
    
    The server (Red Hat Advanced Server) will be accepting logs over both 
    UDP and TCP (and possibly via SSH port forwarding and/or stunnel), 
    sitting on a 100Mb connection, and may potentially have hundreds of 
    machines logging to it, as well as routers, switches, and several very 
    high volume proxy servers.
    
    Any thoughts?
    
    Thanks
    Phil
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    Note:  The information contained in this message may be privileged and
    confidential and protected from disclosure.  If the reader of this message
    is not the intended recipient, or an employee or agent responsible for
    delivering this message to the intended recipient, you are hereby notified
    that any dissemination, distribution or copying of this communication is
    strictly prohibited.  If you have received this communication in error,
    please notify us immediately by replying to the message and deleting it from
    your computer. Thank you.  ThruPoint, Inc.
    
    
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Fri Sep 19 2003 - 09:58:53 PDT