Re: [logs] another gooey performance question

From: Mikael Olsson (mikael.olssonat_private)
Date: Tue Jan 21 2003 - 01:20:33 PST

  • Next message: James F.Hranicky: "Re: [logs] another gooey performance question"

    Tina Bird wrote:
    > 
    > time for another completely unscientific survey:
    > 
    > for those of you who are running centralized loghosts, what's the average
    > amount of data you collect in a day?  what's the average CPU utilization
    > on your server?
    
    Our log receiver is a celeron-633, writing to two EIDE drives
    through soft raid. ext3 file system. (I wanted plain file copy a 
    couple of times a day to better guard against file system failure, 
    but that's another story)
    
    In the beginning, it used syslog-ng. With the default parameters
    it hit the ceiling at about one gigabyte per day.
    With some buffer and queue size tweaking, I got it to about two
    gigabytes a day, but 100% CPU utilization and a sysload of about
    two to three made me give up and write my own minimalistic receiver. 
    
    We're currently at about four gigabytes daily (about 15M events),
    with a CPU load of about 7% and a sysload of about 0.1.
    The system reports about 1% CPU time spent in the receiver
    and 5% time spent in kjournald. (?)
    
    -- 
    Mikael Olsson, Clavister AB
    Storgatan 12, Box 393, SE-891 28 ÖRNSKÖLDSVIK, Sweden
    Phone: +46 (0)660 29 92 00   Mobile: +46 (0)70 26 222 05
    Fax: +46 (0)660 122 50       WWW: http://www.clavister.com
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Tue Jan 21 2003 - 09:36:16 PST