[logs] Logging in the real world

From: Buck Buchanan (lbuchanaat_private)
Date: Wed Jan 15 2003 - 07:57:04 PST

  • Next message: Coffman, Tony: "[logs] Windows shutdown"

    Hi,
    
    "Ogle Ron (Rennes)" <ron.ogleat_private> brings up the point, that OS
    developers may not think that the effort to change syslog or event log to
    provide reliable, detailed, and structured centralized logging is worth it.
    Others will also have that opinion.
    
    For those of you who don't know me, I was a software developer for 8 years,
    a systems administrator for 7, and a security consultant for the last 5.
    Putting on my software developer hat, I would find it hard to justify the
    effort to make logging more reliable.  Reliable logging is not going to
    make much different in how many copies of an OS are going to be sold.
    Switching to my system administrator hat, I see unreliable logging as a
    feature.  Many system administrators disable logging or limit what it logs
    due to a fear of the performance impact.  Network administrators also have
    a very real concern that logging traffic will interfere with "paying
    traffic".  As a security consultant, I have found it a hard sell to get
    people to turn on logging, and a very hard sell to get them to forward
    their logs to a centralized log server.
    
    I ran a centralized log host for a few years with at least 60 UNIX systems
    forwarding *.debug to it.  At the peak it was logging about 100 Megabytes
    per week.  I rarely measured message losses, but I did see a 60% loss rate
    one of those times and it was probably much higher during a very good self
    inflicted denial of service attack.  I was still able to see error messages
    of failing hard disks, printers out of paper, misconfigured systems, and
    the occasional script kiddie probe that got past the front door filtering.
    I was not concerned about the losses, just curious.  What I found to be
    most valuable in the logs was evidence of misconfigured systems.  By
    getting these systems correctly configured, it reduced the amount of
    information logged a week down to about 30 Megabytes while improving the
    systems security and reliability.
    
    Without the demand from a very significant proportion of a vendor's
    customers, I don't see any likelihood of significant changes to commercial
    operating systems to provide reliable, detailed, and structured logging.
    My opinion is that this group's efforts should be directed more to dealing
    with analyzing the content of existing logging systems.  I would like to
    see those of us in this group that work for very large customers to
    exercise their influence on the vendors they deal with to follow our lead
    on improving logging.
    
    B Cing U
    
    Buck
    
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Wed Jan 15 2003 - 08:05:42 PST