Re: [logs] High Network Load

From: Paul Robertson (proberts@private)
Date: Mon Sep 22 2003 - 19:30:33 PDT

  • Next message: Rainer Gerhards: "RE: [logs] High Network Load"

    On 22 Sep 2003, Florin Andrei wrote:
    > > Don't put all your logs in one basket.
    > > 
    > > I can't imagine what design criteria fed into "Log everything over the 
    > > network to a single server," but you should re-evaluate it fairly 
    > > critically.  Disk is slow, everyting going to one logging daemon, logging 
    > > to one filesystem (probably through one route) is going to be 
    > > not-the-best-architectural-idea-anyone's-ever-had.
    > It depends on what are you trying to accomplish.
    Yes, if you're trying to build a system that will fail spectacularly, it's 
    quite a nice start.  If you're trying to build sustainable infrastructure, 
    it's likely to be a bad plan.
    > I can see the truth in your rebuttal, but there is a fair amount of
    > truth in the original message too.
    Given what we've seen in network traffic issues with the last crop of 
    worms, unless you're talking about using out-of-band transport, it's a 
    scheme doomed for failure at the point where you're really going to 
    _operationally_ *need* those logs.  $diety help you if there's a real 
    systemic attack happening at the same time as anything else with such a scheme.
    > Centralising syslog is good if you must analyse the information that
    > syslog provides in a centralised fashion. Sure, there are lots of things
    No, centralizing your analysis of syslog is good, that doesn't immediately 
    equate to centralizing syslog, nor should it.  On small and medium-sized 
    networks, you may be able to get away with it, but on large networks, it's 
    a bad idea.  Especially if you're one of those places where exception to 
    policy is a pain, or where the operational folks won't pay for more than 
    one of anything that's remotely useful if they think they can just tack 
    more on to the one they already have.
    If it's important, then log integrity and availability are also important, 
    and putting all your logging eggs in one basket is going to fail at some 
    point.  Hardware, network load, software, commo, whatever reason, Murphy's 
    going to come a' swinging the cluebat.  It's bad disaster preparedness to 
    design systems thusly.  It's bad infrastructure design to design massive 
    single points of failure in a place where the failure mode is equal to the 
    mode during failure.
    "Hundreds to thousands of systems" all relying on one logging subsystem, 
    most likely via in-band communications?  If the logs are important, that's 
    a bad idea, and if they're not important, then why waste the time looking 
    at them[1]?
    > you could do with SNMP, but i don't think the areas covered by syslog 
    > and SNMP are mutually inclusive (i.e. the same).
    I wouldn't advocate SNMP either.
    [1] Yes, it's rhetorical, they're good to have, but less useful than 
    things done well.
    Paul D. Robertson      "My statements in this message are personal opinions
    proberts@private      which may have no basis whatsoever in fact."
    probertson@private Director of Risk Assessment TruSecure Corporation
    LogAnalysis mailing list

    This archive was generated by hypermail 2b30 : Wed Sep 24 2003 - 11:46:33 PDT