[logs] Re: Centralized Logging + large number of active hosts

From: Marcus J. Ranum (mjr@private)
Date: Wed May 10 2006 - 23:02:24 PDT


I think an important point to keep an eye on is that you can get 5 or
6 mid-range computers for the cost of a single high-end computer,
and you get much more than 5 times the capability and flexibility.
As your logging infrastructure goes you will eventually need to go
heirarchical anyway, so doing it now avoids putting you in the position
of having to figure out how to re-design a non-heirarchical system
into a heirarchical one down the road. "Been there, done that" and
let me show you the scars.

Distributing your processing between the edges and a central
location means you can use relatively inexpensive collectors
at the edges and you can upgrade the hardware on a per-edge
basis as necessary. Trying to guesstimate the size of a central
capable of handling an unpredictable amount of traffic almost
always results in an over-spec solution. So if you start with a
bunch of cheap edge aggregators and a central that isn't
particularly beefy (it doesn't need to be) then you'll find
that mostly all you'll need to do is add disk space to the
central every year or 2, and maybe another aggregator or 2.
If you've built your software infrastructure for heirarchical
collection, that's really really easy.

mjr.

_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis



This archive was generated by hypermail 2.1.3 : Thu May 11 2006 - 11:14:35 PDT