[logs] Re: Centralized Logging + large number of active hosts

From: Tim Sailer (sailer@private)
Date: Wed May 10 2006 - 13:39:16 PDT


On Wed, May 10, 2006 at 12:10:28PM -0400, ScottO wrote:
> - Or, maybe having distributed "collector" syslog servers that somehow 
> dump back to a central syslog server. So a distributed architecture 
> approach.

I've done it this way. Have a number of 'departmental' syslog collectors.
They get all the raw data. Using syslog-ng, forward all data *except* 
the normal stuff you don't want to see (1st pass of data reduction) to
the 2nd tier of servers, but a much lesser amount. For your case of
1000 or so, I'd say 4-6. Use these servers for detailed analysis of your
combined data. Plus, forward all except the stuff you don't ever want to see,
to 1 or 2 machines, where summary analysis can be done, and you will catch
all the stuff you didn't know you did't want to see, or you weren't
expecting, which is an important event in itself.

Tim

-- 
Tim Sailer <sailer@private> 
Information and Special Technologies Program
Northeast Regional Counterintelligence Office
Brookhaven National Laboratory  (631) 344-3001
_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis



This archive was generated by hypermail 2.1.3 : Wed May 10 2006 - 16:23:51 PDT