Comments below. > On Thu, Feb 07, 2002 at 10:20:44AM +1300, Russell Fulton wrote: > > > On Wed, 2002-02-06 at 02:56, Mike Blomgren wrote: > > > > > 1GB in size per day. Doing the analysis once per night is no > > > problem, but, and it's a big BUT, how does one reasonably perform > > > the same analysis for weekly or monthly reports? Or, > heaven forbid, > > > yearly reports? > > > > ummm...I wonder if RRDtool could be bent to work for this sort of > > data? > > > > http://www.caida.org/tools/utilities/rrdtool/ > > RRDtool understands only numerical values. You could reduce > known log messages to counters and track them in an RRD, but > you would lose the actual messages (and any unknowns). So it > is probably not useful for this kind of log analysis. > > In the past, I have used RRDs to store rates of log messages > from web servers, web applications, and the like, and it > works on a small scale. Coincidence, because I was just reading the docs for RRD Tool, and came to more or less the same conslusion, i.e. not quite suitable for my needs. However, judging from the responses so far, I think the best bet is to save the actual loglines that are used for the statistics, and that contain any 'interesting' informtaion. This would reduce the required information to 1% of the original logsize, at least. Some information will get lost, but the majority of it ought to be in the extract. As long as we along the line don't discover that the extract is missing vital information, we should be OK.... But, to summarize, I'm suprised that aren't more solutions to this problem. However, I haven't followed up on one of the products I was suggested to look at. And I'm sure there are a few more who don't read this list... Thanks to all who responded! Regards, ~Mike --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Thu Feb 07 2002 - 12:32:48 PST