I would suggest that before you insert the data into any database that you find some way to remove log entries that are un-important, like images and such. Nistor.Lubomir@Star-21.De wrote: > > I was thinking as well about log processing.. > You just have to design a script, that will take over logs from apache, > split it into fields and put it into SQL, that can be sitting on one > server... > > But, I'm not that good in scripting to split one line into fields.. > everything else is fine.. > > anyway you could do it in C, or in perl or whatever.. I need some binary for > it.. > > anybody got ideas? examples? > > -----Ursprüngliche Nachricht----- > Von: Nicolai Rasmussen [mailto:nicolaiat_private] > Gesendet: Mittwoch, 19. September 2001 21:48 > An: loganalysisat_private > Betreff: [logs] Webserver logs to database - Toward data mining > > Hi! > > We run some websites that generates more than 5 gb logs pr. day on approx. > 50 different sites and we would like to put them into a database so we could > do some data mining on them. > > Does anyone have any idears, input, thoughts or anything on how we should do > this ? > > We thought about making a optimized table definition and then dump each line > into the database. From there we would make some summary reports.. > > Please write us if you have any idears :-) > > We really appreciate it! > > Nicolai Rasmussen > Software Engineer > > --------------------------------------------------------------------- > To unsubscribe, e-mail: loganalysis-unsubscribeat_private > For additional commands, e-mail: loganalysis-helpat_private > > --------------------------------------------------------------------- > To unsubscribe, e-mail: loganalysis-unsubscribeat_private > For additional commands, e-mail: loganalysis-helpat_private -- djenkinsat_private Universal Savings Bank. Security Administrator, Unix Administrator, Alpha Geek The three most dangerous things are a programmer with a soldering iron, a manager who codes, and a user who gets ideas. --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Thu Sep 20 2001 - 09:55:28 PDT