Similar Idea... You could use the DTS services in MS SQL Server as well. Start the DTS package at 1:00 AM, and you should be done before you come in for Coffee the next morning. Keep in mind that you'll need a nice box (Dual-Quad Proc, 1+Gb of RAM). From there, you can toss OLAP services and do some quick mining. In fact, if you have early-risers, you can configure your Cubes for HOLAP storage for decent response time and fast refresh of data. Even better -- Toss the daily findings and aggregation of data to an ASP page where you can get to the reports from anywhere. Hope this helps. hd, CISSP -----Original Message----- From: Hans-Joachim Picht [mailto:hansat_private] Sent: Friday, September 28, 2001 11:31 AM To: Nicolai Rasmussen Cc: loganalysisat_private Subject: Re: [logs] Webserver logs to database - Toward data mining On Wed, Sep 19, 2001 at 09:48:22PM +0200, Nicolai Rasmussen wrote: > We run some websites that generates more than 5 gb logs pr. day on approx. > 50 different sites and we would like to put them into a database so we could > do some data mining on them. > > Does anyone have any idears, input, thoughts or anything on how we should do > this ? > > We thought about making a optimized table definition and then dump each line > into the database. From there we would make some summary reports.. This is the way I implemented such a solution for an ISP. We piped around 4 gb of data into a db2 database (on linux) and used ms access (*duck*) to connect to the database to generate traffic bills. -- With best regards Hans - Joachim Picht <hansat_private> --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Fri Sep 28 2001 - 11:47:59 PDT