Hi, Quoting Desai, Ashish (Ashish.Desaiat_private) on Thu, Sep 06, 2001 at 09:29:50PM +0200: > > Rsync (http://rsync.samba.org/ is your friend. Yup, I know rsync. Pretty nice with SSH as a transport. Unless a ready made solution with direct DB interfaces comes up we will go that way. Disadvantage is the window of oportunitiy for an attacker that you have between the event and the time the log is synced to a remote system. > sites, you could just rsync the log files to your central server and process > the file by "tail -f" Hmm does rsync support pipes? Or syncing files with are still written to? It wouldn't be that efficient then tough. > You can use "gzip" on the central server to append the incoming logs files > to long term logs files, I don't want to uncompress 1GB / day (or even more) for long term analysis. That would be way to much of hassle and make quick queries impossible. I am looking for a log database that can handle storage and any kind of query with real time and scheduled feeds from the log sources. > One of the things we realized early on is having a well defined directory > hier on the central > server help in the long run. We went with the scheme of > /SOMEROOT/yyyy/mm/server-x/service/fnamex_yyyy-mm-dd.txt . > This allowed us to blow away and restore any year's data from tape. It also > help is partitioning your drives for growth. It does. But it makes queries a nightmare. Using Perl/AWK and friends on flat file DBs doesn't scale too well. At a certain volume, I think we need to leave flat files and turn to a DB. cheers afx -- atsec information security GmbH Phone: +49-89-44249830 Steinstrasse 68 Fax: +49-89-44249831 D-81667 Muenchen, Germany WWW: www.atsec.com May the Source be with you! --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Fri Sep 07 2001 - 12:16:26 PDT