On Thu, 29 Aug 2002 14:29:46 -0500 "Andy_Bachat_private" <Andy_Bachat_private> wrote: > I've done a lot of log-munging in Perl, and I must report that for any > significant amount of logs, regexps just aren't fast enough. In some cases > I've found a solution using index() and rindex() that was adequate. But > once > you get to that level of optimisation, Perl becomes as ugly as C, and the > C > solution is generally more flexible (because it doesn't have to be > hand-optimised to death to achieve acceptable speed). Honesty, when I wrote logstats, I found the build-a-subroutine-with- regexps-using-eval method to increase performance by about a factor of 10. Just today, my logstats report checked 460,000 lines of log against 2300 regular expressions (not particularly optimized) in under two hours. This on a 270MHz Ultra 5 with external SCSI and 64M (really!) of RAM. The report itself came in at about 160K bytes, making for a decent summary of 460K lines. I'm looking forward to upgrading our loghost to a 1.1GHz FBSD box :-> Jim _______________________________________________ LogAnalysis mailing list LogAnalysisat_private http://lists.shmoo.com/mailman/listinfo/loganalysis
This archive was generated by hypermail 2b30 : Thu Aug 29 2002 - 16:59:48 PDT