[logs] Re: perl question relating to log analysis

Previous message: Darren Reed: "[logs] Please be aware of syslog-sec IETF group..."
In reply to: Russell Fulton: "[logs] perl question relating to log analysis"
Next in thread: Chris Adams: "Re: [logs] perl question relating to log analysis"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

thompsonat_private

Russell,

Thoughts:

* Use the /o option to indicate Perl should compile only once (see
  the perlop manpage);

* See the qr// operator for "compiling" regexps (also in perlop);

* See the FAQ in the perlfaq6 manpage, searching for "efficiently".

I'm not an expert on regexp efficiency.  Just on references to
the documentation on the topic. :-)

En paz,
Steve, security analyst

> My immediate concern is that the perl scripts builds functions that
> apply lots of regular expressions (REs) to each line of log files.
> 
> sub check {
>     $_ = shift;
>     study $_;   #hopefully speed up matching...
> 
>     return 0 if /re1/;
>     return 0 if /re2/;
>     return 1 if /re3/;
>     return 1 if /re4/;
>     return 1 if /re5/;
>     return 2 if /re6/;
>     return 2 if /re7/i;
>     return 3 if /re8/;
>     ...
>     return 4;
> }
> 
> return code tells the program what to do with this record.
> 
> Anyone know of any tricks to speed this up since this is the innermost
> loop of the process any gains here should be worthwhile.  I know the RE
> optimizer is pretty smart and that it will do some optimization over
> statements but I have never figured out what the limitations are.

-- 
Stephen W. Thompson, UPenn, ISC Information Security, 215-898-1236, WWW has PGP
thompsonat_private    URL=http://pobox.upenn.edu/~thompson/index.html
  For security matters, use securityat_private, read by InfoSec staff
  The only safe choice: Write e-mail as if it's public.  Cuz it could be.
_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis