Re: [logs] perl question relating to log analysis

From: Russell Fulton (r.fultonat_private)
Date: Thu Aug 29 2002 - 14:43:30 PDT

  • Next message: James F.Hranicky: "Re: [logs] perl question relating to log analysis"

    Hopefully my last post on this topic....
    
    I have been doing some test on a specially prepared dataset and have
    reached the following conclusions.
    
    1/ That the order of REs *does* matter (like many tried to tell me).  My
    faith in perl's optimization was misplaced.  I was able to speed up
    runtime by about 50% by reordering REs to match frequencies of matches.
    
    2/ for fixed strings there was almost no difference between RE and index
    (in fact RE were very slightly faster).  That said the test strings were
    short (around 50 characters) and index may win out with longer strings. 
    I would not be surprised if the optimization recognised that the string
    was fixed and replaced it with what is effectively an index() call.
    
    For the moment I am going to stick with REs but let the user flag those
    rules that they expect to trigger most frequently.  A little later I'll
    look at adding an option that uses Sweth's clever code to report on the
    frequence and perhaps modify the config file accordingly.
    
    Thanks again for all those who responded and to the rest of you for you
    patience!
    
    Cheers, Russell
    
    -- 
    Russell Fulton, Computer and Network Security Officer
    The University of Auckland,  New Zealand
    
    "It aint necessarily so"  - Gershwin
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Thu Aug 29 2002 - 16:54:49 PDT