Re: [logs] Re: Generic Log Message Parsing Tool

From: Jon Stearley (jrstearat_private)
Date: Fri Jun 14 2002 - 10:44:53 PDT

  • Next message: Sweth Chandramouli: "Re: [logs] nimda web server logs"

    > someone to figure out what that single bullet is, let's define the
    > problem set more explicitly so that then someone who wants to come up
    > with the single bullet can do so confidently.  Basically: if we can come
    > up with a repository of sample messages, and a set of grammars that
    > describes all of those messages in an easily extensible way, then we can
    > set people loose on the problem of creating an engine to implement those
    > grammars with a high degree of certainty that they're solving the right
    > problem; until we have the abstraction layer that those grammars
    > provide, however, any such engine implementations are just shots in the
    > dark.  In the interim, any engine for testing those grammars that people
    > want to use should be fine, so long as the metagrammar is well-defined.
    
    i think this is a fabulous idea!  i'd be happy to help.  ie- cook a
    dataset(s) which could be used as a common set against which to test
    and compare log analyses.  we'd (the wider the representation the
    better) have to agree on what it should contain, like:
      random and periodic, single and multiple line (ie- login/logout,
      boot sequence, over various durations during which there are various
      other events), data from diverse sources (host, facility/program,
      priority, etc).  data describes events over a time range of Y (1
      month?), ranging from common to rare (ie- the "anomalous" lines only
      occur once, twice, ..n.  etc.
    
    and, enumerate a set of goals for which the dataset is
    prepared/offered.  ie:
        - syntax/grammer flexibility, precision, simplicity, etc
        - analysis novelty (what does the analysis produce?)
        - analysis speed
        - algorithm, implementation, etc novelty 
        - etc
    
    lemme know if interested.
    
    -jon
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    For additional commands, e-mail: loganalysis-helpat_private
    



    This archive was generated by hypermail 2b30 : Fri Jun 14 2002 - 11:56:00 PDT