Re: [logs] Generic Log Message Parsing Tool

From: Sweth Chandramouli (loganalysisat_private)
Date: Mon Jun 10 2002 - 23:23:32 PDT

  • Next message: Tina Bird: "[logs] nimda web server logs"

    On Tue, Jun 11, 2002 at 02:03:03PM +1200, Russell Fulton wrote:
    > I've kept my head down for most of the discussion because I could see
    > that others knew far more about the basic issues than I did.
    	I hope others aren't doing this as well; more people
    participating in the discussion can only help it.  My comp. sci.
    background is very weak, for example, especially with regards to the
    languages that are best suited to writing parsers (more on that in a
    second); having folks with actual comp. sci. backgrounds and the like
    participating would at worst give us some different viewpoints that
    might lead to alternate ways to approach the problem.  Speaking of
    which...
    
    > While looking at the books on my shelf this morning a thought occurred
    > to me:  Has anyone considered ICON http://www.cs.arizona.edu/icon/ as a
    > possible implementation language for a log parser?  Almost certainly
    > not!
    	I actually had thought of ICON briefly, simply because it
    came up a lot when I first started looking into writing parsers.  Also
    on the list of languages like that are a lot of functional languages,
    like LISP and ML.  I personally don't intend to _learn_ all of those
    languages, but one thing I've discovered over the last couple of years
    is that each language makes doing certain things easy and as a result
    programmers in each language learn lots of different ways to solve the
    same problem.  So I've read enough stuff on, for example, functional
    programming that I now have Yet Another way of attacking the problem
    of parsing something.  That's where, at least for me, things like ICON
    would be most useful--programmers in a procedural language that makes
    parsing easy have probably already invented a lot of the wheels that I
    find myself inventing while parsing with another (ostensibly) procedural
    language like, say, Perl.
    	As I said earlier in the thread, my current focus is on
    developing the grammars that will let programmers in any language parse
    log messages; since I'm most familiar with Perl, then, most of my
    prototyping will probably end up being done in that language.  But I'd
    love to see someone take some of those grammars and implement them in,
    say, ICON, just to see what the result is like--things like that can
    only help grow the knowledge base of the entire community.  And (as
    Marcus would be quick to point out, I'm sure :) ) until that first
    version is hacked out in ICON, there's no way of knowing for sure
    whether or not ICON might not just be the most appropriate tool for the
    job in the long run.  (Basically: I personally don't have a problem with
    the "I have a hammer, everything looks like a nail" mentality, so long
    as it's really just a "I have a hammer, I wonder what I can learn by
    using it to try to cut this two-by-four?" mentality.  The worst that can
    happen then is that people learn not to use a hammer to cut; the best
    that can happen is that it turns out that cutting wood with a hammer is
    easier than it is with a saw.)
    
    > ICON is the brainchild of Ralph Griswold (one of the authors of Snobol
    > -- yes, I'm showing my age, does anyone else remember Snobol?)
    	Well, I wouldn't say I _remember_ Snobol, but I'm _aware_
    of it... does that count?  :)
    
    > ICON is
    > a great langage for building parsers, it is interpretive and has
    > powerful string manipulation facilities built in.  It also has the
    > concept or goal evaluation and backtracking built into the language. 
    > There is a section of ["The ICON Programming Language" by Griswold and
    > Griswold; Prenice Hall; ISBN 0-13-447889-4] devoted to writing parsers
    > in the chapter on "strings and pattern matching".
    	If anyone is interested, it appears that the Griswolds
    are having a clearance sale on their stock of this book, so it only
    costs $17 (with free shipping in N. America) through the end of June;
    details are on the website noted above.  I'll probably be getting
    a copy myself, at that price...
    
    > The down side, of course is that it is relatively unknown and (so far as
    > I know) does not have a wide support base like perl, python etc.
    	It doesn't need to be in wide use to help us; all it needs
    to do is have at least a few clueful folks (perhaps you and your 4th
    years? :) ) to use it to attack the problem, see what comes of it, and
    then share their results.
    
    	-- Sweth.
    
    -- 
    Sweth Chandramouli      Idiopathic Systems Consulting
    svcat_private      http://www.idiopathic.net/
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    For additional commands, e-mail: loganalysis-helpat_private
    



    This archive was generated by hypermail 2b30 : Tue Jun 11 2002 - 08:34:19 PDT