RE: [logs] Syslog payload format

From: Ogle Ron (Rennes) (ron.ogleat_private)
Date: Tue Jan 14 2003 - 11:33:23 PST

  • Next message: Frank O'Dwyer: "RE: [logs] Syslog payload format"

    > Good example. Apache uses a structured, machine parseable log format.
    > The URL etc, is always in the same place and clearly delimited.
    > If it didn't, no pretty pictures, at least not easily.
    > This all happened by the way, because the early web server 
    > developers worked
    > out a common logging format. Before that there were as many 
    > formats as there
    > were web servers (which wasn't that many then). Each logging 
    > similar stuff
    > in a slightly different way. So that particular problem was 
    > nipped in the
    > bud.
    Exactly, the application people did it.  Syslog didn't get in the way one
    iota, no massive framework to deal with either.  Their logging format was
    tailored to what they did and the data they produced.
    It's much easier to get a small group of people to do something than
    everyone in the world.
    > > The developers of Apache sent out to logs all of the meaningful
    > > data in the
    > > format that they see fit because it is how they see the data in a
    > > nice easy
    > > to carry package.  Syslog gives me the means to move it off the
    > > machine to a
    > > log server.  Nice and Efficient!  BTW, my Apache server didn't
    > > have to spend
    > > any time chunking/tagging the data.
    > First rule of optimisation:
    > Don't.
    > Second rule of optimisation (for experts only):
    > Optimise later.
    > In fact Apache does spend time structuring the entry, and 
    > could instead
    > spend even more time logging arbitrary crap wherever it was 
    > available in the
    > code, which is what a lot (most?) other apps do. Not that 
    > 'time' is really
    > the issue, since we are hardly talking about factoring large 
    > integers here.
    Perception by administrators is half the battle, but time isn't the only
    resource that logging takes.
    > "Received GET request for from"
    > "Request for complete (code 200, 
    > 12892 bytes
    > transferred)"
    > Try developing tools to parse that kind of thing, where each 
    > vendor does it
    > differently.
    I agree from an individual vendors prospective.  This is why I said
    standards for applications and OSes.  Attack the standard problem on
    categories of applications not individual vendors.  I'd much rather see
    canned formats that are obligatory come out as a standard for logging than a
    smorgasbord of tags come out that are chosen randomly by vendors.
    I could easily a parser for each category.  While I might not be getting
    every piece of data that could be possible from the smorgasbord, I am
    getting what I need to do the job.  From my prospective an 80% solution is
    fantastic because nothing ever is perfect.
    Ron Ogle
    Rennes, France
    LogAnalysis mailing list

    This archive was generated by hypermail 2b30 : Tue Jan 14 2003 - 11:56:32 PST