RE: [logs] Syslog payload format

fodat_private

Ron Ogle wrote:
> I'll give you an example.  My apache web server sends out access
> logs that I
> roll right over into syslog.  (Actually it's syslog-ng because of the UDP
> issue.)  The format is straight forward, easy to parse, and low-overhead.
> Then on my log server side, my parsing and graphing tools make nice little
> pictures.

Good example. Apache uses a structured, machine parseable log format.

The URL etc, is always in the same place and clearly delimited.

If it didn't, no pretty pictures, at least not easily.

This all happened by the way, because the early web server developers worked
out a common logging format. Before that there were as many formats as there
were web servers (which wasn't that many then). Each logging similar stuff
in a slightly different way. So that particular problem was nipped in the
bud.

> The developers of Apache sent out to logs all of the meaningful
> data in the
> format that they see fit because it is how they see the data in a
> nice easy
> to carry package.  Syslog gives me the means to move it off the
> machine to a
> log server.  Nice and Efficient!  BTW, my Apache server didn't
> have to spend
> any time chunking/tagging the data.

First rule of optimisation:
Don't.

Second rule of optimisation (for experts only):
Optimise later.

In fact Apache does spend time structuring the entry, and could instead
spend even more time logging arbitrary crap wherever it was available in the
code, which is what a lot (most?) other apps do. Not that 'time' is really
the issue, since we are hardly talking about factoring large integers here.

"Received GET request for http://www.littlecatZ.com from x.y.com"
"Request for http://www.littlecatZ.com/ complete (code 200, 12892 bytes
transferred)"

Try developing tools to parse that kind of thing, where each vendor does it
differently.

Cheers,
Frank.

_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis