[logs] Re: on log standards

mjr@private

Gosh, we discuss this topic every year. It must be September ... ah, yep.

Enumerating all the log messages: not gonna happen
Changing the structure of log messages: not gonna happen
Change the transport of log messages: I give it a 20% chance
Adopt standard structural elements in log messages: not gonna happen
		(2 years ago, I thought this was actually possible)
All logs in XML: not gonna happen*

So where are we left?

1) Analysis of interesting classes of messages
2) Workflow for identifying new classes of messages

The bad part is that "interesting" is not absolute, therefore all log
analysis remains site-specific. That's a good thing, from where I
sit, and it doesn't mean that tools to facilitate, share, and process
don't make a great deal of sense.

I'm afraid that the current state of security (e.g.: brainless
"give me something in a 1U rack mount configuration that
tells me everything is OK") will not accept a problem that
has no clear and trivial "solution." Of course all of security
has no clear and trivial solution, but that's where all the
customers want to spend their $$ right now. Yet, oddly,
they complain when it doesn't work.

The workflow stuff is easy. That's just automation around
artificial ignorance, whitelists, greylists, and blacklists,
with shared structural templates. I haven't had a chance
to look at splunk, yet (it's on my TODO list) but it sounds
like they are heading in the right direction there. The
analysis of interestingness is a problem we've seen
effectively tackled outside of security. Take a look at
digg.com or even slashdot if you want an example of
community-oriented interestingness filtering.

mjr.
(* because some of us will take you out and "UDP syslog" you if you try,
if you know what I mean...)

_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis