[logs] Re: regex-less parsing of messages

From: Jason Haar (Jason.Haar@private)
Date: Mon Dec 05 2005 - 12:15:11 PST


Solomon, Frank wrote:
> "I don't know what I want to see, but I will know it when I see it."
>
> Is a vague programming specification and will always lead to a vague and
> unsatisfying program.
>
>   

Great statement :-)

> I remember, early on, Tina asking for examples of what kinds of messages
> we look for. . .very few answers.  Perhaps that tells us something about
> how we're approaching this.
>   
I'm in the same boat. We are growing our centralized logging
infrastructure - but to what end? There are specific log analysis tools
for specific problems, but nothing that really does "everything" (which
alludes to your original statement). Even the forensic element is
under-utilized. It requires you have experienced staff who know exactly
how to hunt down whatever it is they are after.

Boring, everyday example:  These days (due to the horrors of antispam
systems) internal users routinely ring the helpdesk and ask "Customer YY
sent me an email and I never got it. What happened?". To figure that out
involves converting what you can learn about customer YY into DNS
records and IP addresses, then tracking any related connections as they
hits the edge of our Internet link. Where it first meets our RBL checks,
then flows through AV and antispam systems, then through a couple more
internal mail relays before hitting our end mail servers. We have logs
all merged together from all those systems, but frankly, I am still the
only one who can link all those events together. And my attempts at
turning that eyeballing into a program have failed so far. And that's
only one example.

-- 
Cheers

Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +64 3 9635 377 Fax: +64 3 9635 417
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1

_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis



This archive was generated by hypermail 2.1.3 : Mon Dec 05 2005 - 18:26:27 PST