Re: [logs] most popular reports...?

jlz@private

Just curious, how do you determine the stuff that you are *not* interested in?
It seems to be a fairly subjective exercise and may result in losing 
important data. (genuine question, not flame bait :)

As an example, deciding all accepts on a firewall are *not* interesting
may not be valid. 

I do agree that some process of elimination is necessary for log analysis,
where this happens is the question.

<RandomThougths>

It seems like in most real-world cases, log analysis is triggered by some
stimuli, e.g. an alert (IDS, SIM, human) or a log report (text or graphical 
format) showing something interesting. Most sysadmins are probably too busy
to consciously go and review logs unless something happens.

It also seems like most of the time, the process of correlation is in 
reverse chronological (temporal) order, as in, you go backwards in order 
to find out what happened (when did the attack occur, where did it come from,
how did it occur). Sometimes you will go forward in time to find out if other 
similar incidents has happened or if the attacker used the host for other
attacks.

There's also a spatial component to the process of correlation as well.
We generally look at logs from multiple hosts/devices across the network
in order to figure out the root cause.

Almost all of the SIM vendors are implementing their systems based on
forward temporal correlation, this is probably useful in detecting some
of the more well known scenarios. If the scenario is not known, most 
likely the attack will be missed.

For most investigations, it seems like reverse spatial & temporal analysis
is more useful.  Pro'ly less processor and memory intensive. However, 
(hey! STRACE = Spatial & Temporal Reverse Analytics & Correlation Engine)

The question is can something like that built to perform the reverse
correlation automatically (or if it's useful at all). There probably has 
to be some rules to tell the engine what to look at next (in reverse).

</RandomThougths>

Tim Sailer (sailer@private) [040819 12:30]:
> The way I've approached it, in my own worthless way, is to tell the tools
> the stuff that I'm *not* interested in. Data reduction is a must in a
> large centrally logging environment. You then pick through the rest
> to find the good nuggets. Approaching logs looking at *all* the data is
> simply a way to lose sleep/hair/braincells. You can then apply the
> fancy tools to visualize the important stuff, and, by default, the 
> stuff you never expected to see, since you didn't way you didn't care
> about it.
> 
> Tim
> 
> -- 
> Tim Sailer <sailer@private> 
> Information and Special Technologies Program
> Office of CounterIntelligence 
> Brookhaven National Laboratory  (631) 344-3001
> _______________________________________________
> LogAnalysis mailing list
> LogAnalysis@private
> http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis