RE: [logs] most popular reports...?

toby.kohlenberg@private

having had to deal with pretty big data sets, I can tell you that
when it's done well, visualization is pretty useful. Most of the
time it isn't done well though and as a result it's tantalizingly
on the edge of being useful. There are a bunch of companies that 
do visualization of various sorts:
http://www.advizorsolutions.com/
http://shoki.sourceforge.net/shoki/hustler_doc/ (kay, this isn't logs
but it's cool enough to look at)
http://www.cs.sandia.gov/VIS/projects.html
http://www.oculusinfo.com/
http://www.proclarity.com/products/default.asp
http://www.spotfire.com/
http://cgi.cs.wisc.edu/scripts/waveletidr/cvs/framenet/welcome.pl?sessio
n_id=1007103

There are people working on this seriously:
http://nvac.pnl.gov/
http://www.cs.umd.edu/hcil/research/visualization.shtml

But you have to figure out what your goals are if you are going to
visualize the data. Do you want to:
find interesting things that you weren't aware of and couldn't define as
interesting?
maximize the amount information available in a single view?
make pretty pictures?

Each one requires a slightly different approach.

t 

>-----Original Message-----
>From: 
>loganalysis-bounces+toby.kohlenberg=intel.com@private 
>[mailto:loganalysis-bounces+toby.kohlenberg=intel.com@private
>oo.com] On Behalf Of Marcus J. Ranum
>Sent: Wednesday, August 18, 2004 1:24 PM
>To: Jose Nazario; loganalysis@private
>Subject: RE: [logs] most popular reports...?
>
>Jose Nazario wrote:
>>is no one doing any trend analysis (ie fasting rising ports 
>being hit)?
>>all i see listed so far are static "top N" reports.
>
>Most of the folks who are looking at that problem are intensely
>focused on visualization. Take a look at "Therminator" and
>"The spinning cube of potential doom" etc.
>http://www.nersc.gov/nusers/security/TheSpinningCube.php
>http://www.arxiv.org/PS_cache/cond-mat/pdf/0402/0402325.pdf
>Lacope has some cool Therminator stuff in their StealthWatch
>product. I haven't talked to anyone who has practical experience
>with it.
>
>I'm not convinced of the value of such systems outside of the
>cool-factor but it's mostly because I keep seeing them as
>just different ways of accessing the same underlying metaphors
>and presenting them in new ways. The underlying metaphors
>are really moving averages, runs tests, and distances from the
>mean. What we haven't figured out how to do is use them in a
>way that helps, so visualizing is really just a cool way of
>graphically twiddling the "gain" "bass" and "treble" to see
>what comes out.
>
>mjr.  
>
>_______________________________________________
>LogAnalysis mailing list
>LogAnalysis@private
>http://lists.shmoo.com/mailman/listinfo/loganalysis
>
_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis