As a few of the folks on the list know, I'm in the process
of writing a book on log analysis, which was originally going to be a
tutorial-style introduction to the topic. Some changes at my publisher,
however, have resulted in my getting a new editor, and we've decided to
change the focus of the book to target a more advanced audience and
discuss enterprise logging issues, which means that I now get to write
about a lot of the cool things that I had originally wanted to write
about. The problem I'm running into, though, is that there are too many
cool topics to discuss, which is why I'm turning to you all, to find out
what topics you would want to learn more about. Here's a very rough
sketch of the chapters as I envision them right now; I had some problems
exporting this info from my outlining program, so not all of the items
at the same level in this outline will be at the same level in the book,
but all of the items with a left-flush asterisk will probably be
individual chapters, so it gives a reasonable idea of how much coverage
each topic will receive:
Intro
* Motivation
* Legal Issues
Data Management
* Data Management Issues
* Performance
* Reliability
* Integrity
* Authenticity
Local Data Collection
* Syslog to file
* Application Logfiles
* Event Log
Data Aggregation
* Aggregation Filtering
* Syslog over network
* Generic Network Logger
* Logfile pull/push
* SNMP
* Not necessarily aggregation but over the network
* Aggregation Gateway host(s)
* Time Sync
* NTP
* SNTP
Data Storage
* Storage Strategies
* Queuing Storage
* Analytical Storage
* Archival Storage
* Storage Rotation
* Flat files
* Local disk flat files
* Removable media flat files
* Tape Archival
* WORM media
* Flat File rotation
* Compression
* Sample rotation scripts/schemes
* Databases
* Disclaimer of incompleteness
* Data Management issues as they apply to databases
* MySQL
* Oracle
* MS SQL Server (?)
* Dedicated Log Repositories (?)
* Addamark
* NFR Secure Log Repository
Data Analysis
* Data Analysis Issues
* Meaningfulness
* Performance
* Browsing
* GUIs (?)
* Filtering
* Positive vs. Negative Filtering
* Regex Issues
Trending
* Historical Analysis
* Statistical Analysis
Correlation
* Procedural Correlation
* Expert Systems
* CLIPS
* Rule-based Analysis
* Object Classification Analysis
* Unsupervised
* Supervised (Neural Net, Instance-based/Nearest Neighbour, Decision Tree)
Response
* Alerting
* Distillation/Reduction
Of these topics, which ones would people find most useful?
Which ones would be least useful? Are there any specific questions
about any of these areas that people would want answered? (Rest assured
that I'll be posting whitepapers based on drafts of some of these
chapters, so at least some of those questions will be answered without
having to buy the book. (Although people buying the book would be
appreciated, of course. ;) )) There are some topics like logging via
serial lines rather than in-band networks that are probably going to get
a cursory treatment at best, because of space considerations (I'll
probably have to trim a few chapters as it is); if you feel that some
topic like that should definitely get more coverage, let me know, so I
can reprioritize.
Also, what about applications of interest? I can't cover
every app out there, so if people want to vote for which apps for which
they'd most like to see coverage, that would be appreciated; in
particular, are there other (free) expert systems frameworks (besides
CLIPS) or different dedicated log repositories that people would want to
hear about?
Similarly, what OSes are people dealing with? Most of
the log analysts whom I know who are dealing with Windows logs at an
enterprise level tend to be most interested in stuffing Event Log data
into some other format like syslog or a database; how much interest is
there in in-depth discussions of "native" Windows log analysis?
I'm going to do as much benchmarking of
things like reasonable throughput rates (a la the recent Apache
Logs/syslog thread) as time allows; now's the time to let me know about
particular areas or implementations for which people would like to see
benchmarks.
So as to not clutter up the list, please reply to me
directly, and I'll post a summary.
Thanks,
Sweth.
--
Sweth Chandramouli ; <svcat_private>
President, Idiopathic Systems Consulting
---------------------------------------------------------------------
To unsubscribe, e-mail: loganalysis-unsubscribeat_private
For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Wed Feb 20 2002 - 13:11:31 PST