RE: Re[2]: [logs] Logging: World Domination

From: Ogle Ron (Rennes) (Ron.Ogleat_private)
Date: Thu Aug 22 2002 - 02:45:52 PDT

  • Next message: Björn Österman: "RE: [logs] Thoughts on log normalization"

    Unless, you get rid of UDP totally for a transport for log messages
    (completely destroying backward syslog compatibility), you are stuck with a
    definite length log message.  That means the more things you put in the
    message to help us poor humans out, the less space is available for
    content/message which is the reason for logging in the first place.
    
    Computers don't care one bit about the format, you just have to program them
    to interpret the bytes.  One of the prime objectives for logging is to get
    data for analysis from the target system.  These logs can be "beautified" at
    a later date with tools such as XML, but they should NOT be "beautified"
    during creation and movement to an off-target repository.
    
    XML is good for supporting a large variation in data types with varying
    structure.  Logs are very structured with little or no variation in data
    types.  The log content could be varied, but again it is still well
    structured.  Therefore, XML is a very poor design choice when creating log
    entries.  The tradeoff is less log content for I'm not exactly sure what
    benefit.
    
    Not everything old is bad and not everything new is good and not every
    technology is right for every environment.
    
    Ron Ogle
    Rennes, France
    
    > -----Original Message-----
    > From: Chris Adams [mailto:cadamsat_private]
    > Sent: Thursday, August 22, 2002 12:00 AM
    > To: loganalysisat_private
    > Subject: Re: Re[2]: [logs] Logging: World Domination 
    > 
    > 
    > On Wednesday, August 21, 2002, at 12:05 , Greg Black wrote:
    > > | if you propose something like this and don't use XML, the first 
    > > question
    > > | you're going to get will invariably be "why didn't you use XML?"
    > >
    > > To which a reasonable answer is: "because it sucks."
    ...
    > The size issue becomes a lot less of a problem if you've 
    > designed your 
    > DTD properly (e.g. resisting the urge to be unnecessarily verbose - 
    > <event host="..." timestamp="1234567890"> instead of 
    > <event><ip_hostname>fqdn.example.com</ip_hostname><timestamp>F
    > ri Feb 13 
    > 15:31:30 PST 2009</timestamp>) and are using compression.
    > 
    > The processing time concern is more of a problem but XML parsers have 
    > advanced considerably over the last few years. A well designed DTD 
    > should be surprisingly close to something like the typical 
    > Perl script 
    > which has to parse all of the slightly different variations 
    > of the same 
    > syslog message.
    > 
    > In both cases, neither would be a significant problem even now and 
    > Moore's law suggests this won't change for the worse.
    > 
    > Chris
    > 
    > _______________________________________________
    > LogAnalysis mailing list
    > LogAnalysisat_private
    > https://lists.shmoo.com/mailman/listinfo/loganalysis
    > 
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    https://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Thu Aug 22 2002 - 09:16:03 PDT