[logs] Re: on log standards

morty@private

On Fri, Aug 25, 2006 at 10:57:30PM -0500, Anton Chuvakin wrote:

> So I was thinking a lot about log standards and taxonomies and the
> release of CEF inspired me to finally finish my brief article on log
> standards - check it out:

> http://chuvakin.blogspot.com/2006/08/on-common-event-format-cef.html

Comments: 

* There is one standard for log content that is in widespread use: the
  MIBs used for SNMP traps.  SNMP traps are not used too widely in the
  host world, but they're fairly widespread in the network world.  The
  SNMP world had standardization from day one, so it's useful to look
  at how standardization has helped SNMP.

  Note: SNMP is more than just traps/events, but we can ignore the
  other aspects for this discussion.

* Even in the SNMP trap world, the standardizations often prove
  insufficient.  That is, there are multiple standardized components:
  there is a standard way to describe SNMP PDUs in machine-readable
  format, i.e. ASN.1 and SMI to write MIBs, and then there are
  actually standard MIBs, so that all platforms can express certain
  common events in a vendor-independent way.  The former is mostly
  what you would call a form standard, while the latter is mostly what
  you would call a content standard.  But the content component is, in
  practice, of limited utility, because most vendors end up wanting
  various events that are not standard events.

  In more concrete terms:

  * There is a standard way to express "interface down".  Any device,
    regardless of vendor, can send an SNMP trap saying "interface
    down", and the NMS (network management station) can understand it
    without knowing anything about that particular vendor.

  * But if a device wants to say "SONET problem with certain
    vendor-specific flags", it is likely that no standard trap exists.
    The vendor is going to utilize a custom trap.  SNMP allows for
    this.  This custom trap can be defined using a MIB written in
    standard format, so the NMS station can read in the MIB and then
    immediately parse it, but actually understanding what to do with
    it would still require trap-specific handling by the application
    or by the administrator.

  SNMP provides vendor-specific traps for a very good reason.  While
  the standard MIBs provide a lot of useful traps, they cannot begin
  to cover all the possible cases of existing technologies.  New
  technologies that vendors use to differentiate themselves from each
  other with necessarily exist and need management before they are
  standardized.

  The lesson of SNMP trap is that having standard content can be
  useful, but in practice, sooner or later, you will have to allow for
  a standard format, and let vendors extend the content.

* In the host software world, life is worse than in the network world.
  Host software is often created in an unstructured way, with ad-hoc
  logging infrastructure that is added during debugging stages.  Log
  content standards sound great, but in practice, they would probably
  not work well with the way software is actually written.

* The release of "CEF" seems like a non-event.  Looks like a
  unilateral "standard" issued by one minor vendor without a lot of
  buy-in from third-party vendors.  They haven't even really released
  it; they're asking people to send them email to get a copy.  No
  thanks.

Morty
_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis