Re: [logs] Syslog payload format

From: wolfgangat_private
Date: Thu Jan 02 2003 - 04:56:41 PST

  • Next message: Tevfik Karagulle: "Re: [logs] Syslog payload format"

    Tina Bird wrote:
    > [..]
    > What I'm trying to assemble is a list of the events on an individual
    > host/operating system, or an application, or across a network, that are
    > the most important for keeping things running smoothly.  Or, the events
    > that define "normal behavior" for a host, an application or a network.
    > (They might not be the same lists.)
    
    > Sometimes I think I must be missing something really obvious.  I don't
    > understand how to discuss a reasonable format for messages if I don't have
    > >some< idea of what sorts of information (values) and events I want those
    > messages to describe!
    
    > Back in the mists of time we'd assembled the start of a list of important
    > events (which at the time I introduced as "state changes" which launched
    > us into another couple of unrelated tangents).  I have a start of a list
    > of "events that define normal" based on responses from students over the
    > years.
    > [..]
    
    Maybe we should set our goals even lower for a start: What I want to
    get out of my logs is information if a) any of my hosts or applications
    is not running smoothly (e.g. because of hardware problems or not
    enough resources) and if b) anyone is trying to do something that he should
    not. 
    
    Problems of the a) type can usually be seen in a single log entry, the
    only thing an application programmer could do to make our lives easier
    is flag that message to mark it as "resource/hardware problem".
    
    To catch problems of the b) type I want to be able to collect statistics
    on the correlation between time, hosts and user ids so unusual values
    could raise an automatic alarm to start a manual investigation.
    I think there is only a limited number of things to log that are of
    a practical value for this kind of investigation. To name the ones I can
    think of right now:
    
    + Type of event: network connection, authentication event,
                   application start/restart/shutdown, hardware/resource problem
       all of these in flavors (ok/failed) as applicable
    
    + User IDs
    
    + Host IDs
      additional information could be e.g. the role of src or dst host
    
    + protocol information
      e.g. port numbers, tcp/udp/icmp/what_ever
    
    + system error messages
      (e.g. errno and corresponding error string)
    
    In addition every log event should contain a clear text message for the
    human reader.
    
    I don't think we need to do much for application specific data, as any
    application specific logs will need their own specific log analyzing tools
    anyway. Let's concentrate on the non-specific events instead.
    
    -- 
    Wolfgang Zenker                                  Mail: W.Zenkerat_private
    JPAVES Unix Online GmbH                          Fon:  (+49) 721 / 955 40 60
    Kaiserallee 87                                   Fax:  (+49) 721 / 955 40 62
    D-76185 Karlsruhe                                Web:  www.jpaves.com
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Thu Jan 02 2003 - 08:16:00 PST