Re: [logs] Syslog payload format

From: Jean-Francois Zwobada (zwobadaat_private)
Date: Mon Jan 06 2003 - 00:56:58 PST

  • Next message: Robyn Bailey: "[logs] Best practice events to watch for"

    Hi all
    
    
    
    If you dont mind, I'd like to go back to the basics : computers are meant 
    to provide specific functions in a IT system that is supposed to provide 
    services to a customer (sorry, i went back to the basics of the basics :). 
    Therefore, as IT professionnals, our goals are to ensure that :
             - all functions are running when asked to
             - they run in a reasonable manner compared to what is expected by 
    the customer in terms of scheduling, processing time, configuration, etc.
             - they get accurate input data from an authorized requester
             - they provide accurate output data to an authorized destination
    which can be summarised with the terms Availability, Integrity, 
    Confidentiality and Proof. This means that we need to log the execution 
    environment and the environment itself.
    
    Each program should therefore be traced so that we can ensure that, C. I. 
    A. and P. can be achieved for any atomic action to be performed by the 
    program. The issue is that all actions in all applications may have to be 
    logged. Therefore, IMHO, we should focus on verbs corresponding to general 
    action types applied on entities that could either be an application, a 
    process/thread, a function, a particular command in the code (either for 
    debugging purposes or not), allowing a more or less detailed logging 
    activity depending on the purpose of the application and its criticity for 
    the business.
    
    Unlike Russel, I'd propose to work with generic verbs like "start", 
    "action", "end", "context dump", "input data dump", "output data dump" - or 
    something similar - agremented with standardized error codes. The usage of 
    such verbs would then be application-dependent, with additional fields to 
    describe the exact event, all fields could be described in a XML DTD, for 
    example... The most important, IMHO, being to share common "breakpoints" to 
    describe the entity execution steps.
    The origin of the message should be specified to take into account the fact 
    that it can be an application, a process/thread or a function, therefore 
    some tree structure would be possible to define all entities forming an IT 
    system and we'll finally come up with a stateful logging system. That's 
    maybe too ambitious, but this could be implemented progressively.
    
    I've been involved in IT security activities and the most frustrating 
    aspect of managing security - and furthermore managing incident response 
    processes - is the awfull business of collecting and correlating logged 
    events (when they are logged) among many log files in various formats 
    (firewalls, web server, application server, database log files, etc.). I'd 
    be really willing to get a standard for the logs as we have with SNMP and 
    MIBs for the administration of equipments. Something that could 
    progressively be extended to all kinds of applications needed to run an IT 
    system : network, security, systems, storage, applications, databases in 
    order to collect and correlate all "interesting events" that will allow us 
    to solve technical, legal or contractual issues. In addition, this would 
    probably change the status of logs to proof, as we'll get a "unique trace 
    log" easy to sign.
    
    It's probably too ambitious to be something other than a dream ... ;)
    
    Regards
    
    Jean-Francois
    
    At 14:36 03/01/2003 +1300, Russell Fulton wrote:
    >On Tue, 2002-12-31 at 15:11, Tina Bird wrote:
    > > And (reiterates the moderator, who's getting tired of slogging this dead
    > > horse)...
    > >
    > > I still maintain that it's pointless to worry about how to format the
    > > messages or transport the messages until you've got at least >some<
    > > guidance about what kinds of information (or events) ought to be recorded
    > > in the first place!
    > >
    >
    >Anything and everything!  sigh...
    >
    >We log things for lots of reasons:
    >
    >1/ to provide audit trails of who did what and when.
    >2/ to provide background information about the state of the world (eg.
    >resource usage).
    >3/ to record unusual or potentially damaging events.
    >4/ to record program malfunction or invalid input
    >5/ to provide debugging information
    >6/ to provide a record on all events in some domain (eg. argus IP audit
    >tool).
    >
    >the list just goes on and on.
    >
    >Even within one system what you log will depend on your particular
    >interest and how much you a willing to pay to record the information.
    >
    >That said all applications do have some needs in common:
    >
    >1/ startup and shutdown info
    >2/ configuration changes
    >3/ abnormal program condition
    >4/ abnormal input
    >5/ resource usage
    >6/ resource exhaustion ( covered by 3?)
    >......
    >
    >for transaction based system
    >1/ source of transaction
    >2/ authentication information
    >3/ transaction details
    >4/ completion code
    >....
    >
    >How comprehensive a list do you want to come up with Tina?  I have this
    >idea of something no longer that a couple of pages of text (including a
    >brief introduction) which lists some of the key things to be logged and
    >why.  Is this what you have in mind?
    >
    >I'm not sure if a mailing list is the right tool to to this sort of
    >thing.  We need some sort of collaborative tool that will allow people
    >to add things to lists, preferably something driven through a browser.
    >I think this is at least one reason why Tina is not getting much
    >response to her prompting to address this fundamental issue.
    >
    >Any ideas?
    >
    >This is the sort of thing that is best done face to face in a room with
    >a *big* whiteboard (and lots of beer and pizza!).  LISA bof would seem
    >like a good venue - pity I wouldn't be able to make it!
    >
    >--
    >Russell Fulton, Computer and Network Security Officer
    >The University of Auckland,  New Zealand
    >
    >"It aint necessarily so"  - Gershwin
    >
    >_______________________________________________
    >LogAnalysis mailing list
    >LogAnalysisat_private
    >http://lists.shmoo.com/mailman/listinfo/loganalysis
    
    Jean-Francois Zwobada
    Chief Technical Officer - BT Fluxus
    Phone : +33.1.44.97.70.00 - Fax : +33.1.44.97.70.14
    30, rue du Chateau des Rentiers - 75013 PARIS 
    
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Mon Jan 06 2003 - 19:50:05 PST