[logs] IPFC Re: [logs] Converting log messages to XML format.

From: Alexandre Dulaunoy (alexat_private)
Date: Sun Nov 11 2001 - 02:00:00 PST

  • Next message: Frank Solomon: "Re: [logs] Converting log messages to XML format."

    Dear all,
    
    I received a lot of comments for issue for our project IPFC and by
    extension to XML format.
    
    I will made a basic description of software :
    
    * IPFC is free software (released under the GNU.GPL license)
      (all source code are available for auditing 8-))
    
    * IPFC is a complete framework divide in three major parts (wrapper,
    dr-server, db-backend)
    
      - Wrapper (it's a small programm that getting information from FIFO,
        syslog files, application (for example : checkpoint FW1,...))
        formatting into XML and sending it to the dr-server. (it's more than that but it's a
        simple explanantion).
      - Dr-server is a bi-directionnal XML server based on Apache.
      - Db-backend is a generic SQL-92 database containing and collapsing all
        the data in a generic way.
    
    * The exchange are simple and secure between the three zones. (only
    TLS/SSL)
    
    * The XML format is only used for data communication and on the dr-server.
      The XML are converted into the database. (response to Mister Buttherworth)
      We can do generic queries into the databse like that :
    
      select url,count(*) from apache_access_log_1
      where sensor_id = $sensor and bytes_sent = 0
      group by url
      order by count(*) desc;
    
      That gives the top urls which did not send any data (useful for
      redirects or http issue)
      You can do a lot of thing like that and more in one SQL statement.
    
    You can also regenerate all the logs in the standard format of the
    source (W3C logs from apache, checkpoint FW1 ascii logs, syslog...) and
    use your favorite reporting tool.
    
    So IPFC is more than a simple XML converter, is a complete open source
    framework to manage security modules.
    
    We plan to make a release 1.0 for the 15 of december.
    
    But can also checkout the current development code from the CVS :
    
    cvs -d:pserver:anonymousat_private:/cvsroot/ipfc login
    (press enter when there is a password prompt)
    
    cvs -z3 -d:pserver:anonymousat_private:/cvsroot/ipfc co .
    
    http://www.sourceforge.net/projects/ipfc/
    http://www.foo.be/ipfc/
    
    If you have any comment or want to participate into the project don't
    hesitate.
    
    Thanks a lot.
    
    Alexandre Dulaunoy (aka adulau)
    
    
    
    On Sat, 10 Nov 2001, BrandonButterworth wrote:
    
    > > We encapsulate syslog, checkpoint, .... into a standard
    > > and simple XML format.
    >
    > The only bad thing about XML is that people try and fit
    > it to everything
    >
    > > <?xml version='1' standalone='yes'?>
    > > <ipfc>
    > >   <smod>
    >  ....
    >
    > So what was simple and concise becomes complex and
    > massive. Thats a lot of extra storage just to repeat
    > those same tags over and over and extra cpu to process.
    >
    > It also means it becomes human unreadable and not suitable for simple
    > quick manipulation (if you grep for a field you don't get the rest of
    > the data as it's spread out over many lines) so you have to rely on the
    > tools that made it which may be fooled by an attacker that knows their
    > limitations or only pick up what the author though you'd be looking for.
    >
    > Data should only be handled as data and not have embedded code
    > that is executed, mixing the instructions for interpreting the data
    > and the data itself is a risk I don't think is worth the gain.
    >
    > regards
    > brandon
    >
    >
    >
    > ---------------------------------------------------------------------
    > To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    > For additional commands, e-mail: loganalysis-helpat_private
    >
    
    -- 
    ---
    Alexandre Dulaunoy			adulauat_private
    					http://www.conostix.com/
    
    
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    For additional commands, e-mail: loganalysis-helpat_private
    



    This archive was generated by hypermail 2b30 : Sun Nov 11 2001 - 16:53:14 PST