Re: [logs] Appliancebased Logging

From: Jian Zhen (Jian.Zhenat_private)
Date: Tue May 06 2003 - 16:58:55 PDT

  • Next message: Darren Reed: "Re: [logs] tbird rant: syslog.conf, tabs and spaces"

    [Disclaimer: I do not work for any of the companies I mentioned below.
    I am not endorsing any of their products. I speak for myself, not as
    a representative of my employer. Anything I say here is my own opinion
    and no one else's.]
    
    a bit of a late response..
    
    I went thru a similar phase in one of my projs, looking at various
    event logging/correlation products...specifically, I've looked at
    Network Intelligence, ArcSight, Intellitactics, and serveral others.
    
    The most interesting ones are ArcSight and Intellitactics IMHO.
    
    Intellitactics can use Addamark for backend storage. ArcSight requires
    some type of RDMS.
    
    If you just want log storage, Addamark may not be a bad solution and it's
    relatively cheap comparing to a full RDBMS. [Again, I don't work for them,
    their solution just seem quite interesting.] They use the file system as
    storage, so basically u collect logs for 5 mins, and then bulk load them
    into their file-based distributed DB. I think they claim to have extremely
    high bulk-loading rate. So you are not exactly inserting into the DB all
    the time.
    
    Just for comparison, I have ran tests on a E420 (2gb mem, 4xproc), running
    MySQL. 10 threads of bulk loading (read, NOT database insertion by running
    insert into), into MySQL, each can load about .8k-2k records per sec. 
    
    Addamark's demo showed a much higher rate. Test using your own data obviously. 
    Results will vary depending on what you are trying to do.
    
    My experience with NI is that their stuff's not as flexible and highly
    depend on the insertion rate of MSSQL (I think that's what they use).
    One can easily write a small C program that can receive a large # of 
    log entries, (I have not tested it for 6k lines, but my current log
    receiver (to file) can receive 3-4k with room to spare), but it's much
    more difficult to get a RDMBS to insert at 6k/sec (I get a consistent 
    800-1000 inserts/sec with MySQL and like 400-500 with PostgreSQL, performance
    will obviously vary depending on how many records i do per insert).
    
    The user interface is also much weaker, tho I looked at them awhile back 
    so it was pro'ly an old version.
    
    If you want event/log correlation, both ArcSight and Intellitactics are
    probably the leaders in the market. I really think that whoever you choose
    will depend on what you need. Other event corr vendors are much less
    impressive.
    
    Now the bad part, the event corr vendors are all VERY expensive. 
    
    have fun looking at the vendors... :)
    
    
    Robert van den Breemen (rvdbreemenat_private) [030316 09:16]:
    > Hi everyone,
    > Yesterday I had a presentation by Network Intelligence Corparation
    > (http://www.network-intelligence.com) of their product suite. I was
    > wondering if anyone on this list can share their experiences with this
    > productline. It seems to be quite a total solution for implementing a
    > logging infrastructure, including reporting & event correlation...
    > 
    > Product is called: Envision.
    > The appliance seems to have a high sustaioned performance of up to 6000
    > events per seconds (loglines per second)...
    >  
    > Anyone any experience, it seems to be a company that focussed on the states.
    > Greetings,
    > Robert
    >  
    > PS. Other products in their shop are: private I for example, which ships
    > with PIX I think.
    > 
    > --
    >       _///_    
    >      /(@ @)\   
    > ==o00o=(_)=o0oo==[ Robert van den Breemen ]====
    > 
    > 
    >  
    --
    Jian Zhen <jian.zhenat_private> [408.884.6826]
    Manager, Defensive Security Development
    Managed Network & Security Development
    Cable & Wireless
    PGP Signature: 8C9B 7A4B 4E18 AE24 BAA7  AD56 9111 B392 9B32 BE4A
    _______________________________________________
    LogAnalysis mailing list
    LogAnalysisat_private
    http://lists.shmoo.com/mailman/listinfo/loganalysis
    



    This archive was generated by hypermail 2b30 : Tue May 06 2003 - 18:36:28 PDT