RE: False-negatives in several Vulnerability Assessment tools

From: Craig H. Rowland (crowlandat_private)
Date: Thu Apr 17 2003 - 10:28:43 PDT

  • Next message: Chris McNab: "Re: LC4 (L0phtCrack) error "Couldn't open SAM\Domains\Account\Users in SAM file. Possibly improper format.""

    > >My current employer, which is a Fortune 10 company, shall be
    > >referred to as "Ralph Co."  I've been with Ralph Co for 2 years now. 
    > >Our security there is relatively pathetic.  I have had to go to 
    > >upper managment because our security manager will run a scan at 
    > >random and decide a given service needs to be terminated because the 
    > >scanning tool that he's demo-ing that week says that it's a 
    > >"critical vulnerablity".  I have had to try to explain to him 
    > >several times that he pays us a lot of money to exercise our 
    > >professional judegement in verifying what is and is not a real 
    > >vulerablity.  His answer is that "The tool says so, so it must be."
    > 
    > The nadir of this process was him insisting that we shut down a "Code 
    > Red Infected Server".  Too bad it turned to out be a developers Apple 
    > iBook.
    > 
    > My point with all this is what you do with the scans AFTER you run 
    > them.  If you want intelligent analysis of the report, you get a 
    > security professional that knows how to check things manually and 
    > knows when output from the scanner looks dubious.  Any reasonably 
    > intelligent person can operate the scanner software and print out the 
    > report when its done.  The skill and expertise comes in interpreting 
    > the output and making meaningful suggestions that actually improve 
    > security.
    
    Exactly. When you go to the hospital for a broken bone you have a X-Ray
    technician operate the machine, and an experienced radiologist who
    interprets the results. They don't simply hand you the X-Ray for
    personal interpretation and the bill. 
    
    This is an important point that is frequently overlooked. I've seen a
    number of audits that were paid for by customers and consisted of
    nothing more than a nicely bound printout of a commercial scanner with
    almost no interpretation. Personally, I think this is a serious breach
    of responsibility. 
    
    The results of a scanner can be misleading if you don't have a good
    knowledge of common vulnerabilities, commonly affected hosts, and
    patterns indicating misuse. Expecting a scanner alone to identify 100%
    of all threats is not practical for several reasons:
    
    1) The author of the vulnerability check may have written it
    incorrectly. Or, more likely, it worked in their testlab environment but
    failed out in the field for a variety of reasons.
    
    2) Performing an exhaustive scan against all the systems in a large
    enterprise is usually not feasible due to network constraints, stability
    of the backbone and scanned systems, and the dynamic nature of network
    deployments (wireless, DHCP, etc.).
    
    3) The scanner does not have an internal view of the host being audited
    and can miss critical mis-configurations that result in an insecure
    setup, but appear "secure" from the outside with automation.
    
    I guess my point in all this is that proper interpretation of security
    tool results is critical. As much as the security industry would like to
    have the software do everything for the inexperienced user, it just
    isn't practical or advisable given the nature and seriousness of this
    business. 
    
    -- Craig
    
    Opinions are my own. There is no endorsement of the (random)
    advertisement appended to this message.
    
    
    ---------------------------------------------------------------------------
    Attend Black Hat Briefings & Training Europe, May 12-15 in Amsterdam, the 
    world's premier event for IT and network security experts.  The two-day 
    Training features 6 hand-on courses on May 12-13 taught by professionals.  
    The two-day Briefings on May 14-15 features 24 top speakers with no vendor 
    sales pitches.  Deadline for the best rates is April 25.  Register today to 
    ensure your place.  http://www.securityfocus.com/BlackHat-pen-test 
    ----------------------------------------------------------------------------
    



    This archive was generated by hypermail 2b30 : Thu Apr 17 2003 - 12:37:50 PDT