Interesting discussion... I understand why various checks won't always be accurate, and may be subject to false positives and false negatives. One of the problems I have with the scanners I've examined, however, is that it would be easier to perform an overall risk assessment if the tools are explicit with some sort of confidence measurement in each of their tests. Some checks may have descriptive text that say "this test is not always reliable," but I don't necessarily see that in all the checks that have reliability problems. Considering the hundreds of checks that most tools have nowadays, it's generally infeasible to hunt through all that descriptive text anyway. A "reliability" field for each check, just like a Risk field, could help me know what results are accurate and what results are questionable. By attaching the Reliability to the check itself as opposed to each individual result for each host, you could avoid the ensuing mountain of data as described by David. I don't mean to turn this into a public wish list, but another thing I'd like to see in auditing tools is a "methodology" description for each check - do you just read banners and compare to known vulnerable versions, or do you perform the exploit? That has some implications on reliability. Also, some checks can have a negative impact on the systems they probe, and yet they aren't labeled "denial of service" (nor should they be; but consider tests that could dump core in /, for example, which may accidentally cause a denial of service). If you're a consultant and you're auditing a network with skittish managers who don't want any adverse impact on their machines at all, knowing the methodology for each check would help you construct an appropriate scan. (The typical "Light," "Medium," and "Heavy" scan configurations provided by some tools may be too coarse in some cases and would require a lot of manual effort to review all the checks). Reliability and Methodology fields, each with say 3-5 unique values, could be effective in informing the user of the inherent limitations in a particular scanner and/or all scanners. This is my two cents, not MITRE's. - Steve Steven Christey | coleyat_private (781)271-3961 The MITRE Corporation | "I'm not really sure what the question is, but 202 Burlington Road | I think the answer is 'I don't know.'" Bedford, MA 01730 | - an anonymous but honest former MITRE employee
This archive was generated by hypermail 2b30 : Fri Apr 13 2001 - 14:34:16 PDT