Re: Security Benchmarks

From: Crispin Cowan (crispin@private)
Date: Wed Mar 09 2005 - 23:13:02 PST


Serge E. Hallyn wrote:

>My plan is to use unixbench, dbench, stream, and hackbench.  Webstone
>  
>
I have never heard of any of them before today.

Webstone is by far my favorite, as it is highly representative of an 
actual workload that requires real protection.

>But I am still curious which of the above (or any not listed) are
>considered more useful.
>
I seriously consider the kernel compile to to be a useful benchmark. It 
is to the most precise in the world, and likely over-represents 
computation vs. I/O, but it does access a massive amount of files and a 
massive amount of memory, so I think it is reasonable. It also has the 
advantage of being familiar and pre-configured to work on most 
developer's machines.

The other macrobenchmark I consider to have serious respectability is 
SPEC <http://www.spec.org/>. But I have never tried to use it because 
the setup effort is high, and it is non-free 
<http://www.spec.org/order.html>.

Finally, for compiler-based benchmarks on things like StackGuard, I have 
used SSH throughput through the loopback interface as an ad hoc test of 
the impact on computational workload for an actual network daemon that 
needs protection.

The benchmarks used in the original LSM paper were:

    * lmbench
    * kernel compile
    * webstone

Crispin

-- 
Crispin Cowan, Ph.D.  http://immunix.com/~crispin/
CTO, Immunix          http://immunix.com



This archive was generated by hypermail 2.1.3 : Wed Mar 09 2005 - 23:13:41 PST