Re: Security Benchmarks

From: Serge E. Hallyn (hallyn@private)
Date: Wed Mar 09 2005 - 18:43:00 PST


My plan is to use unixbench, dbench, stream, and hackbench.  Webstone
requires two machines, but people seem to respect it's results, so maybe
I'll just have to give in and use it.  I suppose I could try apachebench
as well, I'll just need to set apache back up...

(I actually haven't found how to get meaningful results from dbench,
but I didn't try very hard)

But I am still curious which of the above (or any not listed) are
considered more useful.  Though I'm assuming it's mostly that those
numbers will be more meaningful because they've been used before for
selinux performance comparisons?

thanks,
-serge

Quoting Crispin Cowan (crispin@private):
> Stephen Smalley wrote:
> 
> >In the past, we haven't found kernel compile benchmark to be very
> >revealing for SELinux performance analysis.  dbench results would be of
> >interest.  More generally, you might want to repeat the tests done for
> >the AVC RCU work, see 
> >
> >http://marc.theaimsgroup.com/?l=linux-kernel&m=110054824004161&w=2
> > 
> >
> Hmm. What do you mean that kernel compile is "not very revealing" for 
> SELinux performance analysis?
> 
> To be sure, kernel compile (khernelstone :) is a macrobenchmark, and 
> will not give you high-resolution data on the cost imposed on individual 
> operations. For microbenchmarks, I prefer lmbench.
> 
> For macrobenchmarks, you want something that does lots of stuff, so that 
> the run times are long enough to measure, and does the right mix of 
> stuff, so that the workload is representative for "typical" workloads. 
> Of course, "typical" is in the eye of the beholder.
> 
> At Immunix, we tend to use 2 macrobenchmarks: khernelstone, and 
> Webstone. Kernel compile does lots of everything (CPU, disk, and memory) 
> presenting a heavy workload, and thus *should* make a good benchmark, 
> unless the workload mix is somehow wrong.
> 
> Stephen, what do you find wrong about the workload mix in kernel 
> compiles? What would you suggest instead?
> 
> Thanks,
>    Crispin
> 
> -- 
> Crispin Cowan, Ph.D.  http://immunix.com/~crispin/
> CTO, Immunix          http://immunix.com



This archive was generated by hypermail 2.1.3 : Wed Mar 09 2005 - 18:44:11 PST