Re: Stacker performance results

From: Gerrit Huizenga (gh@private)
Date: Wed Mar 16 2005 - 14:40:20 PST

On Wed, 16 Mar 2005 14:55:48 CST, Serge Hallyn wrote:
> On Wed, 2005-03-09 at 11:14 -0500, Stephen Smalley wrote:
> > In the past, we haven't found kernel compile benchmark to be very
> > revealing for SELinux performance analysis.  dbench results would be of
> > interest.  More generally, you might want to repeat the tests done for
> > the AVC RCU work, see 
> > 
> >
> > 
> -- 
> Attached are a set of performance results comparing 2.6.11-rc5 under
> RedHat REL4 on power5 1.5Ghz, 4cpus (smt-enabled=0), 16G RAM, with an
> ext2 filesystem.  -nostack is a kernel with selinux and capability.
> stack is a kernel with stacker, selinux, and cap_stack.
> dbench (run as dbench -c client_plain.txt 4, three times)
> nostack:   872.617 882.029 870.968
> stack:     799.608 798.028 800.122
> hackbench (run as ./hackbench 100)
> nostack:   5.064
> stack:     6.721
> unixbench (full report files are attached)
> nostack:   494.3
> stack:     447.1
> In particular stacker performance impact on tests like the unixbench
> file read and copy tests seem excessive.
> I'm not sure whether to proceed with apachebench, or many runs of
> lmbench.  It seems lmbench may be more helpful in pointing toward the
> culprit.

Hi Serge, do you happen to have oprofile data for these runs?

Also, dbench is not very stable and actually performs better in
some cases when the system is running "unfairly" - hence most of
the community does not trust dbench all that much.

However, in all cases, your numbers indicate a significant, measurable,
large regression.  I wouldn't have expected so much degradation from
a simple stacker.

If you have a friend in austin who can help you submit a job to
the internal ABAT harness with your patches, enable oprofile data
collection, and run a set of tests such as:

bonnie	Performs a number of simple tests of hard drive and file system

chat	chat is a client/server benchmark that simulates a chat server.

contest Designed to test system responsiveness by running kernel
	compilation under a number of different load conditions.

fsx-linux	Exercises an FS by creating, opening, writing, reading,
	validating, closing, and unlinking a test file.

kernbench	Workload generated by compiling kernels

lmbench Simple bandwidth and latency benchmarks

ltp	A collection of micro-testcases

postmark	Simulates load generated by enterprise applications such
	as email, news and web-based commerce

reaim	A multiuser benchmark that tests and measures the performance
	of open system multiuser computers.

sdet	Workload created by parallel execution of common UNIX commands

specjbb Simulation of a Java powered e-commerce website

tbench	Simulates a Samba server _without_ filesystem calls

volanomark	Client/Server benchmark simulating a chat room server

Bonnie, contest, fsx-linux, postmark, reaim and sdet might be good
choices.  Enabling oprofile will collect profiling information while
those tests run.  If you need help interpreting results after you've
run these, let me know.


This archive was generated by hypermail 2.1.3 : Wed Mar 16 2005 - 14:53:41 PST