CRIME/OCATE Meeting, 14 Feb @12:30@OCATE

GeorgeH@private

Folks,

Speaker: Dr. John McHugh from CERT. This is a 'must' attend--a fitting
Valentine's treat for our group.  See you there!

Geo
_____________________

Title: Evaluating IDS Systems (Why testing Security Software is Hard)

Topic: In 1998 (and again in 1999), the Lincoln Laboratory of MIT
conducted a comparative evaluation of Intrusion Detection Systems (IDSs)
developed under DARPA funding. While this evaluation represents a
significant and monumental undertaking, there are a number of issues
associated with its design and execution that remain questionable. The
difficulties associated with this evaluation have been the subject of
several papers and a number of presentations. As a result of our
investigations of Lincoln's efforts, we have been attempting to develop
an appropriate framework in which similar, but meaningful and useful,
evaluations can be performed. This talk will contrast our proposed
approach with the work that Lincoln performed (and is continuing to
perform). Our primary conclusion for signature based systems are that
the we simply do not know enough to generate appropriate artificial
background data for false alarm evaluation, but that there are a
systematic approaches to measuring true positive and negative
performance, under both ideal and appropriate environmental stress
conditions. The situation is much less clear for with respect to anomaly
based systems since the relationships between anomalous and intrusive
behavior are poorly understood. In both areas, there is a paucity of
theory that can be applied to the problem and we feel that the ad hoc
and intuitive approaches that characterize today's efforts may be
nearing their limits.

Dr. McHugh's CV:  John McHugh is a member of the technical staff at
CERT, part of the SEI at CMU. He was a professor and former chairman of
the Computer Science Department at Portland State University in
Portland, Oregon where he held a Tektronix Professorship. His research
interests include computer security, software engineering, and
programming languages. He has previously taught at The University of
North Carolina and at Duke University. He has been an active researcher
in the application of formal methods to the construction of dependable
and secure systems for many years. He was the architect of the Gypsy
code optimizer and the Gypsy Covert Channel Analysis tool. Dr. McHugh
received his PhD degree in computer science from the University of Texas
at Austin. He has a MS degree in computer science from the University of
Maryland, and a BS degree in physics from Duke University. He grew up in
Durham, North Carolina, leaving when he graduated from Duke. Twenty
years later, he returned, demonstrating that Thomas Wolfe was wrong.
After another ten years in Durham, he moved to Portland, demonstrating,
perhaps, that Wolfe knew what he was talking about after all.
_______________________________________________
Nw-ipwg mailing list
Nw-ipwg@private
http://lists.whiteknighthackers.com/mailman/listinfo/nw-ipwg