[ISN] Are there perils in penetration testing?

From: InfoSec News (alerts@private)
Date: Sun Feb 25 2007 - 23:15:27 PST


By John Pulley
Feb. 26, 2007

Are there perils in penetration testing?

Yes, but calm heads can help you choose whether to outsource or do it 

The New York State Office of Cyber Security and Critical Infrastructure 
labored for several years to protect its networks and information 
technology assets with layers of security. It tightened existing IT 
policies, procedures and practices and instituted new ones, conducted 
gap analyses and established plans for mitigating security breaches.

Since then, the state has conducted regular scans to detect IT 
vulnerabilities the equivalent of looking for frayed threads that can 
unravel and leave an agencys networks and systems exposed.

The state will soon throw another protective blanket onto the pile. For 
the first time, it will try to hack into its own systems. For IT 
professionals, penetration testing is the ultimate security measure.

Penetration testing goes beyond tapping at the door, said William 
Pelgrin, chief cybersecurity officer of New Yorks Cyber Security and 
Critical Infrastructure Coordination. Its breaking through the door.

New York officials havent decided who they will let try to break through 
the door. Until recently, penetration testing was the exclusive purview 
of highly skilled technicians who employed extensive toolkits of 
specialized programs to probe and exploit system and network weaknesses. 
A deep-dive penetration test could take weeks to complete and cost 
hundreds of thousands of dollars.

But new automated tools promise to do the job more quickly and at less 
expense. Among those is Core Security Technologies Core Impact, which 
the company says can run a meaningful penetration test in a few hours. 
The company charges an annual fee of $25,000 for an unlimited-use 
license. State and local chief information officers and chief 
information security officers say they must evaluate the pros and cons 
of the new software options, including open-source applications such as 

A growth industry

So when does it make sense to use an automated penetration test?

Advocates of the new tools say the applications give in-house security 
professionals more control, including the ability to perform penetration 
tests as often as they want. Critics of automated tools say they are a 
poor substitute for a thorough and nuanced manual test that a skilled 
practitioner performs. Most experts agree, however, that an automated 
penetration test in the hands of an untrained novice could do more harm 
than good.

A fool with a tool is still a fool, said Bill Harrod, a security 
management consultant at CA, formerly Computer Associates.

Penetration testing and other types of IT security assessments are a 
growing industry. In a world made increasingly unsafe by identity theft, 
online rip-offs and other cybercrimes, IT security pros are under 
pressure to fortify systems and networks and protect information assets. 
A growing number of federal regulations require public- and 
private-sector CIOs to harden their systems and networks against 
external and internal threats.

Vulnerabilities are increasing exponentially. The number reached 5,990 
in 2005, according to Carnegie Mellon Universitys Computer Emergency 
Response Team Coordination Center, an increase from 171 vulnerabilities 
reported a decade earlier.

First line of defense

Every vulnerability represents a potential security risk. Penetration 
testing determines whether the risk can be exploited. If a systems 
owners can break in, an unauthorized hacker can, too.

Either you, the owner, can find them, or the hacker can find them, but 
they will be found, said John Carpenter, a product manager at DevPartner 
SecurityChecker, an automated application security test Compuware 

A vulnerability scan is a first line of defense. It detects missing 
security patches and spots other vulnerabilities that could potentially 
compromise networks or systems. But scans often produce reams of data 
that show false positives and identify vulnerabilities that, for 
technical or practical reasons, no one could exploit anyway.

Penetration tests pick up where vulnerability scans leave off. Both 
manual and automated penetration tests try to exploit network 
vulnerabilities to determine whether they afford an opportunity for 
hackers to take over computer systems, gain access to private data or 
disrupt networks.

Armed with a Web browser and a proxy device, Brad MacKenzie, director of 
IBMs X Force Penetration Test Team, recalled hacking into the network of 
a state prison system and gaining access to detailed records of current 
and former prisoners.

It was as if we were sitting in a branch office of the states prison 
system, MacKenzie said.

South Carolinas strategy

Allowing a consultant to hack into your system and see sensitive data in 
the name of security makes no sense to James MacDougall, chief 
information security officer of South Carolina, which uses Core Impact 
for in-house penetration testing.

We have outsourced some penetration testing, but I didnt think it was 
wise to outsource the testing of critical infrastructure to a vendor, 
MacDougall said. We thought we should arm ourselves.

Government procurement rules requiring agencies to select low-bid 
vendors can undermine the confidence of users, including law-enforcement 
agencies that dont want their data outside on the street, MacDougall 

But detractors of automated penetration testing say the applications can 
give agencies a false sense of security. In reality, the tests are only 
as effective as the people running them. The best tools are unable to 
discern the intent of hackers or reliably prioritize vulnerabilities.

Officials must remember, MacKenzie said, that an automated tool is 
always a step behind the elite attackers exploits.

But not far behind, said Max Caceres, director of product management at 
Core Security Technologies, which develops new commercial-grade exploits 
that mimic what the bad guys do. We continually update the product with 
new attacks every week, Caceres said.

Delawares approach

Delawares Department of Technology and Information has been using Core 
Impact for about a year as part of an overall IT security strategy that 
included a thorough assessment performed by an independent auditor.

The auditor completed technical scans and reviewed policies and physical 
controls, including firewalls and routers. The auditor also scrutinized 
application development practices.

The review also benchmarked the states security program, which complies 
with International Organization for Standardization 17799, a detailed 
international standard for managing information security.

Networks evolve constantly and automated penetration testing is a way to 
do the maintenance in between the full-blown assessments, said Elayne 
Starkey, Delaware Department of Technology and Informations chief 
security officer.

However, Starkey said there are limitations to automated penetration 
tests. Dont think of a tool like this as a silver bullet, she said.

But as part of an overall security program, automated penetration tests 
can add value if handled properly, said Yong-Gon Chon, senior vice 
president of services at SecureInfo.

My biggest fear is that you end up with an untrained new security 
engineer with a few grand in the security budget who buys a tool that 
they run, and it has a massive impact throughout the infrastructure, 
Chon said.

Subscribe to the InfoSec News RSS Feed

This archive was generated by hypermail 2.1.3 : Sun Feb 25 2007 - 23:29:40 PST