[ISN] Big Brother on a budget: How Internet surveillance got so cheap

http://arstechnica.com/information-technology/2012/08/big-brother-meets-big-data-the-next-wave-in-net-surveillance-tech/

By Sean Gallagher
Ars Technica
Aug 28 2012

When Libyan rebels finally wrested control of the country last year away 
from its mercurial dictator, they discovered the Qaddafi regime had 
received an unusual gift from its allies: foreign firms had supplied 
technology that allowed security forces to track nearly all of the 
online activities of the country’s 100,000 Internet users. That 
technology, supplied by a subsidiary of the French IT firm Bull, used a 
technique called deep packet inspection (DPI) to capture e-mails, chat 
messages, and Web visits of Libyan citizens.

The fact that the Qaddafi regime was using deep packet inspection 
technology wasn’t surprising. Many governments have invested heavily in 
packet inspection and related technologies, which allow them to build a 
picture of what passes through their networks and what comes in from 
beyond their borders. The tools secure networks from attack—and help 
keep tabs on citizens.

Narus, a subsidiary of Boeing, supplies “cyber analytics” to a customer 
base largely made up of government agencies and network carriers. Neil 
Harrington, the company’s director of product management for cyber 
analytics, said that his company’s “enterprise” customers—agencies of 
the US government and large telecommunications companies—are ”more 
interested in what's going on inside their networks” for security 
reasons. But some of Narus’ other customers, like Middle Eastern 
governments that own their nations’ connections to the global Internet 
or control the companies that provide them, “are more interested in what 
people are doing on Facebook and Twitter.”

Surveillance perfected? Not quite, because DPI imposes its own costs. 
While deep packet inspection systems can be set to watch for specific 
patterns or triggers within network traffic, each specific condition 
they watch for requires more computing power—and generates far more 
data. So much data can be collected that the DPI systems may not be able 
to process it all in real time, and pulling off mass surveillance has 
often required nation-state budgets.

[...]