[ISN] Explosion has data center scrambling, users venting

From: InfoSec News (alerts@private)
Date: Tue Jun 03 2008 - 02:01:59 PDT


http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9092158

By Patrick Thibodeau
June 2, 2008 
Computerworld

The Planet.com Internet Services Inc. hopes to have all 9,000 of its 
servers in its Houston data center back online later tonight following a 
blast that shut down the facility on Saturday afternoon.

When firefighters arrived at around 5 p.m., they could see "light smoke" 
at the Planet data center -- the aftermath of an explosion in a network 
gear room that produced enough force to move walls. Sprinklers quickly 
doused whatever flames erupted; the fire was attributed to an electrical 
problem with a transformer, according to a Houston Fire Department 
spokeswoman. There were no injuries.

Although the data center says it has power systems that "are designed to 
run uninterrupted" and a "fully redundant network operations center" 
with diesel generators, the electrical problem exposed an apparent 
Achilles' heel in its business continuity planning. Firefighters told 
data center workers to turn off all the power, according Planet 
spokeswoman Yvonne Donaldson. That meant the servers, even though they 
weren't damaged, were offline.

Approximately 6,000 of the affected servers were returned to service 
early this morning. Another 3,000 were due to return online by tonight, 
the company said. The Planet staff provided updates on the restoration 
on its customer forum site, including a message from CEO and Chairman 
Douglas Erwin, who wrote that some servers will be relying on generator 
power for a week until normal utility connections are restored.

The Planet operates more than 40,000 servers at multiple data centers 
and hosts more than 3 million Web sites.

While Planet data center staff worked to restore service, users -- many 
of them small business owners -- wrote of their frustrations over the 
outage on forum posts. Questions about the data center's backup 
capabilities were raised, as well. One person, flynnibus, wrote: "You 
shouldn't put all your money into one bank -- and you shouldn't put all 
your servers in one DC [data center] if you want to be truly resilient."


_______________________________________________      
Attend Black Hat USA, August 2-7 in Las Vegas, 
the world's premier technical event for ICT security experts.
Featuring 40 hands-on training courses and 80 Briefings 
presentations with lots of new content and new tools.
Network with 4,000 delegates from 50 nations.  
Visit product displays by 30 top sponsors in 
a relaxed setting. http://www.blackhat.com



This archive was generated by hypermail 2.1.3 : Tue Jun 03 2008 - 02:17:06 PDT