by CIO Staff

Acts of God (and Others)

Jul 16, 20075 mins
Data Center

Sometimes the blame for information technology catastrophes cannot be laid at the feet of the IT department.


Storm of the Century: Katrina

Wind-wracked shelters, waterlogged homes, soaked vehicles, drenched pets and haggard people. The public knows those scenes of Hurricane Katrina and sister Rita, which roared ashore along the United States’ Gulf Coast in late summer 2005. Hidden just out of view, though, is the role IT played in both collapse and recovery. In the networked world, IT systems fall like dominoes. Executives with organizations based along the Gulf Coast give harrowing accounts of how they dealt with the Storm of the Century, and how they got their operations back up and running.

“The most important thing we learned is that, no matter how good your plans are, things happen that you didn’t anticipate. What really saved the day is what I call ‘aggressive improvisation.’ That means people look at a problem and do whatever it takes to solve it.” — Tom Oreck, chief executive and president of cleaning products manufacturer and retailer Oreck, which had two facilities along the Gulf Coast.


Political Hobgoblins: Regulations

Sarbox. HIPAA. Gramm-Leach-Bliley. New supply chain regulations. More privacy mandates. IT audits. Frank good sense, or Frankenstein’s monster? That view depends on which side of compliance you and your business stand: You’re either crowing over these regulations’ merits or you’re cursing their creators. The new requirements weren’t solely intended to keep you up all night fighting off the hobgoblins, but in the end you know it’s usually the CIO and staff with their necks on the chopping block when audit time comes around againas inevitably as the full moon. But fear not. CIO’s compliance resource center can help you keep from being swept away by the maelstrom of paperwork and red ink.

“The most important things that a CIO can do in working with internal and external auditors is be involved, to be supportive, and to be open and honest in all situations. An independent view of a system, a process or control can only help CIOs more effectively support their companies.” — Lisa Harris, then-senior vice president and CIO with Gevity, a human resources outsourcing firm


Dark Shadows: East Coast Blackout

Aug. 14, 2003. 4:12 p.m. In an instant, the largest power outage in American history left much of the area between Detroit and New York Cityand on into Canadain the dark. Some 50 million people were left without power, but the world rushed on. Companies still needed access to data centers. Employees needed to reach managers, colleagues and families. Nervous residents rushed through darkened streets to ATMs to stock up on cash, in fear they would never emerge from this dim state. Affected IT execs quickly found new perspective on their business continuity plansor lack thereof. The smart ones were ready. The unlucky others were left in the dark.

“We did a ton of testing on procedures during Y2K preparations, and when I woke on January 1, 2000, I thought, boy, what a waste that exercise was. Now, [after the blackout], it’s come in handy.” — Jim Simmons, then-chief executive with SunGard Availability Services


Terrorism: 9/11

The all-too-real attacks on New York City’s World Trade Center towers on Sept. 11, 2001, horrified people across the globe. Thousands were slaughtered. Tens of thousands more were left jobless. When terrorists boarded airplanes on that day, their intention was not simply to kill people and destroy buildings; rather, the attacks were an assault on the American economy. Nowhere was their impact on IT systems felt more strongly than in New York City’s financial district, specifically on Wall Street. IT executives from three multinational financial services firms located near ground zero in September 2001American Express, Lehman Brothers and Merrill Lynchtold CIO the lessons they learned about disaster recovery, about how their businesses reacted, and why they decided to stay or not to stay in lower Manhattan.

“A CIO I worked for a long time ago used to say, ‘You lose a whole data center every 10,000 years,’ which was his excuse for not having disaster recovery, which was stupid then. …You have to assume it’s more likely to happen now.” — Glen Salow, then CIO of American Express


The Day the Beepers Died: Satellite Failure

Galaxy 4 spins out of control! Indeed, on May 19, 1998, in what many consider the most widespread digital failure in history, the Galaxy 4 telecommunication satellitewhich hosted paging services, television networks and financial services applicationsconked out. A very intense flux of electrons may have caused failure of the attitude control systems and their backup. The outage pulled the rug out from under CIOs and IT managers dependent on beepers and similar devices for instant communications with employees and suppliers. Roughly 45 million pagers went beep-less until providers were able to find capacity on other satellites.

“We have the best communications system in the world, until it has a hiccup and then we realize just how dependent we are.” — Jeffrey Kagan, then-president of Kagan Telecomm Associates of Atlanta, a communications consulting firm, as told to CNN in 1998


“What did we miss? Nominate your own Worst IT Disaster by sending it to Chris Lindquist at”