Lessons from Beth Israel Network Meltdown

Lesson 1 Treat the network as a utility at your own peril.

Actions taken:

1. Retire legacy network gear faster and create overall life cycle management for networking gear.

2. Demand review and testing of network changes before implementing.

3. Document all changes, including keeping up-to-date physical and logical network diagrams.

4. Make network changes only between 2 a.m. and 5 a.m. on weekends.

Lesson 2 A disaster plan never addresses all the details of a disaster.

Actions taken:

1. Plan team logistics such as eating and sleeping arrangements as well as shift assignments.

2. Communicate realistically?even well-intentioned optimism can lead to frustration in a crisis.

3. Prepare baseline, “if all else fails” backup, such as modems to query a network and a paper plan.

4. Focus disaster plans on the network, not just on the integrity of data.