by Sarah D. Scalet

IT Executives From Three Wall Street Companies – Lehman Brothers, Merrill Lynch and American Express – Look Back on 9/11 and Take Stock of Where They Are Now

Sep 01, 200217 mins
Disaster Recovery

From the American Express executive library on the 51st floor of 3 World Financial Center, the ferries crossing from New Jersey look like toy boats, and Manhattan seems clean and orderly. With plush leather furniture and book-lined walls, with atlases arranged on the coffee table and brass telescopes at the window, everything about the room’s decor is supposed to make those who use it feel invincible.

That would be a big change from a year ago. Because of suspected structural damage to the building when the adjacent World Trade Center towers fell and because of questions about the future of the financial district, no one knew who?if anyone?might return to this room.

On Saturday, Sept. 15, 2001, American Express CIO Glen Salow stood in Jersey City, N.J., and looked across the Hudson to New York’s altered skyline. It was 7 a.m., and he had just the night before returned from Minneapolis.

“It was the first time I’d physically seen what had happened, saw the horrible smoke coming up,” he recalls on a day in May two weeks after he returned to his old office. “I saw our building. It had shrapnel sticking out of it, which if you didn’t know [you] would think was actually our girders bent out. I looked at our building and said, It’s only a building.”

Of course, that wasn’t how American Express executives acted when they staged a triumphant return to headquarters this spring, a move which should be complete by the end of this month. Step by step, Wall Street?the state of mind?is being painfully reconstructed. But Wall Street?the place?will never, can never and should never be the same.

With IPO cash dried up, investor confidence down and layoffs underway, 2001 would have been tough for financial services. Then came Sept. 11.

Thousands of people were killed. Tens of thousands more lost their jobs, and countless others had their foundations shaken. In hard dollars, the attacks on the World Trade Center cost New York City $83 billion, according to an economic impact analysis by the New York City Partnership, cited by the General Accounting Office as the most comprehensive one completed.

“In a lot of comments I hear from the Street, they sort of view this past year as a lost year,” says Rob Gould, a PWC Consulting financial services partner, who participated in the study.

Cantor Fitzgerald, a small financial services and trading company perched atop one of the towers, came to represent the human tragedy of the attacks on the World Trade Center. But next door, three multinationals with headquarters in the World Financial Center?American Express, Lehman Brothers and Merrill Lynch?came to represent what the future of Wall Street might look like.

Two of the companies returned to Lower Manhattan; one did not. One is still recovering from 9/11, another is trying to recover an image tarnished by scandal, and another wants to project the image of a powerhouse undaunted by the disaster.

CIO sat down with the top IT executives at those companies and others to find out where they are now. What a visitor hears, time and again, is a new respect for business continuity, vendor relationships, real estate strategy and what it means to be a leader.

The Leader That Roared

“I’m not big on touchy-feely.”

-Jonathan Beyman, CIO, Lehman Brothers

When CIO Jonathan Beyman’s office was in the World Financial Center, he used to worry about having a fire in a data center. Now, from Lehman Brothers’ flashy new headquarters near Times Square, Beyman worries about how the investment banking firm would survive if?deep breath?all of Manhattan had to be evacuated.

“Our imagination is a lot more fertile these days,” says Beyman in his new, 15th floor corner office. Behind him is a panorama of other midtown skyscrapers; to his left is a poster of firefighters raising the American flag at Ground Zero.

Not that imagination necessarily would have done him any good on Sept. 11, when one of the 800 Lehman Brothers IT employees in the World Trade Center was killed, a data center was destroyed, and headquarters and the trading floors in the damaged World Financial Center were rendered useless.

“In a different world, maybe we would have had a playbook that said, here’s what you do if 6,500 people are dislocated from downtown. I probably would have forgotten to take it with me,” Beyman says. “Our ability to recover was not based on me being so smart. It was predicated on us having the environment where people knew what their jobs were and had the courage to do things that under normal circumstances, somebody would have said, What are you doing?” For instance, one person at Lehman’s Jersey City office ordered 2 million feet of Cat-5 network cabling on Sept. 11, knowing it would be precious. Quick thinking like that allowed Beyman’s staff to build a functioning trade floor within 48 hours of losing headquarters.

But don’t let the praise for employees fool you into thinking that Lehman has gone soft. “I’m not big on touchy-feely,” says Beyman, 46, who has worked for Lehman for 12 of the past 16 years. “There’s a great line that Winston Churchill used about England, which was something on the order of, This country has the heart of the lion, and I just happened to be the roar. That’s exactly how I felt. I didn’t view that I did that much. I just prodded people and yelled at people and did exactly what I normally do.”

In the frenzied days and weeks that followed 9/11, Lehman squeezed people and trading floors into the Jersey City building. They also found other space in Jersey City and Manhattan and even took over the Sheraton Hotel in midtown, where beds were moved out, desks were moved in, and a ballroom was reinvented as an IT hub. Other employees worked at home, accessing applications through a secure portal that had been in the pilot phase.

They weren’t the only ones to make a pilot project mission-critical. At Deutsche Bank, CIO Mitchell Lenson had been working on a virtual private network to allow remote offices to access the bank’s infrastructure. “Sept. 11 forced us to test the solution for real,” Lenson said in an e-mail interview from London. “We were able to deploy location-independent access, for which we now have more than 11,000 users.” That’s handy, since executives for the Frankfurt, Germany-based bank haven’t decided if its building near Ground Zero is salvageable.

At Lehman Brothers, executives didn’t want to wait to find out whether the building could be saved. In October 2001, Lehman bought a 32-story building at 745 7th Ave. being built by Morgan Stanley. This is no ordinary building and certainly not one that brings to mind the frugal investment bank spun off from American Express in 1994. First-time passersby stop to gape at more than a dozen LED screens that cover the first several floors of the building and show artistic video montages of, perhaps, the light on a wheat field as a thunderstorm rolls in.

Despite the eye candy, Lehman has been defensive about what many New Yorkers see as an abandonment of Lower Manhattan. Employees point out that they’re still in the city and say they couldn’t have waited until May for their old offices to be ready. “Wall Street is virtual; it can be anyplace,” Beyman insists. “From a technology standpoint, there’s no reason to be downtown, uptown, midtown, even in New York City.” The only constraint is the need to be close to customers and New York’s talent pool. Also, the trading floor’s backup systems must be within 35 miles?the distance that the optical glass fiber used for instantaneous backups can reach before the signal degrades.

Almost a year after 9/11, Lehman is still very much in recovery mode. The interior of the new building is being finished, and a new data center is under construction. “It’s a greenfield, if you will,” Beyman says, trying to be optimistic about the fact that Lehman still lacks redundant storage. “It’s a chance to start over again and correct whatever sins of the past you have.” There will be higher availability of systems and more real-time backups, and all the direct phone lines to customers won’t terminate in one place, as they did at the World Financial Center.

Lehman also is developing business continuity plans tied to the color-coded Homeland Security Advisory System devised by the Office of Homeland Defense. If the country moves from yellow (elevated) to orange (high) warning level, for instance, certain executives might be instructed to work across the river in Jersey City, and nonessential employees might be told to work at home. In fact, home offices were so crucial in the recovery that Bridget O’Connor, Lehman’s head of business continuity, is trying to get employees to work from home one day a month to make sure they have the know-how to keep working if headquarters is closed.

But decentralizing operations can be more complicated than it sounds. “The way Wall Street operates, it’s still very face-to-face relationship driven,” says Sarah Diamond, managing director for KPMG Consulting in Boston, who has seen a lot of financial services companies begin to rely on IT?not geography?to stay together. “They’re looking now at less concentration of people within one real estate location, as well as taking advantage of technology and doing more telecommuting. It changes the style of the way they do business.”

Before, Lehman had been so focused on personal interactions that it had considered ditching its Jersey City building and concentrating all its operations in Lower Manhattan. “If we’d gone down that path, we’d probably be out of business,” Beyman says. Now, Lehman will keep not only the property but also the backup trading floor there.

“Who knows what we’ll do three years from now, but for right now we have hot standby for our traders. I don’t know if that prepares us for World War III and the devastation of New York City, but it certainly prepares us for any kind of the really foreseeable future-type of events, like the evacuation of Manhattan, as horrible as it may be. We never would have gone down that path before, but now we go down that path.”

The Savvy Networker

“You can never be too rich, too thin or have too much telecom recovery.”

-John McKinley, head of global techno-logies and services, Merrill Lynch

John McKinley isn’t just the man who makes technology decisions for Merrill Lynch. He’s the guy who, with the weight of the country’s largest brokerage behind him, talks to financial analysts and journalists about what his Fortune 500 peers are buying and therefore which stocks will pay off. These days, when he speaks at luncheons about investment strategies, he likes to talk up voice over IP and wireless LANs. And when he talks about Merrill’s recovery after 9/11?well, he likes talk up voice over IP and wireless LANs then too. It beats talking about the disasters that have plagued Merrill for the past year. “You can never be too rich, too thin or have too much telecom recovery,” says McKinley, the 44-year-old executive vice president and head of global technologies and services for Merrill Lynch.

Sage words, because even companies whose headquarters weren’t damaged on 9/11 had significant problems with telecommunications. A Verizon hub at 140 West St. was severely damaged, which affected phone service throughout the region. Thomas Moogan, managing director of the general services division of Citigroup Corporate and Investment Bank, says he learned that it wasn’t enough to have more than one telco provider. “When you think you have redundancy, you may have two [vendors’] circuits running through the exact same trough downtown,” says Moogan, who heads business continuity planning. Moogan adds that his company changed its contracts with carriers to include the actual path a signal is taking, in an effort to diversify service.

At Merrill, “we had all the classic game plans in place: diverse vendors, physical diverse routing, different central offices blocks apart,” McKinley says. None of the diverse technology in Lower Manhattan mattered, of course. After Sept. 11, 9,000 Merrill employees from four buildings downtown suddenly didn’t have a place to go to work, let alone working telephone connections. The first step was rebuilding trading floors and moving employees into crowded offices in four towns in New Jersey, where thousands of voice-over-IP handsets saved the day. “That allowed us to effectively deliver our own dial tone overnight Tuesday night when the vendors were rightfully concerned about health and human safety, government services and the exchange itself,” says McKinley, who can now justify expanding his once-risky voice-over-IP plans, begun in 2000. (He won’t say how much Merrill spent on the technology.)

The buildings used for recovery were also outfitted with wireless LAN devices governed by the 802.11 set of standards. With a WLAN setup, rather than running cables to individual workstations, technicians install access points that broadcast the network signal to multiple wireless network cards. That setup gave Merrill tremendous flexibility in reconfiguring office space and was so crucial that McKinley is now making WLANs part of new building specifications.

The second step of recovery was moving back into headquarters at 4 World Financial Center starting in October. At first glance, the building seemed to be in good condition (though 900 windows were blown out of another nearby Merrill office building at 2 World Financial Center). But because loss of electricity immediately after the attacks caused high temperatures in the headquarters office tower?and also because the warranties were voided?a great deal of the computer equipment had to be replaced. Most of the equipment was covered by insurance. Merrill wasted no time, moving 500 to 1,000 employees every weekend?this compared with American Express’s decision to move 400 employees back every two weeks. “Having rebuilt three trading floors in six days, management wanted to give me 12 to do this,” jokes McKinley.

On Oct. 21, 2001, McKinley was one of the first people who moved back downtown. Since then, he’s watched the area start showing signs of life. Public transportation, although not ideal, has improved, and some of the restaurants have reopened, along with the Century 21 Department Store, an Embassy Suites Hotel and a movie theater. A Merrill publication called the North Tower Newsletter kept employees informed about what services were available in what felt like an island in the midst of a disaster scene.

“I can’t understate the emotional impact of it,” says McKinley, sitting at a table in his airy corner office on the 32nd floor. “It’s still emotional. You are no more than one or two degrees of separation removed in financial services. Either you knew somebody or you know a family impacted by it. It’s tough,” he says. Merrill lost three people in the attacks. “You’re making me sad again,” he says with a wry smile, trying to lighten the mood.

He can’t afford to be sad, though, as he struggles to help make the company profitable again. Revenue was so bad that more than 10,000 employees were laid off. (The brokerage has also launched an ad campaign to try to mend its image, scarred by a $100 million settlement that ended an investigation of stock analysts’ possible conflicts of interest and, more recently, the suspension of Martha Stewart’s stockbroker after allegations of insider trading.)

McKinley’s efforts, while tied to Merrill’s quarterly fortunes, also transcend them. For example, based on lessons learned during 9/11, he and his colleagues have made some changes in Merrill’s overseas operations, looking at data centers and alternate trading floors in London and Japan. And discussions about how concentrated to be continue.

“The dialogue around real estate strategy has gone up several notches,” McKinley says. “Where should you be geosourcing your talent base, your human capital? You’re constantly balancing the synergies of colocation with the resiliency and recoverability of a more distributed model, and where should you be on that continuum? I think it’s not a binary answer.”

For a while after 9/11, there was a lessening of the competition?competitors helping each other out, sharing ideas about recovery. “Now it’s back to the brass knuckles,” he says, pounding the knuckles of one hand with the palm of another. He laughs. That’s a good thing, he says. It means business is running again.

The PragmAtist

“You’re more dependent on your vendors than you think.”

-Glen Salow, CIO, American Express

Glen Salow is pretty unexcitable for a man who helped mastermind a 4,000-person move back to Lower Manhattan that some observers called heroic. “There always will be nuts in the world, and a building that calls itself the World Financial Center in New York City is going to be a target,” says Salow, the 46-year-old CIO and executive vice president of American Express, his voice flat. “A lot of my job is assuming things are targets. If you have technology assets here and they don’t need to be here for a purpose, they shouldn’t be here. So get them out.”

Salow is getting them out in a grand way?and we’re not just talking about the obvious decision, say, to start storing backup tapes for the office automation servers farther away from headquarters than the basement of the Trade Center. American Express’s technology operations were already spread out, but the company decided to house only a few technology operations in its New York headquarters, such as the bare requirements of telephones and networked desktop computers.

It’s part of a move to make continuity planning geography-based, not building-based. “We had a disaster-recovery plan for this building. We had a disaster-recovery plan for 7 World Financial Center, where we had our bank back office. Each building continuity plan worked, but we hadn’t contemplated losing all of Southern Manhattan,” he says. Now, Salow is looking at what might happen if business shuts down in Phoenix or Minneapolis, where the company’s IT operations are based. “A CIO I worked for a long time ago used to say, ’You lose a whole data center every 10,000 years,’ which was his excuse for not having disaster recovery, which was stupid then. You have to assume it’s more likely to happen now, and it’s affecting larger geographic areas.”

Salow also signed a seven-year, $4 billion outsourcing deal with IBM Global Services, announced in February. About 2,000 members of his 6,000-person IT team are being transferred to IBM. (So far, about 99 percent of them have accepted the offer, he says.) Salow says the deal will save hundreds of millions of dollars and likens it to purchasing electricity rather than generating it. For less money, he says, American Express can draw on the resources it needs, when it needs them.

Salow went ahead with the IBM deal despite the fact that 9/11 taught him how much he already relied on his vendors. In the future, he says, he’ll spend more time delving into how his vendors would recover from a disaster. But some vendors have already proven themselves with their actions. “If I look at the role IBM Global Services played for us, at the role Compaq played for us, they’ve got every bit of help we could ask for, and it’s there before we could ask for it,” he says. With the outsourcing deal, he says, “On the one hand, we’re more dependent on IBM Global Services, but on the other hand, I think I got a really cheap insurance policy.”

Between the employees being transferred to IBM and the thousands laid off since September, though, things are certainly looking different from the 51st floor of the American Express building?even for Salow who doesn’t happen to have a view of the Trade Center site. “Maybe in a few months, I’ll stop long enough to think about it and may in fact be taken aback by the level of change and how we got through it,” Salow says. “But during the process we just did it.”