Enterprise Software Upgrades: Less Pain, More Gain

Prologue: The Scream

Upgrading software from one version to the next appears like a small step?after all, you’re not ripping the stuff out and replacing it, you’re just improving it a little bit?but Nextel Senior Vice President and CIO Dick LeFave finds himself about to step off a precipice with his Oracle ERP software upgrade.

The numeric change?from version 10.7 to 11i?sounds minor, but in this case the numbers and letters lie. “This isn’t just doing an upgrade; it’s a whole new system,” says LeFave. “It’s a different design; it’s a whole different set of solutions; it’s a whole different way of doing business. You may as well start out from scratch.”

This is not what LeFave expected. And he’s not alone. Denise Quinlan, an assistant vice president and PeopleSoft product manager at Boston-based MFS Investment Management, is still recovering from the sticker shock she got when her ERP vendor quoted her the price for dispatching its consultants to do an upgrade. When Quinlan heard the estimated cost to go from version 7.5 to version 8?$490,000 for the HR software and $872,000 for the financial software?she blurted, “For an upgrade?”

For PeopleSoft’s part, a spokeswoman says the price quote was preliminary and that there were no discussions about using MFS staff to reduce costs (see “Vendors: ’It’s Not That Bad,’” Page 49).

CIOs who have to wield the wrench in these efforts are shocked when they look under the hood of major software upgrades, not just from Oracle and PeopleSoft but from J.D. Edwards, SAP and Siebel?all the enterprise software vendors?and see that they’re facing overhauls, not tune-ups. The CFOs and CEOs who foot the bills for these upgrades are equally nonplussed. They ask questions like, Didn’t we just pay millions for this stuff?

Yes. Between $40 million and $250 million for an enterprise software system for a Fortune 500 company, according to AMR Research.

And the CFOs and CEOs want to know why they’re paying again.

Why It Hurts So Bad

This story focuses on ERP users, but the same problems and advice applies to CRM, supply chain management systems and other major enterprise software products. They have become so complex, so expensive and so integral to business processes that the term upgrade is a misnomer. Enterprise software upgrades can cost up to 30 percent of the original software installation price, according to Gartner, take more than a year to complete and require companies to revamp their technology infrastructures and business practices. CIOs have to present a strong business case for why, in these difficult economic times, their company should go through the trouble and expense. A tough sell to most corporate boards.

Three factors further complicate the upgrade process.

1. Upgrades are unforgiving when it comes to customization.

Enterprise software is the one-size-fits-all suit that you tailor to be a 38-short on one side for your manufacturing group and a 54-portly on the other side for your salespeople?along with some extra pockets from other vendors stitched onto the suit. Don’t expect the vendors to touch that suit with a 10-foot needle when it comes time to upgrade. It’s up to you to redo the customizations and connections with third-party software on the new version. Customizations that need to be carried over from one version of enterprise software to the next are the biggest technology headache and ROI killer that CIOs face in upgrades.

2. The move to Internet architecture changes everything for the IT staff and end users.

All the big vendors have moved their software from a client/server architecture, in which PC-based software accesses a central server through a company’s network, to an Internet architecture, in which a Web server joins the mix and the software is accessed through the public Internet and a Web browser. It’s new turf for both IT staffers and end users. Staffers suddenly must see the Internet as the platform for the company’s most important applications. End users must learn different ways to access programs and absorb all the business process changes that come when they can collaborate with people outside the company. These added capabilities are a good thing, of course. But the planning and change management that is required make them a migraine minefield. (For what one vendor is doing to respond to this problem, see “SAP’s Big Freeze,” Page 56.)

3. The vendor’s “desupport date” frustrates a CIO’s best-laid plans.

The desupport date is the ticking time bomb of enterprise software upgrades. A vendor says, We won’t support version XYZ of our software after this date. Technology changes and customers?especially new ones?demand fresh functionality. Vendors can’t afford to support six different versions of their software simultaneously.

But CIOs object to what they call surprise desupport announcements that most major vendors have made in the past few years. They say they aren’t getting enough time to do one upgrade before a vendor announces a desupport date for an older version.

Nextel’s LeFave has been through this. “When the vendor comes to you and says the product is at the end of its life cycle, that’s code for saying, I’m going to be putting my maintenance and development resources on a new product, and you’re not going to get anything unless you pay for it yourself,” he says. Even if the vendor continues to support old software versions, it will shift the bulk of its people and resources to new versions so that finding someone knowledgeable about your old version becomes like trying to find a department store salesperson on red-tag clearance day.

“I’d like to have a road map of when upgrades are planned and when to expect them so that we can do our planning better,” says LeFave.

The average time between upgrades has shrunk from three years in the early 1990s to 18 to 24 months, according to AMR Research, and CIOs have lost the ability to keep up. “Vendors are pushing new code out as fast as they can?so rapidly that you may have updates coming at you almost monthly,” says Pat Phelan, an ERP analyst for Stamford, Conn.-based Gartner. “The vendors don’t seem sensitive enough to the fact that the average buyer can’t absorb that kind of change.”

With all these difficulties, a few brave CIOs are fighting to push back the desupport dates. In July 2001, 58 members of the 2,200-member independent Oracle Applications Users Group (OAUG) signed a petition urging Oracle to extend the support date for version 10.7 of its ERP software from June 2002 to December 2004. This petition came after Oracle had already extended the desupport date for 10.7 from June 2001 after customers complained bitterly about all the bugs?about 5,000 of them, according to customers?that appeared in the initial release of the 11i software in June 2000. (Oracle declined to comment on the number of bugs in 11i.)

In the end, Oracle and the OAUG compromised, and the desupport date was extended to June 2003. The other major enterprise vendors?J.D. Edwards, PeopleSoft, SAP and Siebel, which, like Oracle, have released ambitious upgrades of their software in the past few years?have all extended desupport dates for previous versions in response to customers’ complaints about bugs and performance. (In addition, some companies have found third-party support for software after a vendor-announced desupport date.)

Download These Scream Savers

CIOs have two choices when it comes to upgrades: go along or fight. But both options require more planning than most CIOs do now. Even if upgrades aren’t a continuous process (and some overworked CIOs may think they are), planning for them must be. It’s the only way you can make upgrade decisions without your back against the wall of a desupport date. We’ve uncovered best practices for building a solid internal governance structure for managing upgrades and staying on top of your vendor’s upgrade schedule; for minimizing customizations to ease future upgrades; and for organizing with your peers to negotiate for desupport dates that won’t cripple you. (For more on this process, see “The Seven Lively Steps to an Upgrade,” Page 52.)

Strategy No. 1: Don’t Stop Thinking About Tomorrow

Companies put off upgrading their enterprise software because the process is a shock to their system?not just the IT system but the business system. Some postpone the process until they will lose support for an old software version if they don’t upgrade (see “It’s the Functionality, Stupid,” this page).

To avoid desperate upgrades, CIOs need to create a continuous planning process that includes users, not just IT.

At Ventura Foods, a Los Angeles-based food distributor, Richard Chamberlain has set up a cross-functional team of IT and businesspeople who are responsible for each module of the company’s J.D. Edwards ERP system. Those module leads are the superusers who help their business departments understand and use the system. The committee has a flowchart of the entire ERP system that shows all the business processes affected by the different components. When an update is released, the committee can map the new functions to the current system and the business processes that are affected to determine whether the updates are worth incorporating.

The module leads, along with IT managers, are responsible for tracking the updates. That keeps them involved and makes it easier to enlist their support for doing an upgrade?or resisting it. “The different module leads come together and say, OK, here’s an update, what does this mean to all of us?” says Chamberlain, who is director of innovations and e-solutions.

Like most ERP customers, Chamberlain keeps three different ERP environments running at once: the system that the company uses day-to-day?the production environment; a much smaller development environment for playing with new software and any customizations that are necessary; and a testing environment to see if new versions of the software will work properly before they are brought to the production system.

But in a testament to how complex enterprise software upgrades can be, Dennis Upton, CIO at Brother International, a Bridgewater, N.J.-based office equipment maker, had to set up a second test environment when he upgraded from SAP 3.0E to the SAP 4.5B product because the new software was so different from the old. “You have to create a duplicate environment for your upgrade,” he says. “It’s a fact of life.”

It was also about $200,000 in extra cost.

Strategy No. 2: Flush the Custom Code

CIOs know that putting an upgrade through its paces in a test environment is critical because most companies wind up customizing their enterprise software to fill gaps in the functionality, adapt to a specific industry or legal requirement, or simply because the users want the software to act differently than it does out of the box. It’s not that customizations represent a failure?they are often legitimate and may even add a bit of competitive advantage to an otherwise vanilla software program. But companies tend to customize overmuch in an attempt to appease the businesspeople who use the system.

That’s why some CIOs see upgrades as a prime opportunity to flush custom code from the system and replace it with a standard software program or interface from a vendor. Removing custom code makes it easier to upgrade because you don’t have to rewrite the code each time a new enterprise version comes out. It also makes it easier to integrate the software with other software systems inside the company.

“I think the effort required to upgrade is directly related to the amount of custom code you have,” says Rafael Sanchez, CIO of Burger King. The Miami-based restaurant chain built a custom software program to handle its real estate dealings when it first installed SAP R/3 in 1995. But this year Sanchez replaced it with standard functionality from SAP. “We reduced the custom code in the real estate package by 90 percent, in finance we got rid of 60 percent, and in HR we’ll lose 60 to 80 percent,” he says. “All that means is that we’ll have fewer software applications to support internally.”

But CIOs must be careful before they get rid of their customized code during an upgrade. Business users may love that code, so you need their backing?or at least their understanding?for why you’re dropping it, says Cheryl T. Smith, the former senior vice president and CIO of KeySpan, a Brooklyn, N.Y.-based gas and electric company. (Smith became CIO at McKesson in September.)

During recent upgrades to KeySpan’s Oracle ERP system, Smith and Don Stahlin, director of IT, identified all the customized components of the software and the opportunities to go to standard functions in the package’s new version. They met with users to see which customizations they were willing to give up. When the two groups disagreed, they brought the discussion before a steering committee composed of functional and business leaders from across KeySpan.

Smith and Stahlin came up with costs for two options: modifying the actual ERP code of the new version (expensive and difficult) or adding a custom program outside of the ERP system to perform the function. That helped users understand just how much effort IT had to put into writing and maintaining those customizations, says Stahlin. But the users also had to do some research before they could go before the steering committee to plead their case. They had to come up with a “no-technology” option, usually a change to the business process to fit with the software out of the box.

“Sometimes the users had valid reasons for keeping the customizations, like labor contracts that required things to be done a certain way,” Smith says. “But we found nine and a half times out of 10 we could change the way we do business because it wasn’t that critical; it was just habit. You just unfreeze the organization and rearrange the way people do things, and now you just do them a slightly different way and that’s OK.”

The steering committee tended to shoot down the requests for customizations, says Smith, because they were invariably more expensive than the no-technology options. “When someone from HR came to them and said, ’We have to do it this way,’ the committee would say, ’No you don’t?change your process. I’d rather spend this money in my business.’”

Strategy No. 3: Do It Yourself (Forget Consultants)

Having a continuous internal planning system for upgrades makes it easier for CIOs to limit the number of outside consultants they need to bring in to help with upgrades. An AMR study found that companies that handed over responsibility for their upgrade projects to outside consultants spent twice as much ($2.3 million versus $1.5 million) and took longer (10 months versus six) than those that kept the project leadership and as much of the work as possible in-house. “The costs skyrocket because you will have people on the project who don’t know your business,” says Judy Bijesse, an analyst at AMR Research, “and you’ll have a lot of consultants who are being trained while you’re paying them.”

By retaining leadership yourself and tracking the fortunes of colleagues who are upgrading, “you can avoid being the one that bleeds on that first release,” says Nextel’s LeFave. Indeed, most enterprise software is so bug-ridden in its first release that CIOs can wind up installing the upgrade all over again when the vendor comes out with a “point release” to fix the initial bugs. Manish Khadepau waited until Oracle was on the second point release of Oracle 11i before implementing it at Infogrames, a New York City-based video game publisher. (There have been six point releases of 11i since it was first introduced in 2000?each requiring a complete reinstall if the customer has customized it.) But it was still bleeding edge at the time, Khadepau says, and full of bugs.

Each time Oracle would send Khadepau a new bug fix, the fix would destabilize the rest of his system and require him to rewrite the customizations his company had made. “Oracle 11i is so big and so interconnected that when they fixed one piece, three others would break somewhere else in the system,” he says. The earlier version of Infogrames’ ERP software was so heavily customized that the upgrade wound up costing as much as a new installation?between $400,000 and $500,000, according to Khadepau.

Quinlan, the MFS systems manager who blanched at the cost of consulting fees for her company’s upgrade, has decided to install a new HR application herself (she has a background in HR and is a former PeopleSoft consultant) with support from a few internal staffers. But the financial upgrade is bigger and much more complex than she and her staff can handle. “We haven’t decided what to do there yet,” she says.

Strategy No. 4: Wait for the Bugs to Pass You By

Companies that can afford to wait for an upgrade are best positioned to get it done quickly?at least from a technical standpoint, says Khadepau, the Oracle ERP veteran who is now manager of financial systems for Multilink Technology, a Somerset, N.J.-based maker of optical network components.

Khadepau knows because he is the volunteer director of the New Jersey chapter of the OAUG and has seen his colleagues struggle with 11i. He says one New Jersey company (which declined to be identified) bought the license for 11i and did not install it. Instead, it sent an IT staffer to monitor meetings of his group until the wails of despair changed into guarded optimism about the software. The company is now installing version 11.55 of the software, which works well, according to Khadepau, and has new functionality that early releases of 11i did not. “Now they come to me and say things like, What was all that fuss you were making about?” he laughs.

Companies have to pick their time to upgrade carefully?especially small companies that don’t pull much weight with their vendors. KeySpan’s Smith remembers that when she was with Verizon, the telecom giant used to take on upgrades earlier than most customers because it had enormous clout with its vendors. “The vendor would put a big staff of its top people onsite to help us through it,” she says. “At KeySpan, we have a tough enough time getting our vendors’ attention without risking an early upgrade.”

Epilogue: The Upgrades Bottom Line

There is one other strategy that CIOs could pursue: treat the upgrade process as a greenfield. At Nextel, making the shift to the new version of Oracle’s ERP software will cost enough and take long enough that LeFave is calling it a new installation and inviting Oracle’s main competitors, SAP and PeopleSoft, to bid on installing their system at Nextel. “We’ve worked hard to build an integrated architecture with ERP” that is heavily integrated with software from other companies, says LeFave. “All of a sudden you have to make this major leap of faith, and that opens up a discussion around should we be doing it with these guys or should we look into the future with someone else?”

The stakes have risen for enterprise upgrades. These projects need to add value, and value is no longer defined by squeaking through an upgrade before the desupport date for an older version. A good planning process, along with regular meetings with fellow users at other companies who can help you organize to get what you want from vendors, is critical.

But old habits die hard. Amazingly, 21 percent of the AMR survey respondents sold their enterprise software upgrades to their business based on vendors’ announcing desupport dates for the software. That’s not planning. That’s desperation.

As CEOs and CFOs tighten the reins on IT spending, Brother International’s Upton pities the fool that tries to get away with such a sales pitch in the future. “If you’ve got a multimillion-dollar ERP system, there better be a better reason for upgrading than the fact that the vendor won’t support it anymore,” he says.