News Feature

How to Conduct Post-Implementation Audits

By Meridith Levinson

October 01, 2003 — CIO — As soon as Michael Baker Corp.’s IT department finished installing a Web-based procurement system from ePlus, Bruce Higgins, CIO of the $405 million engineering and construction company, was bombarded with inquiries from colleagues about the system’s effectiveness. He had ample anecdotal evidence suggesting that orders were getting turned around more quickly, but he didn’t have tangible proof that the system was improving efficiency.

So he decided to conduct his first ever post-implementation audit (PIA) of the new system. A PIA is a top-to-bottom evaluation of the hard and soft benefits derived from a strategic information system, the security of that system and the project management process for deploying it. From the PIA, Higgins learned that because IT miscalculated the number of people needing to use the new system, the ROI was driven down by the cost of ordering additional licenses. The PIA also showed that the system was saving the company more than $150,000 yearly, however, so Higgins was able to prove the system’s worth to his colleagues. And he learned some valuable lessons on what not to do on subsequent projects.

CIOs would do well to follow Higgins’ lead in realizing that a PIA is a worthwhile way to prove the value of IT. But many don’t. In fact, Barbara Gomolski, a research director with Gartner, estimates that a mere 20 percent of companies take the time to conduct PIAs.

Companies avoid post-implementation audits for many reasons: They take too much time and drain away valuable personnel resources—two things currently in short supply. They require reams of documentation so that processes and results can be validated. Finally, project sponsors and implementers fear that the results of an audit, if unfavorable, will be used against them.

The CIOs at companies successfully performing audits have identified critical success factors, including: getting the right people involved, timing the audit properly, and collecting enough documentation to facilitate the smooth execution of a PIA. They pick the right projects to audit (for more information on this, see "How to Select the Right Project for Your First Post-Implementation Audit" at www.cio.com/printlinks). And these CIOs share several key traits on how they ensure that PIAs become a sustained practice in their organizations: They are all committed to continuous improvement, they’ve made PIAs a part of their project management methodologies, and they have their CEOs’ support.

But companies not performing PIAs are missing out on the important benefits such data provides. Although the challenges and risks associated with PIAs seem formidable, 66 percent of this year’s CIO 100 honorees always or frequently conduct them, according to a CIO survey. Since the honorees were chosen for their exceptional resourcefulness, their prevalent use of the audit establishes it as a best practice among highly effective organizations. PIAs provide a thorough approach for proving the value of high-cost, mission-critical IT investments and for gleaning project management best practices, which CIOs can then apply to keep subsequent projects on track. At a time when the value of CIOs, IT departments and IT investments are under increased scrutiny, PIAs are now more critical to CIOs’ success and survival than ever before. "With the pressure on business today and the responsibility of IT to help the business units understand where their dollars are being spent, I really have trouble with anyone saying you shouldn’t be doing [PIAs] in this [hard economic] time," says Jim Smith, CIO of Sun Life Financial and a strong advocate of PIAs.