Gauging Project ROI

In Iowa state government, as in most organizations, there was a time when few outside of the IT department gave technology much thought. If an agency needed money for an IT project, officials slipped it into the budget. If a legislator asked what the money was for, a good sales pitch would close the deal and a poor one might mean no funding at all.

But along came Y2K, and officials in Iowa government started to think about the $22.5 million needed to fix the bug, and how best to spend it. To ensure taxpayers got their money’s worth, Iowa’s Year 2000 Project Office audited agencies’ progress toward compliance and reviewed every expenditure before reimbursing them from a central fund. At the end of the project, an estimated $3 million was still left, unspent, in the Y2K fund. Meanwhile, state leaders decided to examine the value they were getting from other IT expenditures. “I was a legislator for six years, and we had no clue,” says Gov. Tom Vilsack, a Democrat who was first elected in 1998.

And so the idea for the Iowa Return on Investment Program was born. Today, three years after it was launched, the program has become a national model for documenting value and prioritizing IT investments in the public sector. Last year, the National Association of State Chief Information Officers named the program the winner in its competition for the best state IT management initiative. The Iowa program helps state leaders identify the most pressing new IT investments at a time when competition for funds is fierce. In addition, it has saved taxpayers more than $5 million since its inception by enabling officials to combine duplicative projects and flag unnecessary expenditures.

Value Model

As one of his early initiatives, Vilsack, now in his second term, began promoting performance-based government, an approach to public sector management that focuses on measuring the results of government programs. He realized that the principles used to validate the Y2K expenditures could be applied to any IT project. “I think [Vilsack] saw there was a lot of value in independently verifying and validating,” says Paul Carlson, director of the IT department’s Enterprise Quality Assurance Office, which is in charge of the ROI Program.

When Carlson and then-CIO Richard Varn devised the new IT value methodology in 1999 using various ROI research sources, they incorporated the two main features of Y2K management. First, new investments are paid for primarily from a pot of money called the Pooled Technology Account, which is appropriated by the legislature and controlled by the IT department. Pooling the funds makes budget oversight easier and fights duplication. Second, the IT department reimburses agencies for expenses from this fund only after verifying that they’re necessary. If an agency’s expenditures aren’t in line with the project schedule, it’s a red flag for auditors that the project could be in trouble. The Enterprise Quality Assurance Office will consider conducting post-implementation audits next year. But Carlson says the details concerning how projects are chosen for audits and how those audits will be conducted are up in the air, pending decisions by John Gillispie, the executive director of the Iowa Communications Network, who is expected to manage the IT department after a proposed agency reorganization is complete.

To support spending decisions, agency managers have to document the expected costs and benefits according to a standard set of factors. The maximum score for each factor ranges from five to 15 points, for a maximum total score of 100 points. In addition, they must specify metrics related to those factors to determine the project’s success (see “How Iowa Scores Investments,” Page 102).

The ROI Program saved taxpayers more than $2.5 million a year during 2001 and 2002, about 16 percent of what the state spent overall on new IT projects during that period. Because the scoring system is weighted toward projects that comply with standards and directly benefit citizens, the program promotes initiatives like information security that are critical but don’t have vocal political constituencies. By encouraging cooperation between agencies, it has also transcended decades-old bureaucratic fiefdoms. Agency managers “started to see that joint projects and collaborative projects were the ones that were getting done,” observes Varn.

The program hasn’t ended political wrangling over the IT budget, however. Varn was asked to resign in December (along with several other cabinet directors), and his resignation was accepted partly because he and Vilsack did not agree on how big the technology budget should be (Varn wanted more money and a more integrated IT organization). Agencies still battle with the legislature for approval of specific projects. But it’s easier for officials to make informed decisions. “We’re able to make decisions about what should be funded based on data, rather than based on politics,” says Debbie O’Leary, the acting director of the IT department.

Everyone Knows the Score

Like many processes in government, IT investments start with a form. A Return on Investment Program Funding Application must be completed for any expenditure over $100,000, Pooled Technology Account requests, any request for money from a source outside of an agency’s annual appropriation, and any expenditure that’s not routine. Waivers that let agencies skip the ROI process are allowed for some large, routine expenses but are quietly discouraged. During the fiscal year 2003 budget approval process last year, agencies asked for 17 waivers and were granted only six. They got the message and requested only four waivers in their fiscal year 2004 proposals.

To complete the form, agencies describe the project according to 10 criteria that are used to define value. “In the private sector, what you decide to invest in is substantially based on a cost-benefit analysis,” says Carlson, who culled the criteria from value methodologies used by both public- and private-sector IT organizations. “Because we’re in the public sector, we have to take into account more than just a dollar.”

An information technology council—including the Iowa CIO, the CIOs from major state agencies, representatives from the state judicial and legislative branches, and private citizens—scores each application and ranks them. Projects can get up to 15 points for fulfilling a legal mandate or complying with enterprise technology standards. Improving customer service may garner another 15 points. The bulk of the remaining score takes into account such factors as ROI and risk, whether the project will streamline business processes, whether it affects multiple agencies and, for multiyear projects, the success of earlier phases. The application includes a methodology for assessing benefits that—as is the case with many public-sector initiatives—aren’t easily quantifiable.

Projects that protect citizens, like security, and initiatives that improve access to government services, like electronic tax administration, score consistently high.

Save a Penny,Save Millions

Iowa’s statewide budget for new IT projects has recently averaged about $15.7 million annually. The available funds typically cover about 24 percent of the total money Iowa agencies request for new investments—about $76 million next year. O’Leary has to find a way to stretch every dollar. One way she’ll do it is to roll similar projects into a single implementation. Another will be to shave costs once projects are under way.

The ROI Program requires agencies to detail their technology requirements and functional needs, which enforces standards, but it also helps officials identify duplicative expenditures. During a review of funding requests for 2001, Varn and his staff noticed that several agencies were proposing to build pieces of an ERP system, such as electronic procurement and human resources management. Varn suggested the state could deploy a single ERP system that agencies could share for less money. The project, which has an estimated cost of $9.6 million, could easily cost many times more, says Varn, if agencies were allowed to go it alone. Meanwhile, agencies are finding ways to collaborate on their own. Some requests from agencies for money to fund compliance with the federal Health Insurance Portability and Accountability Act, which took effect last month, came already bundled.

Once a project is funded, the state saves money by scrutinizing agencies’ expenses. Agencies have to submit their purchase orders and invoices to the Enterprise Quality Assurance Office for approval before they can be reimbursed. As officials discovered during the Y2K project, purchases may cost less than agencies estimated they would when they were first proposed.

New Rigor

After four years, agencies have adopted a more rigorous approach to assessing value that has brought agency technologists, finance officials and business line managers together. Business analysts in IT departments typically don’t understand finance well enough to complete the ROI analysis, so agencies need a cross-functional team to write a successful application.

“It gave us structure to say to our customers—because it’s not an IT responsibility—that you must provide answers to these questions,” says Rich Jacobs, administrator of technology and information management in the Iowa Department of Revenue and Finance. At first, agency executives were all too willing to identify the projects that they wanted and let IT handle the paperwork. That changed once officials realized that legislators were responding to the list of priorities presented to them from the ROI Program. “We felt like we didn’t just have to ask for it; we had to demonstrate that we could accomplish it,” says Jacobs.

Still, many agencies resented having to cough up so much information because they didn’t understand why all of it was relevant. Agency officials now know how the data is used, but the ill-feelings haven’t completely dissipated. The absence, to date, of full audits is one sore point, according to Jacobs. “If you say you’re going to audit, you have to be able to honor that commitment, or [providing the necessary data] becomes a waste of people’s time. No one has excess staff to start up an audit program,” he says, but the long-term credibility of the ROI program depends on it.

Getting funding through the ROI Program is, however, better than doing business the old-fashioned way, when getting a project funded had less to do with its merits than “networking and your ability to monitor budget requests,” Jacobs says. Lawmakers still choose some projects for political reasons. Carlson says that in such cases, the methodology helps improve projects by encouraging officials to comply with state IT standards. Varn thinks the program also positions agencies better for success once they get funded because they have to document technical and business risks. Before the ROI Program, he says, “a lot of people didn’t do project management, and they didn’t know what the technology was going to be. [They said] just give me money, and I’ll do good work with it.”

Jacobs confirms the benefit. The revenue and finance agency had started several small electronic tax administration initiatives in the mid-’90s that were largely funded through its operating budget. But as the state tightened its belt during the past few years, IT wasn’t a priority with the legislature. To move forward with online filing—already a reality in other states—and some related initiatives in 2001, the department applied for a grant from the Pooled Technology Fund. The process forced officials to look at the project holistically; they identified six phases that would be completed over several years.

Not long after the ROI Program was established, the legislature enshrined it in law by enacting the Accountable Government Act, which requires the Iowa Department of Management to establish methodologies for evaluating all state investments, not just IT projects. “One of the concerns I have is that people don’t see the relevance of government in their lives day to day,” says Vilsack. “As I went out on the stump, I wanted to be able to tell people that for your investment in X number of tax dollars, this is what you get.”