Improving a Credit Union’s Software Quality–and Customer Service–with Test Automation

“It used to take four people five days to test the software every month,” says Kevin Bingham, vice president of IT at Arizona Federal Credit Union. “Now, we can do it in six hours.”

Bingham was explaining how Arizona Federal got an ROI of five months by implementing test automation and a structured process using TestPartner. But that’s just the measurable, quantifiable benefits. The real payoff, he points out, has been in the IT department improving the credit union’s ability to do its real job: Paying attention to making its members happy.

Related Articles

The Importance of Automated Software Testing

Podcast: 20 Top Tips for Software Testing

Arizona Federal has 750 employees and manages over $1.9 billion in assets from 232,000 member accounts in 27 branch locations. Like most credit unions, Arizona Federal relies on a vendor’s software package to manage day-to-day financial transactions. The suite of applications helps them do everything from creating a new account to preparing loan applications.

Most of the applications in Fiserv Telenavigator are updated quarterly, and the credit union has always tested each release thoroughly before deployment. Doing so provided an opportunity to resolve issues with the vendor before the software went into production.

But the testing was disruptive and time consuming, especially since it was done by Arizona Federal staff members who were not IT specialists. “The people who did the testing have day jobs,” explains Bingham. For those staff, testing the software was an additional job requirement, something that pulled them away from smiling at a customer across a desk or from crunching accounting numbers. As a result, it took 60 to 90 days to test a new version of Telenavigator; if the release arrived on January 1, it’d be deployed in March. And a new release comes out every quarter.

Plus, the line-of-business employees weren’t QA specialists, so the IT department had no way to validate the testing methodology. While the accounting and finance users had the advantage of knowing how they used the software every day, it would never occur to them to check for code coverage—much less to be familiar with the term.

Clearly, there was room for process improvement.

To learn more about process improvement and workflow, see ABC: An Introduction to Business Process Management and Workflow Gone Wrong.

Bingham had used test automation tools in a previous role, so when he came to Arizona Federal, he suggested a similar approach and began shopping for software to create maintainable and reusable test scenarios. Price was an issue, he explained, because the credit union is not-for-profit. Whatever they chose had to clearly add member benefits and improve the organization’s ability to serve those members.

Compuware’s TestPartner fit the bill. At the credit union’s request, Compuware prototyped simple scenarios for mainframe, Web and Windows applications. The functional prototype demonstrated that the process would work. The software acquisition was made in June 2007, along with a week of training. They’ve been using TestPartner since last August, and, says Bingham, it’s already made a big difference.

Arizona Federal dedicated one person for QA duties full-time; this wasn’t a new hire, but a techie coming out of the IT department who already had a relationship with the non-IT staff. “There needs to be a dedicated resource,” says Bingham. “Someone has to be focused on testing at all times.” The QA guy—his name is Jason—worked with the line of business people to create scripts that capture how the software is used. Those scripts are run against each iteration of the vendor’s software to identify problems. Moreover, the same scripts can be run every time to ensure that everything is tested every time.

For more on software testing, see Quality Doesn’t Just Happen.

It isn’t always possible to measure improvement across an entire company, but Bingham had one clear example. One key application in the Fiserv suite, called Maestro, is updated every month (not quarterly). Four people would spend five days to test it, every time. Now, with the automated tests—which can run overnight in batch mode, since the IT department has them standardized—the process is done in six hours.

Because they knew how many resources were sucked up by that single application, says Bingham, it could be used to determine the five-month ROI for the TestPartner investment. Even though, he adds, the test automation tool had a big effect across the entire application suite (not just Maestro) and Arizona Federal’s ability to deploy updates quicker.

But for Bingham, the software benefits and ROI are secondary to helping the business and serving the membership. Any time that a banker has to spend not banking is a problem, in his estimation. It’s less time that they have available to do their real jobs. The real enhancements, in his view, are in streamlining IT to provide business benefits. “IT organizations are here to support the business,” Bingham points out. “But sometimes they’re more busy doing that than learning how to improve processes internally.”

“I’m surprised that more companies don’t use test automation scripts,” adds Bingham. “It really makes your life a lot easier.”