Analytics 50: How big data innovators reap results

Five winners of the 2016 and Drexel University Analytics 50 awards share details of their projects, lessons learned and advice.

1 2 3 4 5 6 Page 4
Page 4 of 6
The New Mexico Department of Workforce Solutions team. New Mexico Department of Workforce Solutions

The New Mexico Department of Workforce Solutions team.

New Mexico Department of Workforce Solutions: Predicting bogus payments

The New Mexico Department of Workforce Solutions (DWS) has struggled for years with erroneous unemployment insurance (UI) payments. It isn’t alone — government agencies across the country face the same problem. In 2014, more than $4 billion in erroneous payments were made in the United States. The DWS has applied predictive analytics and behavioral science techniques to curb the problem.

In 2014, nearly one dollar out of every eight distributed under UI programs in the U.S. went to someone who was ineligible, says Joy Forehand, deputy cabinet secretary of the DWS. While identity theft and similar criminal schemes have grabbed headlines, they actually account for less than 5 percent of the total cost, Forehand says. In an effort to tackle the other 95 percent of activity that results in improper payments, the DWS set some goals: Enhance program integrity, reduce overpayments without hurting eligible claimants, and increase collection efforts without expanding the collections team.

“We needed to really understand the realities of our improper payments,” says cabinet secretary Celina Bussey. She adds that the department has taken steps to combat criminal fraud schemes, “but, under the surface, there are the core issues that cause the overwhelming majority of improper payments.”

In collaboration with Deloitte Consulting, the DWS found that improper payments are generally the result of claimants doing one or more of the following things: not looking for new jobs, not properly reporting income they earn while collecting benefits and incorrectly reporting the reason for the separation from their employer.

With that data in hand, the agency launched a project that it called the Improper Payment Prevention Initiative (IPPI). Working with Deloitte, the DWS developed a predictive model based on patterns of past overpayments. It identifies individuals at a higher risk for overpayment. Behavioral science and “nudge” techniques are then used to prevent overpayments by reminding claimants to follow the rules.

Pop-up reminders

The department uses messaging, including certification boxes and pop-ups, to remind claimants to review their information for accuracy and completeness at three critical moments: filing the initial application, reporting work and earnings, and making plans to seek new employment.

“We wanted an innovative approach to prevent improper payments from happening in the first place,” Bussey says. “Individuals have to submit required information on a weekly basis in order receive unemployment benefits. We were able to determine who is at a higher risk for reporting inaccurate information. The predictive algorithms were developed and tuned to historical cases of overpayment to isolate situations at the highest risk of overpayment. As a team, we knew that we could possibly prevent improper payments if we nudged the individual to change behavior and provide accurate information upfront.”

Moreover, Bussey adds, “we needed to not only understand the analytics, but then also understand why our customers make certain decisions.” Armed with that data, the agency turned to “the science of behavioral nudges” to encourage claimants to make the right decisions, she says. “We chose to test three types of behavioral nudge techniques: certification boxes, enhanced screens and pop-up messaging.”

To ensure that the combination of predictive analytics and behavioral science would be effective, Bussey says the state set up a randomized trial to test hundreds of combinations of message layouts, wording and more.

Successful rollout

The IPPI project launched smoothly in May 2015, and Bussey says claimants who see the reminders are 40 percent less likely to file improper claims. The tools have helped state investigators find 28 percent more overpayments with the same level of staffing. They also detect overpayments an average of eight weeks faster. Agency officials say the approach is expected to reduce earnings fraud by 35 percent, amounting to $1.9 million in savings for New Mexico annually.

“The best advice I could offer for other organizations, particularly government agencies, is to not feel overwhelmed by the concepts of predictive analytics and behavioral science,” Bussey says. “While they will challenge you to rethink many internal processes, procedures and current ways of thinking, the potential benefits of projects such as this are worth the effort.”

1 2 3 4 5 6 Page 4
Page 4 of 6
7 secrets of successful remote IT teams