"When Britney Spears comes out with her new CD, you have no idea how it’s going to do," points out Forrester’s Kilgore. Given this lack of historical data, models that can take into account potentially significant variables suggested by a knowledgeable human being, such as record company promotions or regional influences, can be effective.

"There’s this mental voodoo that people are doing based on some report they print out and bring to the meeting," says Randy Mattran, IS leader for business intelligence at Best Buy in Eden Prairie, Minn., a prospective user of predictive analytics. "We’re trying to capture some of that voodoo in the system."

Predicting Success

Customers that implement predictive intelligence technology almost uniformly emphasize three success factors: being clear about the goals of the project, having good data inputs, and having people who can understand the models and help translate their output into action.

"You need to take it a little bit at a time, set some short-term objectives and understand exactly what you’re looking to get out of it," says Allen Brewer, CIO of AIG’s e-business risk solutions group in New York City. Brewer’s organization uses predictive software from Computer Associates along with homegrown algorithms to evaluate potential credit insurance customers, helping to accurately predict, for example, three bankruptcies in the first quarter of 2002. He says that without the software, AIG couldn’t have entered the midsize business market for its credit insurance products.

Start with a specific goal in mind, users say, and figure out how to frame the problem and which models to use. "Does the model fit? Can you include the relevant variables?" asks Best Buy’s Mattran. Focusing on the right data, he explains, is more important than the underlying mathematical algorithm in determining whether "your scorecard improves versus what you did with a mental model and Excel spreadsheets."

Second, focus on getting the right data inputs and on working with clean data, which means having the right cleansing routines and sanity checks on data quality. There are a lot of exogenous variables that most companies don’t account for in the data warehouse, says one IT manager at a large manufacturing company, noting such things as data on upcoming retailer promotions, which may not be routinely fed back into a manufacturer’s forecasting system. The manager also notes that making predictive analysis work requires significant up-front investment in providing current data.

And finally, make sure there are people on staff who know how to operate the model, understand the output and are prepared to take action based on it. "Companies should be very leery if they think they can use these tools in-house without any kind of expertise," says Forrester’s Kilgore, referring both to modeling experts as well as people who can translate the models into plain English for end users. The models need to be understandable by the managers who will use the output?they can’t just be a black box.