Study: Untamed analytics will take your business on a wild ride

Predictive analytics can speed up business decisions and create a competitive advantage, but the models need better governance to avoid having some predictions turn into business fiascos, according to research supported by the Society for Information Management’s Advanced Practices Council.

Data science is a nascent field where organizations are trying to find the best governance approach to mitigate the risks of analytics failures, according to the research paper by academic Michael Goul. The paper is titled “Taming the Wild West of Predictive Analytics and Business Intelligence.”

For all the good that data analytics can do, the models can also “go awry, resulting in unintended consequences,” if the underlying assumptions are faulty or become outdated, the report says. Analytics can also produce business results that are controversial or run afoul of legalities, such as anti-discrimination laws.

[ Also on CIO.com: 5 interview questions for big data engineers… and how to answer them ]

Data analytics models need “accountability, transparency and traceability,” the report says. Governance teams should be set up to assess data quality, security and compliance, for example. One governance approach is periodic “back-testing” of predictive models against new data and assumptions to see if the model still holds up.

The report summarizes the emerging best practices for governance at various organizations, including Adidas, American Express and State Farm Insurance. “State Farm established a testing culture in which analytic solutions are assessed to determine whether they are performing as designed and promoted,” the report says. “A testing team comprised of a business partner, a statistical designer, an execution specialist, and a structural and creative design expert” run controlled A/B experiments to evaluate the performance of analytics applications.

The Advanced Practices Council, a program for senior IT professionals, fosters independent research on member-chosen topics.

Goul is the associate dean for research at the W. P. Carey School of Business at Arizona State University. He is also a professor of information systems, with research interests in business analytics and data model ownership.

[ Next read this: QlikView vs Tableau: Which BI tool is best? (enterprise user reviews from IT Central Station) ]