Modeling complexity: Iterative risks and opportunities

Leveraging technology to help pinpoint, understand, and model the impact of potential points of failure.

Lego Death Star
Dude of Lego (Creative Commons BY or BY-SA)
Current Job Listings

Single points of failure rarely happen in real life – they’re literally the stuff of movies.

Take the original Death Star, for example: This planet-destroying weapon of terrifying proportion and power, this ultimate expression of sophisticated evil and utter darkness—had a thermal exhaust port through which Luke Skywalker fired a couple missiles. Boom—problem solved. (At least until Death Star II came along.)

It’s worth noting that this cinematic single point of failure phenomenon was essentially the same thing Eddie Murphy capitalized on in Beverly Hills Cop when he disabled the vehicle of two LAPD cops assigned to follow him—he stuffed bananas in the tailpipe.

Boom—problem solved.

In real life, however – whether thankfully or not – points of failure are rarely so discrete or so easily identified. In our galaxy, more often these points of failure are complex, myriad, masked by successive iterations or processes that compound them and make it difficult to distinguish root cause from symptom. On the surface, these undesirable outcomes or “failures” may even seem amorphous or so singular as to be considered one-offs, anomalous outliers attributable to nothing except the whims of fate or the perfect storm. Impossible to anticipate and thus difficult to guard against; difficult to recognize in real time or in time to do anything about them, because no two ever look exactly the same.

The good news is that technology exists to help us pinpoint, understand, and model the overall long-term impact of potential points of failure. We can quantify and forecast the cost to correct them, the cost not to correct them, and the ever-changing probability of them occurring again in the future (with greater or lesser frequency).

This is but one of the benefits of data mining, machine learning, and predictive modeling. When we turn loose the algorithms that uncover previously hidden clusters, correlations, and patterns—and as well-built, continuously self-improving predictive models test and assimilate new hypotheses and learnings into their iterative decision-making processes—we are identifying the very nuances and complexities that make real-world root-causes so impossible to identify manually.

At the scale of big data, no team of human or intergalactic beings—no matter how gifted or diligent—can ever uncover these myriad connections and trace/model the impact of each one. And even if they could, the pace at which they would proceed would never keep up with the velocity of big data (to say nothing of its variety). These millions and millions of potentially relevant data points—and the difficult-to-uncover correlations and connections between them—can only be managed and understood through applied mathematics and data science.

Returning to the example of the Death Star’s exhaust port: it was ultimately the most important anomaly because it was the one that Luke Skywalker and the Rebel Alliance knew about. But they only learned of it through chance – through luck. In the real world, no organization can count on luck alone to identify risks and opportunities for improvement. The opportunities are too many and too diverse; the risks are too complex and too subject to change.

Yet with comprehensive data aggregation and effective technology to mine and model that data, organizations can pinpoint the risks and opportunities that matter most—when they matter most.

Star Wars fans can be thankful that The Empire didn’t have better modeling capabilities, or thankful that the Rebel Alliance had such good luck. But all of us can be thankful that, in the real world, we don’t have to count on luck (or manual analysis) alone.

This article is published as part of the IDG Contributor Network. Want to Join?

Copyright © 2016 IDG Communications, Inc.

How do you compare to your peers? Find out in our 2019 State of the CIO report