Beware the Frankenstack!

By Bryan Kirschner, Vice President, Strategy at DataStax

During this Halloween season, we can learn something from Mary Shelley’s classic horror story “Frankenstein.”

In the original, it’s a tragedy about “good intentions” gone terribly wrong.

Dr. Frankenstein was smart and ambitious. By discovering how to impart life, he hoped to create a beautiful new form of it; unfortunately, the animation process resulted in the so-called monster having a fearsome appearance.

For its part, the monster wasn’t evil by nature and craved acceptance. He befriended a blind man, only to have his hopes dashed when his appearance provoked terror and hostility from sighted members of the family.

Things went downhill from there.

When it comes to your data infrastructure, even the best of intentions can still result in something of a beast—a “Frankenstack,” if you will, composed of piece parts that collectively wind up provoking fear and frustration. Organizations need to move quickly, but as teams race to deliver new products or update existing ones, they risk building technical debt by taking a slapdash approach to building their data architecture.

Over time, this can add up: McKinsey and Company found that technical debt (defined as “off-balance sheet accumulation of all the technology work that a company has to do in the future”) amounted to up to 40% of enterprises’ entire technology estates. A jumble of bolt-on, non-standardized solutions that are poorly integrated and can contribute to a lack of agility, increased costs, and reduced developer productivity.

Why Frankenstacks are scary

Some of the characteristics of a “Frankenstack”, and the technologies that it constitutes include:

Lock-in

Previous technology choices can make data portability or app modernization daunting. Teams may wind up bearing higher costs or suffering from diminished agility because they feel locked-in by earlier decisions.

Sprawl

If you’ve adopted a microservices architecture, teams might have selected databases while laser-focused on their velocity in shipping an app, without thinking of the bigger picture: how do these choices fit into the broader context of their organization’s architecture? Challenges will begin to crop up once data itself is treated as an asset; the important goal of getting it to flow across the organization could become difficult.

Orphaned solutions

Beauty is in the eye of the beholder—but once the beholder is gone, things can get ugly. A tool or technology may have been chosen on the basis of one passionate advocate, and now they’ve moved on. No one’s keen to master it, nor to take on the work of replacing it.

Balkanization

There’s rarely a “magic bullet” in technology. But ideally teams converge on a standardized set of solutions around which communities of practice coalesce, enabling knowledge exchange and robust collaboration. One sign of a Frankenstack can be different teams using “not-quite-the-same solutions” for similar use cases—without good reason to do so.

Taming the monster

The good news is that there’s a variety of proven, effective patterns that help to avoid creating a Frankenstack.

The first is most associated with leading data-driven enterprises and those that are already generating material revenue from data and analytics: intentionally assembling a “best-of-breed” data infrastructure. Those who also have the highest expectations for the growth of data lean this way: two-thirds (67 percent) of organizations expecting 50X data growth told us this was their strategy in a survey of over 500 organizations.

Today, leading-edge, best-of-breed data stacks that are designed to push the boundaries of compelling experiences lean heavily into the proven track record of open source technologies including Apache Cassandra, Kafka, Spark, and, lately, Pulsar and Flink. In another survey we conducted, 47 percent of respondents from organizations building containerized apps with Kubernetes told us they use open source whenever possible, while only four percent said they use open source only occasionally.

But the second success pattern is also a way to create a foundation for evolving your data estate intentionally: 43 percent of organizations in the same survey were looking to use a single vendor as the basis of their modern data platform. This strategy is less associated with outsized results—but it’s a viable way to create a coherent core that can then be complemented or extended.

The key in either case is to recognize that managing your data estate needs to be intentional and ongoing. In his loneliness, Frankenstein’s monster lamented: “I was dependent on none and related to none.” Your developers, operators, and data scientists are dependent on your data stack. If you’re a CIO or CTO, how your data stack evolves is dependent on your vision and leadership. Make it dependable and thoughtfully supported, now and in the future.

Learn about more attributes of companies that we describe as data leaders in the free DataStax report, “The State of the Data Race 2021.

About Bryan Kirschner:

Bryan is Vice President, Strategy at DataStax. For more than 20 years he has helped large organizations build and execute strategy when they are seeking new ways forward and a future materially different from their past. He specializes in removing fear, uncertainty, and doubt from strategic decision-making through empirical data and market sensing.