It’s a good day to corral data sprawl

By Bryan Kirschner, Vice President, Strategy at DataStax

The dictionary definition of “Wild West” is “the western United States in its frontier period characterized by roughness and lawlessness.”

In our latest survey,* respondents expressing the most confidence in their organization’s progress toward success as a data-driven enterprise were also most likely to strongly agree that “our data situation is basically the Wild West.”

These same organizations are also leading adopters of microservices. We’ve observed that microservices can create velocity and agility for development teams at the price of data sprawl for the enterprise.

Teams can build fast and iterate rapidly when they are empowered to choose tools and technologies they find best fit for purpose to meet a service level objective (SLO).

But that SLO is typically an API contract for the service itself (such as “update an order” or “log in a user”). There typically hasn’t been a focus on a SLO for the data generated by the service.

As organizations get more data-driven, demands for data and “data about the data” increases.  “Who called the service, when, and what was the latency they experienced” may become valuable for robotic process automation (RPA), real-time, algorithmic recommendations, a  360-degree view of customer interactions, or all three.

Free, fast, and reliable flow of data may enable A/B testing in real time–routing some customers to chatbots versus human support agents,  for example.

But the databases behind most microservices were allowed to be heterogeneous, making this challenging.

As one developer told me: “I think [our databases] are kind of hodgepodge. I mean, we have them obviously. We use them for all different kinds of things…There hasn’t been a clear cut strategy.”

Or as one manager confessed: “Our developers do not think about the operational impact of just going for developer ease of use.”

Digital natives (including Netflix and Yelp) addressed this with gateways in front of strategically chosen databases. Development teams can use simple APIs without needing to learn about the underlying database.

Data-driven companies are following suit. Two-thirds (67 percent of them) of respondents from the leaders  say all three of these things are true in their organization:

• Our data situation is basically the Wild West.
• We are engineering a modern data stack, piece-by-piece.
• We have a clearly defined role or team responsible for ensuring the quality of data from core processes or building a data platform that abstracts developer interaction with databases.

Teams trying to corral data sprawl have a new tool in their toolbox with Stargate for Apache Cassandra.

Imagine, for example, every microservice team that decides getting and putting JSON documents is fit for purpose for building fast, with a familiar vernacular can turn to APIs backed by a centrally managed (or as-a-service) Cassandra backend.

Database heterogeneity is reduced, a consistent standard for availability is established, and the horizon for scalability on demand is increased.

My colleague Denise Gosnell’s enthusiasm for JavaScript development with schemaless JSON is a great way to understand the potential for a win-win for both microservice and data platform teams.

*DataStax and ClearPath Strategies surveyed 515 executives, managers, and technical practitioners in U.S. companies during October, 2020.

Numbers point the way to your data-driven enterprise here.

About Bryan Kirschner:
Bryan is Vice President, Strategy at DataStax. For more than 20 years he has helped large organizations build and execute strategy when they are seeking new ways forward and a future materially different from their past. He specializes in removing fear, uncertainty, and doubt from strategic decision-making through empirical data and market sensing.