System thinking: the clearest path to Data 3.0

We live in transformational times, and data has evolved to become the trusted currency that is transforming businesses, governments and every one of us across the globe. This evolution of data has taken place in three phases.

Data 1.0, between 1960 and 2000, was focused on specific applications such as payroll automation, airline reservations and inventory control. Even operational analytics was siloed using single systems or individual datamarts. Then Data 2.0 evolved, where well-defined, enterprise-wide processes such as order to cash and supply chain management were created to let users across the entire organization leverage data to improve their productivity. Analytics also evolved with multi-source data warehouses, data quality and MDM.

Social networking, mobile computing and cloud have now given rise to Data 3.0, where data truly powers digital transformation and fuels the next-generation intelligent enterprise. This generational shift has already created enormous value for the world economy. Organizations using data intelligently to develop new business models and capture growth opportunities have disrupted a variety of industries. In fact, the market caps of disruptive, data-driven companies like Amazon, Facebook, Google, Uber, Airbnb, Alibaba and of course Apple — are larger than the GDPs of many nations.

Data 3.0 presents five key data management challenges:

1. The sheer growth of data to be analyzed. The total amount of data worldwide is predicted to reach 163 zettabytes by 2025, a ten-fold increase.
2. We are also seeing massive fragmentation in data. From the mainframe to the cloud, and now IoT, siloed data is literally everywhere within the enterprise, and sifting through it has become extremely time-consuming and complex. Siloed data also limits the ability to analyze it.
3. There is a whole new velocity of data. Stakeholders want today’s data yesterday in order to make quick and informed decisions. The need for streaming data from IoT devices often requires near-instant responsiveness.
4. Data is becoming increasingly democratized. Everyone across the enterprise is a “data consumer” requiring access to relevant data to be effective in their roles.
5. Regulatory complexity. Finally, the growth in data also means more regulations to govern the use of that data to protect individuals from improper use.

Digital transformation can’t succeed with ad hoc tactics and disconnected systems. The technology stack is expanding – it now includes cloud, big data, mobile, IoT and social. New buyers and influencers are emerging. Companies have new stakeholders to reach and work with, inside and outside the organization. There is a whole new approach to buying, and it is all about flexibility – for example, subscription versus perpetual. And the ecosystem is evolving. Businesses are now working with new IaaS, PaaS and SaaS vendors.

All this requires an end-to-end, enterprise-wide vision of how data, applications, processes, and people work in concert to drive innovation and change. These challenges must be addressed in the process of data-driven digital transformation, and in doing so it’s critical to have system thinking around data – System Thinking 3.0. This new vision provides the flexibility to accommodate multiple vendor ecosystems, new user roles, new technical requirements, new data types and a range of new security considerations.

But vision is one thing. Execution is another.

There are five pillars that support System Thinking 3.0 and enable companies to operationalize the world of Data 3.0.

Platform. Companies must learn to think of data as a platform. That means that data is the organizing principle to innovate for the future. Data solutions must be built upon a complete and modular intelligent data platform, utilizing a micro services architecture that fits into the existing reference architecture. A unified data platform that serves as a common ground across the enterprise provides a huge advantage in today’s increasingly hybrid IT environments.

Scale. There’s a temptation to start small with any initiative, but with System Thinking 3.0 it’s very important to consider scale at the earliest stages of development. Solutions must keep up with the hyper-growth of data today, especially data hungry AI/ML applications. Companies that don’t plan for scale at the outset will be stuck once they start to operationalize their workloads.

Metadata. In today’s world of fragmented data, it’s not practical to move all enterprise data into one place. Metadata provides a map (Google Earth) understanding of all of an enterprise’s data assets which makes it easy to find, use the right data for the right applications with business context, providing trust and meeting compliance needs. By thinking “metadata-out” lets companies know where all their valuable data resides. The ability to discover data not only by where it’s stored but by what it means to a business is mission critical.

Artificial intelligence. In this world of complexity, data professionals who aren’t thinking about AI as their personal assistant are losing something big. Leveraging AI to help do the things that are difficult and time-consuming increases productivity, freeing up time to focus on higher value work. AI compliments workloads. It’s not an either/or discussion. 

Governance. System Thinking 3.0 also ensures the proper use and protection of sensitive data. Data governance, protection and privacy are design principles in this new world. Missing this means paying a big price – in reputation, customer loyalty and ultimately the business itself.

Data-driven digital transformation based on Data 3.0 is here, it’s real and it’s impacting the daily lives of executives, managers, front-line staff, and most especially the customers they serve. By relying on the five pillars of system thinking 3.0, companies can find success in the world of Data 3.0 as they move forward in their ongoing evolution.