Partnerships: Building a Big Data Village

It is time to focus on breaking down the social silos that impede efforts to adopt agility and build internal partnerships.

small businessmeeting copy

The (big) data village: Get value through fostering internal partnerships

Within most corporate organizations, many (big) data programs can and will fail to create sustained business value. To avoid this risk, and to gain maximum value from data investments, organizations must overcome a number of stumbling blocks within the analytical supply chain — notably a lack of appropriate people, process or technology. Primarily we must focus on breaking down the social silos that impede efforts to adopt agility and build internal partnerships.

Entering Big Data 3.0, where agility has taken the spotlight, the traditional linear paradigm of data management is being challenged to return value. Even with access to large computing power, analysts, DevOps and IT, we know that without full alignment of the business and IT and a focus on breaking down the data silos, outcomes are likely to be less than optimal. In the worst cases, it becomes challenging to the point where project teams may perform their work with as much duct tape as best wishes.

Oftentimes I hear that organizations complete a project only to discover there was a parallel project, sometimes even two other projects, in the pipeline by other teams. This lack of synergy almost predicts that the output will be brittle and fall short of targeted outcomes. From there, moving that process into business value incurs additional technical debt translating it to other groups.

 Business units need to create  and own a data strategy

One method to combat this complexity is assuring that the people closest to the data collaborate and take ownership in the project, to guarantee its refinement and utilization, just as a business would do with any precious commodity.

In a perfect world, data and the information it carries for use in the rest of the company should be integrated, protected and shared with the teams who bring a business case for access to the data. It’s imperative that the business units within an organization understand the value of the data and the goal of their program, and that the custodians of the data work with the teams who need that access to the data.

Here are some ways to facilitate data protection and collaboration:

  • Centralize and democratize.  The team members who work on classifying the site’s search engine know what has been in demand for customers and can share their unique interpretation with the customer team evaluating positive chat experiences. Valuable and related, having their data accessible to others lets them to begin to experiment and ask questions. If another team has a better way to perform the work, let the teams come to an agreement.
  • Create a business domain stewardship team. The first step to reducing complexity is providing awareness of the data and its business interpretation. Designate a data librarian and a group of subject matter experts within the business units who can help teams know where to find the data and foster those communities to come to common agreement.
  • Apply pre- and post-implementation teams. Before concerns of creating data swamps begin to creep into conversations, begin putting in place consultation at the front end. This upfront work can help you find the right solution and communicate the project’s intent, which is one of the keys to finding partners who can help in the analysis. At the tail end, have a team who can take apart the end product to harden it and make the output reusable to other teams.

Agility, not anarchy

Analysts and data workers will always find workarounds — on their desktops or in their private silos — which can lead to data swamps and increased frustrations on all sides. While the business strategy and infrastructure is being designed, take advantage of bottom-up approaches:

  • Invest in self-service capabilities. With the goal of centralization, business data teams should be able to perform their discovery and prototyping in a manner that is shared, secure and most of all trackable. Knowing who and where data is being used provides that transparency to grow.
  • (Re) Evaluate your processes for ETL. Take a good look at your existing infrastructure and determine where Hadoop systems could decrease the cost of pre-processing many of the jobs. That valuable processing time of your most expensive hardware can be better suited for answering business questions, and the process will undoubtedly bring down the number of separate versions of the data.


As with many companies, to meet deadlines, this balancing act is being performed every day: Business teams strive to get quality answers at the fastest time to market. Across the skill set spectrum, due to challenges of data access, availability and proper governance, these teams spend the majority of their time scrambling for data acquisition and cleansing of the “sounds right” rather than the “best available” commonly shared data. For large-scale projects, the risks are compounded due to scope changes and having those results being out of alignment because they might have been rebuilt from data sources of the “readily available.”

Big data programs are often founded upon successive wins, showing that agility is possible. With the assistance of an information management team and its related support mechanisms, successful initiatives can be fostered and conditioned across an enterprise structure.

Although the effort to build a data program oftentimes requires a great deal of upfront work, a successful outcome is worth the investment. Finding like-mined, optimistic, collaborative and talented colleagues to build the company-wide data village is the key stepping stone to ensure you are building a program focused sharply on the steps it takes to find success.

Jeff Weidner is the Director of Information Management for Marketing, Dell EMC.

Copyright © 2016 IDG Communications, Inc.