Mastering your data

Over the years there have been numerous attempts to solve a problem that most large enterprises face – how to get a coherent view of business performance across an enterprise, given the diversity of systems out there. It may seem trivial to want to know the profitability of your products, customer and channels, yet for most large companies it is highly problematic to answer such apparently simple systems.

Definitions and categories of “product” are scattered among separate ERP, CRM, supply chain and other systems: a survey we ran in 2008 found that the median number of systems generating product data in large firms was nine, and many had over 100 such applications.

The ERP wave was sold on the promise that the problem would go away if you replaced all applications with one, yet in reality large firms found they needed to implement multiple ERP instances, and anyway ERP only covers part of the scope.

Data warehousing was an approach to tackling the issue by gathering copies of the source data from the various applications and adding numbers up centrally, but in practice data warehouses are generally too unwieldy to respond adequately to rapid business change.

In the last few years master data management (MDM) has emerged as a further attempt to tackle the same root problem. The idea is to have central repositories that deal just with ‘master’ data such as customer, product, asset and location. A golden copy of data is held in a central master data repository and this data can feed data warehouses with clean dimensional data and potentially drive this master data back into the operational systems, removing duplication and so leading to many operational benefits such as largely eliminating duplicate mailings or incorrect deliveries.

Read Andy Hayler’s previous Opinion column on data quality

One reason to hope that MDM has at least a chance of succeeding is that it is now recognised that such a project can only work in the context of an enterprise-wide data governance initiative. The idea is that business users take ownership of their data, and set up an organisation and processes to define master data that needs to be shared (such as international product classifications). Responsibility is assigned for resolving organisational disputes, and for keeping such master data to a high standard of quality, ensuring that the same version of such data is used throughout the enterprise.

In practice, most organisations are at an early stage with such initiatives, though those who have made progress seem to be finding early success: in a recent survey we found that two-thirds of organisations with MDM initiatives rate them as at least moderately successful. However, the scale of such projects can be daunting. One company we work with has 26 people working on data governance, but after more than a year they are still working on business processes and ownership. We found a median of five full-time equivalent staff is required to support live MDM implementations, and in many cases the figure is much higher.

The barriers to successful data governance are as much political as technical. Companies with a decentralised culture are not used to working on centralised definitions of data and frequently resist the loss of control that can be implied by an enterprise-wide data governance initiative. Business lines that have traditionally operated as separate profit centres are used to being in control of their own data, and find it difficult to give others control.

There are many technical issues too. Many MDM products have evolved from technologies designed to handle a specific type of data (often either customer or product) and many simply cannot handle a wide range of data domains.

For very large organisations it may simply be impractical, both technically and politically, to have a single MDM über-hub and they may need to rely on a federation of separate hubs for different business lines or geographies, in which case there need to be mechanisms to synchronise these. It is a measure of the immaturity of MDM that most vendors get a ‘rabbit in the headlights’ look if asked about how they intend to support such an architecture. Generally MDM projects today have a more limited scope, such as customer data only in certain markets, yet our surveys show that around two-thirds of organisations want their MDM project to be enterprise-wide in scope.

For those pioneering organisations that have driven data governance through their enterprise and started to get control of their master data, the rewards can be great. I have seen several examples of projects with benefits in the hundreds of millions of dollars. Although the projects to achieve these were multi-year in duration, the rewards were sufficient to fully justify the investment. However, they require strong and sustained high-level business sponsorship.

About the author

Andy Hayler is founder of research company The Information Difference. Previously, he founded data management firm Kalido after commercialising an in-house project at Shell