A long and winding roadmap

A few years ago the extract transform and load (ETL),master data management (MDM) and data quality markets were distinct ones.

There was a vague, dawning recognition that MDM might possibly need some sort of data quality component at times, and that somehow you needed to move data from your source systems into an MDM hub, but that was about as far as it went.

Even a few years ago most approaches to MDM were data domain-specific, with either customer hubs or product hubs.

Recently there has been a visible reshaping of these markets as large enterprise software vendors decided that they wanted a slice of MDM pie. Each made acquisitions to fill gaps in their portfolio.

Acquisition frenzy IBM purchased DWL, an early customer hub, as well as Trigo, a product hub, and Exeros, a data discovery tool. It has produced a roadmap as to how these products may coalesce over time, and recently complicated this picture by buying Initiate, another customer hub.

It already had a data quality tool through the purchase of Ascential, which also provided ETL capability. Getting all these products to be at least on speaking terms is the next, and far from trivial, task.

SAP purchased Business Objects, which itself had ETL capability via ACTA and a data quality tool called First Logic. The MDM component was provided through the purchase of A2i, which was relabelled SAP MDM. Again, a significant integration project lies ahead.

Oracle did more of its own developing but still managed to produce a family of MDM products, with two customer hubs (one acquired via Siebel), as well as a product hub and even an analytic MDM product acquired when it snapped up Hyperion.

ETL was provided by the purchase of Sunopsis, and product data quality by purchasing Silver Creek. The Fusion middleware project is its integration journey.

Finally, Microsoft purchased an MDM product called Stratature (now relabelled MDS) and a data quality product called Zoomix, which at some point will reappear as Data Quality Services; it already had its own ETL capability.

Such serial acquisitiveness has not been confined to the industry giants. Informatica purchased data quality vendors Similarity Systems and Identity Systems, then finally completed the picture by buying MDM vendor Siperian.

Tibco bought Velosel, a product hub, and Netrics, a matching vendor.

Software AG recently purchased MDM vendor Data Foundations, while open source ETL vendor Talend rounded out its portfolio with its own data quality offering and an MDM product based on technology acquired from Amalto.

Sensible direction What does all this mean for customers? Bringing data quality and MDM together makes a lot of sense to me, since our own research shows that 30 per cent of any MDM project is actually spent sorting out data quality.

Moreover, few companies have actually made an enterprise-wide commitment to data quality technology: as many as 30 per cent of companies don’t bother with a data quality tool at all, and those that have bought one have typically deployed it only in isolated projects.

Having data quality built into an MDM tool should encourage a wider use of the technology, which given the shambles that is the state of data in most firms can only be a good thing.

The link between ETL and MDM is a little more tenuous, but still makes sense: a great deal of the actual work of transforming and moving data about is actually driven by the need to consolidate systems, and much of the cause of the incompatibility is inconsistent master data.

However plenty of companies have made a fairly broad commitment to either ETL or real-time integration technology or both, so a tight bundling of MDM and data quality into an ETL platform may be a little more problematic for customers with an established ETL infrastructure.

They will face the choice of taking the MDM and data quality solution that their favoured ETL vendor happens to have or linking up their favoured MDM hub to their ETL infrastructure.

So what happens to those MDM technologies that have not been swallowed up in the feeding frenzy? They will need to carve out niches where they can add value to customers while touting the benefits of an independent approach.

This may be easier for companies that have specialised in specific verticals as some of the product hubs are mostly all about cross-channel retailing.

A general-purpose MDM hub message will be an increasingly tough sell as buyers will be tempted to go for the easy option of standardising on one of the platform suites.

For customers, they must hope that their chosen suite actually delivers on the integration promises of their selected vendor: history has shown that integrating multiple acquired products can be a long and bumpy road.

Andy Hayler is founder of research company The Information Difference. Previously, he founded data management firm Kalido after commercialising an in-house project at Shell