Data rules take centre stage

Technology conferences have struggled for attendance over the last couple of years, yet at a data governance conference in San Diego in June the organisers were almost overwhelmed by demand, with double the expected number of attendees, and a surprising proportion of people there carrying business rather than technology job titles.

At this hitherto obscure conference the few vendors who had bothered to pay for exhibition stands were barely able to keep up with demand. This upsurge of interest in data governance is not confined to the US – I recently gave a talk on the subject to a packed house in Stockholm.

I think there has been a gradual realisation that the only way companies can really get a grip on their key data (customers, products, assets) is by engaging the business to take ownership of their data, rather than just blaming data problems on the IT department. Companies have now tried the various silver bullets of data warehousing and ERP in a bid to get a picture of how their business is running, and yet still struggle to answer very basic questions about their business, mostly due to competing data sources and inconsistent data quality.

My company recently conducted some market research to try to find out whether data governance initiatives were being linked to master data or data quality activities. A survey of 257 large companies (just over half from North America) showed that the most popular reason for kicking off a data governance initiative was supporting business intelligence and data warehousing, which confirms that companies are not getting the answers they need from their current data warehouses.

Curiously, only around half had developed a proper business case for data governance, but an encouraging 80 per cent claimed to be measuring data quality, which is an improvement on previous surveys.

The responses showed an encouraging degree of business involvement, with 79 per cent of initiatives either led by the business or jointly by business and IT. The level of effort involved is non-trivial, with a median of six full-time business staff and four IT staff dedicated to maintaining existing data governance initiatives.

Forty-two per cent of these teams had enterprise-wide scope and authority, and 68 per cent had named individuals with the authority to resolve disputes over data definitions and ownership, which seems to me to be at the very heart of data governance.

Half the respondents thought that having data governance processes in place prior to embarking on master data management (MDM) or data quality activities was key to the success of the latter activities, and 74 per cent thought that implementing data quality was essential to a successful MDM project – interesting as the technology market has always regarded MDM, data quality and data governance as fairly distinct sectors.

Encouragingly, although data governance is pretty new for most firms, many of those organisations who have implemented an initiative have been happy with the results: 60 per cent claim at least moderate success, and 19 per cent high levels of success, with only six per cent reckoning outright failure.

The main benefits were around the speed and quality of decision making, though less than a third had quantified these benefits.

Because data governance has sneaked up on the industry, there is relatively little in the way of tool support. Thirty-nine per cent of the firms surveyed were relying on spreadsheets and the like, with another 32 per cent building their own data governance tools; this represents a clear opportunity for the vendor community.

I find it encouraging that data governance, while still nascent, is clearly moving up the agenda. I have worked in companies with vast IT budgets, and seen huge amounts of money spent on rolling out ERP systems and building data warehouses.

Yet all too often such initiatives falter because the business people that sponsor such projects are insufficiently engaged, and after an initial burst of enthusiasm return to the old ways of doing things. Data quality in the systems deteriorates, and with it the trust in the reports they produce.

Then individual business departments tire of waiting for their information so set up systems on the side in spreadsheets or local databases, and after a while they end up where they started, with a variety of competing systems and an uncertainty as to which data is the right data.

Only by convincing the business that they own this data and are partly responsible for this mess will large organisations be able to get a true picture of their global business performance at a level of granularity and reliability that they need.

Data governance on its own will not deliver this, but it seems to me a key pre-requisite to success, and this finally seems to be dawning on more and more organisations.