Night of the Zombie Databases

Zombie movies have been a big sell over the last few years. They can be kind of scary (“My Name is Legion”) or kind of amusing (“Zombieland”) but one thing they always are is gross. Much the same could be said of end-user created databases …

As consumer technologies percolate into the enterprise and non-IT staff become progressively more comfortable with handling their own data management needs, an old IT problem will raise its ugly head in a new guise: Data hygiene. 

The challenge of data hygiene is keeping the data you collect and use correct (spelled correctly, in the right order, deleted if obsolete, and so on), unique (no unnecessary duplicates) and current. When you’re managing data sets at the enterprise level this quickly becomes a non-trivial exercise (and like Sisyphus pushing his boulder up a hill / http://en.wikipedia.org/wiki/Sisyphus, the task never ends).

To meet this challenge, some corporate users bring in all sorts of data collection and management software that most definitely aren’t  at enterprise scale. Suddenly, that little (annoying and non-critical) data capture project that wasn’t big enough for IT to worry about is solved by the user and is up and running in a day or two at the most.

But while IT might, if they knew you’d done such a thing, be happy that you solved your own problem so they can get on with Big Boy IT stuff, there’s a huge danger to this kind of productivity: All of these minute (from IT’s viewpoint) data management projects become a source of data “noise” in the enterprise. It’s like suddenly finding a load of zombies wandering the corporate HQ. So what do you do with them? Kill them off? Trap them and look for a cure? Ignore them and hope they’ll go away?

There’s no getting around it: As end-user databases multiply and collect more and more data, the rot will creep in. Mistakes will be made, and data will become corrupted. Entire databases will be lost, found, and recycled. They’ll be partially merged, entries duplicated in whole or in part, and mangled with wild abandon. It will become the Night of the Zombie Databases!

That handful of tiny, ostensibly inconsequential databases will morph over the course of a few months into a huge data army of zombies that the IT villagers will wind up attacking with digital pitchforks … and everyone will waste huge amounts of time and energy over what will be a combination of clean up and turf wars.

But there is a better way!

IT can take charge and educate their users on how to effectively use both IT recommended and user selected consumer tools and welcome and be supportive of staff efforts. Teach the staff how to backup their data, make sure they know how to build useful databases, be ready to help the staff when they (inevitably) run up against problems.

While there’s not a hope in hell that staff-driven data collection and management will be error- and problem-free, IT can, for a very low cost in terms of time and manpower, head off the problems before they become major time consuming (and expensive) headaches by simply accepting the inevitable and making sure that employees are educated.

The end result will be better relationships between IT and the users, better quality data, and no more gross zombie databases in the enterprise.