The Cost of Data Management

One of my all-time favorite movies is All the President’s Men. The best line in the movie is when Deep Throat (played by the great character actor Hal Holbrook) meets with Bob Woodward (Robert Redford) and hears him explain how his investigation has hit a dead end.

After a pause, Deep Throat coolly advises: “Follow the money.” Those words become the driving force behind the Watergate investigation that ended with President Nixon’s resignation.

This memorable scene comes to mind as I’ve listened to many of you talk about your biggest pain points today. Right after the economy, the three most common words would be “follow the data.” While the cost of storage keeps falling, the cost of data management keeps climbing.

IDC (a sister company to CIO) reports that, with a compound annual growth rate of almost 60 percent, the digital universe is projected to hit nearly 1.8 zettabytes (1,800 exabytes) by 2011. That’s a tenfold increase within five years.

What are the core factors driving this enterprise data explosion? Eric Knorr, editor in chief of InfoWorld, identifies those factors in five categories:

Security log files. Driven by compliance mandates, companies now record all user file access activity in raw form.

Network and system events. By default, modern systems log everything, while enterprises still struggle to make better use of this unstructured data.

Multimedia. From training videos, podcasts and medical images to multimedia communication, media of all sorts pour onto storage servers daily.

Transaction records. All that structured transaction data has to be readily accessible for business intelligence and other uses.

E-mail and other messages. E-discovery legal requirements call for all sorts of communications to be archived and available.

All of the above is causing the likes of HP, Oracle, IBM, SAS, Microsoft, EMC, NetApp, CA, SunGard, VMWare, Amazon, Google, Riverbed, Cisco and others to expand their offerings to help manage, secure, search, log, back up and archive data as we attempt to turn it into actionable business knowledge. No easy task, but clearly a rising corporate priority these days.

I’d love to hear what you’re doing to “follow the data” at your company. Drop me an e-mail anytime at mfriedenberg@cxo.com.