Under this letter is a memo from the PR department (labeled URGENT) advising you that marketing’s plan to use face recognition technology to identify prime customers entering the store may be getting some negative reviews in the press, with phrases like invasion of privacy and Big Brother being tossed around.

And under that is a directive from management asking for a technical analysis of a market simulator that may be able to predict to 5 percent accuracy the response to a new product of single parents between the ages of 29 and 33 living in Illinois suburbs with incomes in the third decile.

Who are you? You’re a petabyte CIO, the person responsible for developing and maintaining the science-fiction-like applications that will be running on tomorrow’s immense storage capacities. Currently, petabyte responsibilities are mostly restricted to the IT departments of universities, research organizations, and microbiology and genetics labs. But the law of technological adoption ("If you build it, they will use it") says that sooner or later most CIOs will be crossing that line, discovering a new world of applications, responsibilities, costs and problems. So it’s time to start thinking about...

Petabyte Power

Petabyte levels of storage will make possible three new categories of applications. One group depends on retaining and processing vast amounts of visual data, especially data from video cams. Imagine cameras trained on the sales floor, recording the minute-by-minute flow of customer traffic. Imagine that data feeding an application that analyzes the relative effectiveness of a given product placement or the impact of a markdown. Marketing might be interested in learning how the proportion of couples to singles entering the store changes during the course of a two-week promotional campaign, or at what time of day the number of women shopping with kids rises and at what time it falls. With petabyte levels of storage, HR will be able to crunch a few months’ worth of video camera data to flag personnel responsible for traffic bottlenecks on the sales floor.

A second category of petabyte potential is in supporting the transition to device networks. If the first generation of networks connected people to data and to each other, the second will do the same with both physical (counters, meters, cameras, motors, switches, telephones, digital printers) and virtual devices (applications and program objects). The great virtue of device networks is that they allow any interested constituency to have remote access to any link in the production cycle. CNN, for example, is digitizing and networking all its production equipment so that pagers, cell phones, PDAs, desktops and websites will all have equal, continuous and simultaneous access to programming.