Under Development: Digital Reanimation Graphics

GREAT NEWS for film and TV actors: Dying doesn’t have to be a fatal career move.

Tomaso Poggio and Tony Ezzat, researchers at MIT in Cambridge, Mass., have created a video system that will allow film and television directors to animate images of people?living or dead?and make them appear to say things they’ve never actually said. The technology could pave the way for historic comebacks for long-gone performers like Marilyn Monroe and John Wayne, among others.

The system, which Poggio calls “human animation,” records facial expressions and then associates specific images with certain sounds. Sophisticated algorithms smooth out the jumps between video frames to achieve almost flawless lip-syncing. A demonstration project?Mary 101?required eight minutes of video, which is 15,000 digitized images, to create 46 unique mouth movements. “The computer takes these 46 mouth images and recombines them into a high-dimensional ’morph space,’” says Poggio, an investigator at the McGovern Institute for Brain Research at MIT (the institute is funded by IDG’s founder, Patrick McGovern). “Using a learning algorithm in the morph space, the computer is able to figure out how Mary 101’s face moves.”

Besides regenerating the career of long-dead actors?with voices provided by old audio clips or impersonators?the technology also promises a variety of serious applications. The technique could, for example, expand the reach of media organizations by letting them translate broadcasts into multiple languages. (Imagine Dan Rather delivering the news in flawless Hindi.) Businesses might use the system to produce training videos localized for global markets. On the dark side, governments could exploit the technology to create video propaganda?literally putting words into enemies’ mouths.

The current system provides only static, head-on views and therefore isn’t quite ready for prime time. “An even better technology dealing with 3-D and expression will probably require two years,” says Poggio. If it does, Elvis may be ready to reenter the building.