Your computer knows you're in a bad mood. Or, at least, that's one possible future. Researchers have developed an algorithm that can recognize a person's facial expressions, and categorize them as anger, disgust, fear, happiness, sadness and surprise.

Researchers at the Department of Artificial Intelligence (DIA) of the Universidad Politécnica de Madrid's School of Computing (FIUPM) worked with Madrid's Universidad Rey Juan Carlos. The prototype software they developed can process a sequence of moving faces and recognize the person's facial expression. "The software can be applied to video sequences in realistic situations and can identify the facial expression of a person seated in front of a computer screen," according to a university statement.

The technology looks at 30 "boxes" per second, evaluating the emotion displayed in the user's expression.

The software monitors facial movements in several parts of his face, examining up to 30 images per second. The data is compared to expressions captured from 333 sequences of different people from the Cohn-Kanade database, with an 89 percent success rate. "It can work under adverse conditions," according to the statement, "where ambient lighting, frontal facial movements or camera displacements produce major changes in facial appearance." The results of this research were published in the January issue of Pattern Analysis and Applications.

Putting Your Best Face On It

According to the researchers, applications that might take advantage of these capabilities include advanced human-computer interfaces, metaverse avatars and e-commerce. One example offered is its suitability for better e-commerce responsiveness, including judging a prospective customer's response to a remote sales pitch or product demo.

Gaming is high on the list—well duh!—but that doesn't mean the technology is irrelevant for more common business endeavors.

One obvious use is in social networking. For example, Jacques Van Niekerk, a software architect at MIH Internet, imagines that when the algorithms are optimized and ready for commercial use, they could help explore the social graph. You could upload a picture of a friend, he suggests, and find all matches. "Presumably, the search engine would scrape/search social networks, and use the avatar images to match to the uploaded image," he suggests. "The result of the search would be a list of websites and social networks on which the image appears, which could be transformed into social graph information." The emotional value might be used to reduce the number of image matching errors by matching only images that register the same emotional content, Van Niekerk contemplates.

Paul Williams, a software architect at LexisNexis Examen, believes the technology would make a great usability testing tool, because it would help developers learn whether users were frustrated by the software or device. "I would think this kind of objective measurement would be far more useful and accurate than subjective measurements, such as surveys, questionnaires or even third party observation," he reflects. Reliable facial expression software might be useful during interviews to gauge personality responses to situations, he suggests, or even in education, to determine how well students respond to a teacher's methods.