'Humanised' computers that know what you're thinking

Peter Dinham
Friday, 23 October 2009 06:12

IT Industry - Market

Researchers at the University of Surrey in the UK have created an automatic system to spot non-verbal social signals in natural conversation, allowing computers to better understand meaning in speech, and ultimately enabling more intuitive computer interfaces. 

The findings of research by the University of Surrey team were presented at the recent IEEE International Workshop on Human-Computer Interaction in Kyoto, Japan, following initiation of a study into lip reading which identified the need to provide more than the literal words for useful understanding. The research is being carried out within the Centre for Vision, Speech and Signal Processing at the university.

The leader of the research team, Tim Sheerman-Chase, said with the technology social cues such as agreement, understanding, thinking and questioning are detected in continuous video. The two other members of the research team are Dr Eng-Jon Ong and Dr Richard Bowden.

According to Sheerman-Chase, humans unconsciously use body gestures, emotions and gaze direction to understand the meaning of spoken language.  The automatic recognition of communication signals provides a valuable tool for computer interfaces and the study of social situations.

In trials, Sheerman-Chase said human conversation was recorded with minimum intervention of the experimenter, and interesting clips from these conversations were rated by21 annotators in a web browser. 

“This provided clear examples of 'thinking' and 'not thinking', along with positive and negative examples of the other non-verbal signals. A computer learned which parts of the face could be used to identify each social signal in video.

Commenting further on the research, Tim Sheerman-Chase said “this is a new direction in emotion recognition.  Most previous work focused on actors or artificial social situations.  The ability for computers to understand meaning in natural conversation is key to being able to use our innate communication skills to use computers.

"Although the accuracy of the system is far from perfect, it is comparable to human performance for some types of social signals.  The complexity of everyday conversations makes even humans disagree on what is happening."

According to Sheerman-Chase, recognition of communication signals can be applied to a range of applications including making computer game characters interact in more natural fashion, determining user experiences in real or virtual environments and safety critical applications. 

He said future work will involve studying other social situations and cultural differences.