The leader of the research team, Tim Sheerman-Chase, said with the technology social cues such as agreement, understanding, thinking and questioning are detected in continuous video. The two other members of the research team are Dr Eng-Jon Ong and Dr Richard Bowden.
According to Sheerman-Chase, humans unconsciously use body gestures, emotions and gaze direction to understand the meaning of spoken language. The automatic recognition of communication signals provides a valuable tool for computer interfaces and the study of social situations.
In trials, Sheerman-Chase said human conversation was recorded with minimum intervention of the experimenter, and interesting clips from these conversations were rated by21 annotators in a web browser.
“This provided clear examples of 'thinking' and 'not thinking', along with positive and negative examples of the other non-verbal signals. A computer learned which parts of the face could be used to identify each social signal in video.
Commenting further on the research, Tim Sheerman-Chase said “this is a new direction in emotion recognition. Most previous work focused on actors or artificial social situations. The ability for computers to understand meaning in natural conversation is key to being able to use our innate communication skills to use computers.
"Although the accuracy of the system is far from perfect, it is comparable to human performance for some types of social signals. The complexity of everyday conversations makes even humans disagree on what is happening."
According to Sheerman-Chase, recognition of communication signals can be applied to a range of applications including making computer game characters interact in more natural fashion, determining user experiences in real or virtual environments and safety critical applications.
He said future work will involve studying other social situations and cultural differences.