View article

During the last decade, automatic sensing of affect and social signals (for definitions see the Glossary) has emerged as a hot topic in the field of Human-Computer Interaction (HCI). In the long run, the ultimate goal of these efforts is to build machines that allow for natural and fluent interaction. This inevitably means that the time that may elapse between the occurrence of a social cue (eg, a facial smile or a change in intonation) and the moment when it is perceived and interpreted by the machine is limited. Hence, the recognition of social behaviour is not only a question of accuracy, but also of efficiency. In fact, we may prefer a quick guess over a precise but time-consuming decision. At the same time, social behavior reveals itself as a complex interplay between various modalities (face, speech, body, etc.). Hence, to perceive the full picture of an interaction, we must not look at the modalities in isolation, but combine …