Browsing by Author "Zhalehpour, Sara"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • No Thumbnail Available
    Item
    Audio-visual affect recognition
    (Bahçeşehir Üniversitesi Fen Bilimleri Enstitüsü, 2014-08) Zhalehpour, Sara; Eroğlu Erdem, Çiğdem
    Humans express their emotions through multiple modalities, including facial expressions, speech prosody and body gestures and various biological signals. Therefore, multi modal emotion recognition has been a major interest in applications requiring natural man-machine interaction and ambient intelligence scenarios, such as security, driver safety, health-care, behavioral science, education, marketing and advertising, where the response of the system to the user depends on the estimated emotional and/or mental state of the user. In the literature, various state-of-the-art techniques have been employed for emotion recognition from single modality (mainly facial expressions and speech); but there are relatively few works that combine different modalities in a single system for the analysis of human emotional state. Recent research has started focusing on extraction of emotional features from each modality and then combining the outputs of each modality for improved recognition of the user’s emotional state. In this thesis, we present an effective framework for multimodal emotion recognition based on a novel approach for automatic peak frame selection from audio-visual video sequences. Given a video with an emotional expression, peak frames are the ones at which the emotion is at its apex, and hence are expected give higher emotion recognition results. The objective of peak frame selection is to summarize the expressed emotion over a video sequence. The main steps of the proposed framework consists of extraction of video and audio features based on peak frame selection, unimodal classification and decision level fusion of audio and visual results. We evaluated the performance of our approach on eNTERFACE’05 containing six basic emotional classes recorded in English and BAUM-1 audio-visual database containing eight emotional and mental state classes recorded in Turkish. Experimental results demonstrate the effectiveness and superiority of the proposed system over other methods in the literature.