Gaussian mixture model based estimation of the neutral face shape for emotion recognition

Abstract

When the goal is to recognize the facial expression of a person given an expressive image, there are mainly two types of information encoded in the image that we have to deal with: identity-related information and expression related information. Alleviating the identity-related information, for example by using an image of the same person with a neutral facial expression, increases the success of facial expression recognition algorithms. However, the neutral face image corresponding to an expressive face may not always be available or known, which is known as the baseline problem. In this work, we propose a general solution to the baseline problem by estimating the unknown neutral face shape of an expressive face image using a dictionary of neutral face shapes. The dictionary is formed using a Gaussian Mixture Model fitting method. We also present a method of fusing shape-based (geometrical) features with appearance based features by calculating them only around the most discriminative geometrical facial features, which have been selected automatically. Experimental results on three widely used facial expression databases as well as cross database analysis show that utilization of the estimated neutral face shapes increases the facial expression recognition rate significantly, when the person-specific neutral face information is not available. © 2014 Elsevier Inc. © 2019 Elsevier B.V., All rights reserved.