Face Frontalization Enhanced by Deep Learning

Abstract

In this study, a new approach based on 3-D models and deep learning to frontalize face images is proposed. Specifically designed for facial expression analysis, the proposed approach aims to reduce possible negative effects that a posed face image can generate, by normalizing the face region. In the first phase, the face image is semi-frontalized, with a pre-established 3-D reference model based approach. Then, missing regions on semi-frontalized images due to geometric transformation are reconstructed with the help of a denoising stacked autoencoder network. In this phase, missing regions created by line of sight are learned, with a deep architecture, using numerous images. When examined, it can be said that, faces acquired with the proposed approach, are objectively better than the faces acquired with a deep learning or 3-D based method alone. Therefore, it is assumed that the proposed approach can be used in face based computer vision methods as a beneficial pre-processing step.