Context-Aware Facial Expression Recognition in Group Photos

Abstract

Facial expression recognition has made significant progress in controlled laboratory environments, thanks to advancements in computing power, method development, and data availability. However, recognizing expressions in real-world (in-the-wild) settings remains a challenge, particularly for expressions that are often confused with others. While misclassifications in single-faced images primarily depend on the model's performance, the dominant expression in images containing multiple faces can help correct these misclassified expressions. By employing a two-stage classification algorithm to identify the most common expression in the image and reclassify faces with different expressions accordingly, we observed a 4.29% increase in the success rate of expression recognition among 21,270 face images. This approach leverages collective information within group images to achieve comprehensive facial expression recognition, overcoming limitations in analyzing single static images. Such advancements are crucial for real-world scenarios where facial expression recognition faces challenges due to environmental factors, the presence of multiple faces, and low-resolution or degraded image quality. © 2023 Elsevier B.V., All rights reserved.