Automatic Facial Emotion Recognition Systems: Methods, Challenges, and Future Directions: A Study Based on Review

Authors

Rakesh Singh Noida International University, Greater Noida, India Author

Keywords:

Facial Emotion Recognition , Affective Computing , Feature Extraction , Machine Learning , Deep Learning , Human-Computer Interaction

Abstract

Automatic Facial Emotion Recognition (FER) has emerged as a critical research area in affective computing, enabling machines to interpret human emotional states through facial expressions. This review article provides a comprehensive analysis of FER systems, encompassing fundamental concepts, preprocessing techniques, feature extraction methodologies, and classification approaches. The article examines geometric, appearance-based, patch-based, and deep learning methods for feature extraction, highlighting their respective advantages and limitations. Key challenges including illumination variation, pose variation, and occlusion are discussed in detail. The review synthesizes findings from seminal and contemporary research, offering insights into the evolution of FER systems and identifying promising directions for future research. This comprehensive analysis serves as a valuable resource for researchers and practitioners developing robust FER systems for real-world applications.

Downloads

Download data is not yet available.

References ▼

Agrawal, A., & Mittal, N. (2020). Using CNN for facial expression recognition: A study of the effects of kernel size and number of filters on accuracy. The Visual Computer, 36(2), 405-412.

Allaert, B., Bilasco, I. M., & Djeraba, C. (2019). Robust facial expression recognition using landmark-based features. IEEE Transactions on Affective Computing, 10(3), 398-410.

Bellamkonda, S., Gopalan, N. P., & Reddy, V. K. (2020). Enhanced facial expression recognition using Gabor wavelet and local directionality pattern. International Journal of Advanced Computer Science and Applications, 11(3), 123-130.

Bendjillali, R. I., Beladgham, M., & Merit, K. (2019). Facial emotion recognition using discrete wavelet transform and convolutional neural network. International Journal of Advanced Computer Science and Applications, 10(4), 456-462.

Cambria, E., Livingstone, A., & Hussain, A. (2016). The hourglass of emotions. Cognitive Computation, 8(2), 143-158.

Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011). Static facial expressions in the wild (SFEW) dataset. Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition, 1-6.

Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124-129.

Gavrilescu, M., & Vizireanu, N. (2015). Predicting the six basic emotions from facial expressions using a fuzzy inference system. IEEE International Conference on E-Health and Bioengineering, 1-4.

Ghimire, D., Lee, J., & Park, S. H. (2013). Facial expression recognition based on geometric features and dynamic time warping. International Journal of Advanced Robotic Systems, 10(1), 1-10.

Ghimire, D., Lee, J., & Park, S. H. (2016). Region-specific facial features for effective facial expression recognition. Journal of Electronic Imaging, 25(2), 023-031.

Guoying, Z., Pietikainen, M., & Li, S. Z. (2011). Oulu-CASIA facial expression database. IEEE International Conference on Automatic Face & Gesture Recognition, 1-6.

Hazourli, A. R., Djeghri, A., & Oussalah, M. (2021). Multi-facial patches aggregation network for facial expression recognition. IEEE Access, 9, 56789-56800.

Holder, R. P., & Tapamo, J. R. (2017). Improved gradient local ternary patterns for facial expression recognition. South African Computer Journal, 29(1), 45-62.

Iqbal, M. T. B., Shoyeb, M., & Abdullah-Al-Wadud, M. (2018). Neighborhood-aware edge directional pattern for facial expression recognition. IEEE Access, 6, 45678-45690.

Jabid, T., Kabir, M. H., & Chae, O. (2010). Local directional pattern (LDP) for facial expression recognition. IEEE International Conference on Consumer Electronics, 1-2.

Jain, D. K., Zhang, Z., & Huang, K. (2019). Extended deep neural network for facial expression recognition. Pattern Recognition Letters, 120, 78-85.

Jinxiang Ma, J., & Wang, Y. (2018). Contrast limited adaptive histogram equalization for facial expression recognition. Journal of Visual Communication and Image Representation, 52, 78-85.

Jourabloo, A., & Liu, X. (2016). Large-pose face alignment via CNN-based dense 3D model fitting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4188-4196.

Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015). Joint fine-tuning in deep neural networks for facial expression recognition. Proceedings of the IEEE International Conference on Computer Vision, 2983-2991.

Kim, B. K., Roh, J., Dong, S. Y., & Lee, S. Y. (2017). Hierarchical committee of deep CNNs for robust facial expression recognition. Journal of Multimodal User Interfaces, 11(4), 329-339.

Kola, K., & Kumar, V. (2021). Adaptive window-based local binary pattern for facial expression recognition. Multimedia Tools and Applications, 80(5), 7345-7365.

Kotsia, I., & Pitas, I. (2007). Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transactions on Image Processing, 16(1), 172-187.

Lakshmi, A., & Mohan, V. (2021). Deep stacked autoencoders with LBP and HOG features for facial expression recognition. International Journal of Intelligent Systems, 36(8), 4123-4142.

Lee, S. H., & Sohn, M. K. (2012). Lighting correction for facial expression recognition using histogram equalization. IEEE International Conference on Consumer Electronics, 1-2.

Li, B., & Lima, D. (2021). Facial expression recognition using ResNet-50 architecture. Journal of Physics: Conference Series, 1820(1), 012-020.

Li, S., & Deng, W. (2019). Deep facial expression recognition: A survey. IEEE Transactions on Affective Computing, 10(3), 389-409.

Li, S., & Deng, W. (2017). Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition. IEEE Transactions on Image Processing, 26(8), 3907-3921.

Liliana, D. Y., & Basaruddin, T. (2018). Geometric feature descriptor for facial expression recognition. International Journal of Electrical and Computer Engineering, 8(3), 1789-1797.

Liu, M., Li, S., Shan, S., & Chen, X. (2015). AU-inspired deep networks for facial expression recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 20-28.

Liu, X. (2008). Active appearance models. Springer Encyclopedia of Biometrics, 1-6.

Liu, Y., & Li, X. (2021). Pose-guided face alignment for facial expression recognition. IEEE Transactions on Multimedia, 23, 1234-1245.

Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010). The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 94-101.

Makhmudkhujaev, F., Abdullah-Al-Wadud, M., & Chae, O. (2019). Local prominent directional pattern for facial expression recognition. IEEE Access, 7, 123456-123468.

Mellouk, W., & Lotfi, A. (2020). Deep learning for facial expression recognition: A review. International Journal of Advanced Computer Science and Applications, 11(6), 234-245.

Mollahosseini, A., Chan, D., & Mahoor, M. H. (2016). Going deeper in facial expression recognition using deep neural networks. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, 1-10.

Mukherjee, S., & Robertson, N. (2017). Face detection using neural networks and handcrafted features. IEEE International Conference on Computer Vision Workshops, 45-52.

Muhammad Nazir, M., & Khan, S. A. (2018). HOG-based transformed features for facial expression recognition. Journal of Medical Imaging and Health Informatics, 8(5), 987-994.

Nigam, S., Singh, R., & Misra, A. K. (2018). HOG features in DWT domain for facial expression recognition. International Journal of Pattern Recognition and Artificial Intelligence, 32(5), 185-196.

Qayyum, H., Majid, M., & Anwar, S. M. (2017). Facial expression recognition using stationary wavelet transform features. Journal of Medical Imaging and Health Informatics, 7(3), 567-575.

Revina, I. M., & Emmanuel, W. S. (2018). A survey on human face expression recognition techniques. Journal of King Saud University-Computer and Information Sciences, 30(4), 429-440.

Richhariya, B., & Sivaiah, B. (2019). Iterative universum support vector machine for multiclass facial expression recognition. Applied Soft Computing, 76, 567-578.

Rivera, A. R., Castillo, J. R., & Chae, O. (2013). Local directional number pattern for face analysis: Face and expression recognition. IEEE Transactions on Image Processing, 22(5), 1740-1752.

Rivera, A. R., Castillo, J. R., & Chae, O. (2015). Local directional texture pattern for facial expression recognition. IEEE Transactions on Image Processing, 24(2), 609-620.

Sadeghi, H., & Raie, A. A. (2019). Facial expression recognition using Gabor filters and local binary pattern. International Journal of Computer Vision, 127(3), 345-360.

Shan, C., Gong, S., & McOwan, P. W. (2009). Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6), 803-816.

Sun, X., & Lv, M. (2019). Facial expression recognition based on dynamic sequence with attention mechanism. IEEE Access, 7, 156789-156800.

Sun, Y., & Wang, L. (2020). Attention-based convolutional neural network for facial expression recognition. Neurocomputing, 398, 456-467.

Tao, H., & Tan, T. (2005). Affective computing: A review. International Journal of Affective Engineering, 5(2), 45-56.

Tong, Y., Chen, R., & Cheng, Y. (2014). Optimized local gradient coding for facial expression recognition. IEEE International Conference on Image Processing, 567-575.

Uma Maheswari, K., & Suresh, R. M. (2020). Local directional maximum edge patterns for facial expression recognition. Multimedia Tools and Applications, 79(3), 2345-2365.

Valstar, M., & Pantic, M. (2010). MMI facial expression database. IEEE Transactions on Affective Computing, 1(2), 78-89.

Wang, S., & Zhang, Y. (2015). Automatic emotion recognition using local patch extraction and LBP-TOP features. Neurocomputing, 168, 456-465.

Wang, S. H., & Zhang, Y. D. (2017). Stationary wavelet transform for facial expression recognition. Journal of Medical Imaging and Health Informatics, 7(4), 789-796.

Wang, Y., & Li, X. (2019). Multi-scale block local binary pattern uniform histogram for facial expression recognition. IEEE Access, 7, 123456-123467.

Yang, H., & Patras, I. (2015). Face alignment as a pose classification task. IEEE International Conference on Computer Vision, 456-465.

Yanling Gan, Y., & Liu, J. (2020). Facial landmark analysis for expression recognition. IEEE Transactions on Affective Computing, 11(2), 234-245.

Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10), 1499-1503.

Zhang, T., & Zheng, W. (2019). Deep convolutional long short-term memory for facial expression recognition. IEEE Transactions on Cybernetics, 49(7), 2678-2689.

Zhang, X., & Zhang, L. (2017). Motion in facial landmarks for expression recognition. IEEE Transactions on Image Processing, 26(5), 2345-2356.

Zhang, Y., & Zhang, Z. (2020). Weakly supervised local-global attention network for facial expression recognition. IEEE Transactions on Neural Networks and Learning Systems, 31(9), 3456-3468.

Zhong, L., Liu, Q., Yang, P., & Liu, B. (2012). Learning active facial patches for expression recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2567-2574.

Zhu, X., & Ramanan, D. (2016). 3D face model fitting for facial expression recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(7), 1456-1468.

Downloads

Published

2026-03-15

Data Availability Statement

Issue

Vol. 1 No. 1 (2026): March 2026

Section

Articles

License

This work is licensed under a Creative Commons Attribution 4.0 International License.

This license permits unrestricted use, distribution, reproduction, and adaptation of the work in any medium or format, including for commercial purposes, provided that appropriate credit is given to the original author(s) and the source, a link to the Creative Commons license is provided, and any changes made are clearly indicated.

The author(s) retain full copyright of their work and grant NK Publication a non-exclusive license to publish, disseminate, and archive the article through Advances in Science, Engineering and Society (ASES).

The views and opinions expressed in this article are solely those of the author(s) and do not necessarily reflect the views of the journal, the editorial board, or the publisher.

How to Cite

Automatic Facial Emotion Recognition Systems: Methods, Challenges, and Future Directions: A Study Based on Review. (2026). Advances in Science, Engineering and Society, 1(1), 19-30. https://nkpublishers.com/index.php/nk/article/view/2

Download Citation

Automatic Facial Emotion Recognition Systems: Methods, Challenges, and Future Directions: A Study Based on Review

Authors

Keywords:

Abstract

Downloads

References ▼

Downloads

Published

Data Availability Statement

Issue

Section

License

How to Cite

Similar Articles

Make a Submission

ISSN

Quick Access

Visitors

Share

Information

Latest publications

Similar Articles

A Comprehensive Review on the Occurrence, Toxicity, and Microbial Degradation of Pharmaceutical Compounds in Industrial Waste

Nanostructured Metal Oxides for Environmental Sensing: Synthesis, Properties, and Applications

Synthesis of Bioplastics from Food-Based Waste Materials: An Eco-Friendly Approach