MODIFIED LOCAL TERNARY PATTERN WITH CONVOLUTIONAL NEURAL NETWORK FOR FACE EXPRESSION RECOGNITION
Abstract
Facial expression recognition (FER) on images with illumination variation and noises is a challenging problem in the computer vision field. We solve this using deep learning approaches that have been successfully applied in various fields, especially in uncontrolled input conditions. We apply a sequence of processes including face detection, normalization, augmentation, and texture representation, to develop FER based on Convolutional Neural Network (CNN). The combination of TanTriggs normalization technique and Adaptive Gaussian Transformation Method is used to reduce light variation. The number of images is augmented using a geometric augmentation technique to prevent overfitting due to lack of training data. We propose a representation of Modified Local Ternary Pattern (Modified LTP) texture image that is more discriminating and less sensitive to noise by combining the upper and lower parts of the original LTP using the logical AND operation followed by average calculation. The Modified LTP texture images are then used to train a CNN-based classification model. Experiments on the KDEF dataset show that the proposed approach provides a promising result with an accuracy of 81.15%.
Full Text:
PDFReferences
Katsaggelos, Aggelos K, "IEEE Signal Processing Magazine: Farewell," IEEE Signal Processing Magazine, pp. 2-4, 2002.
Assari, Mohammad Amin, and M. Rahmati, "Driver drowsiness detection using face expression recognition," in 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pp. 337-341, 2011. https://doi.org/10.1109/ICSIPA.2011.6144162.
A. Kapoor, W. Burleson, R.W. Picard, "Automatic prediction of frustration," Int. J. Hum. Comput. Stud, pp. 724–736, 2007. https://doi.org/10.1016/j.ijhcs.2007.02.003.
Fei, Zixiang, Erfu Yang, David Day-Uei Li, Stephen Butler, Winifred Ijomah, Xia Li, and Huiyu Zhou, "Deep convolution network based emotion analysis towards mental health care," Neurocomputing 388, pp. 212-227, 2020. https://doi.org/10.1016/j.neucom.2020.01.034.
Levi, Gil, and Tal Hassner, "Emotion recognition in the wild via convolutional neural networks and mapped binary patterns," in Proceedings of the 2015 ACM on international conference on multimodal interaction, pp. 503-510. 2015. https://doi.org/10.1145/2818346.2830587.
Tan, Xiaoyang, and Bill Triggs, "Enhanced local texture feature sets for face recognition under difficult lighting conditions," IEEE Transactions on Image Processing 19, no. 6, pp. 1635-1650, 2010. https://doi.org/10.1109/TIP.2010.2042645.
Murala, Subrahmanyam, R. P. Maheshwari, and R. Balasubramanian, "Local tetra patterns: a new feature descriptor for content-based image retrieval," IEEE Transactions on Image Processing 21, no. 5, pp. 2874-2886, 2012. https://doi.org/10.1109/TIP.2012.2188809.
R.P. Holder, J.R. Tapamo, "Improved gradient local ternary patterns for facial expression recognition," Eurasip J. Image Video Process, 2017. https://doi.org/10.1186/s13640-017-0190-5.
Y. Huang, F. Chen, S. Lv, X. Wang, "Facial expression recognition: A survey," Symmetry (Basel), 11, 2019. https://doi.org/10.3390/sym11101189.
Y.-L. Tian, T. Kanade, J.F. Cohn, "Facial Expression Analysis," in Handbook of Face Recognition, 2005. https://doi.org/10.1007/0-387-27257-7_12.
C.P. Papageorgiou, M. Oren, T. Poggio, "General framework for object detection," in Proc. IEEE Int. Conf. Comput. Vis., pp. 555–562, 1998 https://doi.org/10.1109/iccv.1998.710772.
P. Viola, M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit, no. 1, 2001. https://doi.org/10.1109/cvpr.2001.990517.
S. Yan, S. Shan, X. Chen, W. Gao, "Locally Assembled Binary (LAB) feature with feature-centric cascade for fast and accurate face detection," in Proc. 26th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR, 2008. https://doi.org/10.1109/CVPR.2008.4587802.
S. Wu, M. Kan, Z. He, S. Shan, X. Chen, "Funnel-structured cascade for multi-view face detection with alignment-awareness," Neurocomputing no. 221, pp. 138–145, 2017. https://doi.org/10.1016/j.neucom.2016.09.072.
D.E. King, "Dlib-ml: A machine learning toolkit", J. Mach. Learn. Res. 10, pp. 1755-1758, 2009.
T. Ojala, M. Pietikäinen, D. Harwood, "A comparative study of texture measures with classification based on featured distributions", Pattern Recognit. 29, pp. 51–59, 1996. https://doi.org/10.1016/0031-3203(95)00067-4.
L. Liu, P. Fieguth, G. Zhao, M. Pietikäinen, D. Hu, "Extended local binary patterns for face recognition", Inf. Sci. (Ny). no. 358–359, pp. 56-72, 2016. https://doi.org/10.1016/j.ins.2016.04.021.
W. Huang, H. Yin, "Robust face recognition with structural binary gradient patterns", Pattern Recognit no. 68, pp. 126–140, 2017. https://doi.org/10.1016/j.patcog.2017.03.010.
Z. Li, N. Yang, B. Xie, J. Zhang, "A two-phase face recognition method in frequency domain", Optik (Stuttg). no. 124, pp. 6333–6337, 2013. https://doi.org/10.1016/j.ijleo.2013.05.057.
E. khadiri I, C. A, E. merabet Y, R. Y, T. R, "Local directional ternary pattern: A New texture descriptor for texture classification", Comput. Vis. Image Underst. no. 169, pp. 14–27, 2018. https://doi.org/https://dx.doi.org/10.1016/j.cviu.2018.01.004.
D. Huang, C. Shan, M. Ardabilian, Y. Wang, L. Chen, "Local binary patterns and its application to facial image analysis: A survey", IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. no. 41, pp. 765–781, 2011. https://doi.org/10.1109/TSMCC.2011.2118750.
A. Kurniawardhani, N. Suciati, I. Arieshanti, "Klasifikasi Citra Batik Menggunakan Metode Ekstraksi Ciri yang Invariant terhadap Rotasi", JUTI J. Ilm. Teknol. Inf. no. 12, pp. 48, 2014. https://doi.org/10.12962/j24068535.v12i2.a322.
D. E.Lundqvist, A. Flykt, A. Öhman, "The Karolinska Directed Emotional Faces - KDEF", CD ROM from Department of Clinical Neuroscience, in: Psychol. Sect., Psychology, Karolinska Institutet, 1998.
D.E. King, "Max-Margin Object Detection", Comput. Vis. Pattern Recognit. 2015. http://arxiv.org/abs/1502.00046.
Y.-H. Lee, S. Zhang, M. Li, X. He, "Blind Inverse Gamma Correction with Maximized Differential Entropy", ArXiv. abs/2007.0, pp. 1-12, 2020.
M.A. Farooque, J. S.Rohankar, "Survey on Various Noises and Techniques for Denoising the Color Image", Int. J. Appl. or Innov. Eng. Manag. no. 2, pp. 217-221, 2013.
W. Yang, X. Zhang, J. Li, "A local multiple patterns feature descriptor for face recognition", Neurocomputing. no. 373, pp. 109-122, 2020. https://doi.org/10.1016/j.neucom.2019.09.102.
J. C. Russ, "The Image Processing Handbook, Fifth Edition (Image Processing Handbook, Fifth Edit", CRC Press, Inc., United State, 2006.
DOI: http://dx.doi.org/10.12962/j24068535.v19i1.a1031
Refbacks
- » —
