IMPROVING ROBUSTNESS OF FACE EXPRESSION RECOGNITION USING MULTI-CHANNEL LOCAL BINARY PATTERN AND NEURAL NETWORK

Andaru Kharisma Bimantara, Nanik Suciati

Abstract


ABSTRACT

Facial Expression Recognition (FER) is a subset of Artificial Intelligence (AI) that relates to human non-verbal communication. The development of Convolutional Neural Network (CNN) based FER is subject to noise, mainly because of the usage of RGB Original Image as training data. Many research explored texture feature methods which noise resistant, such as Local Binary Pattern (LBP) and Gray Level Co-occurrence Matrix (GLCM), which mainly worked on grayscale images. Multi-Channel Local Binary Pattern (MCLBP) is derived from LBP which analyzes texture on color images.

This research aims to develop FER using MCLBP as a method of hand-crafted texture feature and NN as a classification method. The combination of MCLBP and Neural Network (NN) is expected more robust to noise. First, preprocessing is applied to the facial image for contrasting with Adaptive Gamma Correction Weighted Distribution (AGCWD). Next, the facial image is converted to MCLBP images. Then MCLBP images are converted to vectors as a NN architecture training data with 5 Fully Connected layers. Batch Normalization and Rectified Linear Unit (ReLu) activation are used in every Fully Connected layer. At the last Fully Connected Layer, ReLu activation was replaced with SoftMax activation. This NN uses Stochastic Gradient Descend (SGD) optimizer with a learning rate of 0.005.

Performance testing was held by comparing the epoch required to reach F1-score 1 and F1-Score from many scenarios in FER with LBP + NN with 140 × 190 image size, LBP + NN with 70 × 85 image size, and MCLBP + NN with 70 × 85 image size approaches. From all scenarios we have tried, the best method is MCLBP with F1-Score =1 in 22 epochs. The method of hand-crafted texture feature with NN can increase the desirable FER performances.

                                                                                       

Keywords: Local Binary Pattern, Multi-Channel LBP, Neural Network, Face Expression Recognition, Gamma Correction


Full Text:

PDF

References


M. Imani and G. A. Montazer, “GLCM features and fuzzy nearest neighbor classifier for emotion recognition from face,” 2017 7th Int. Conf. Comput. Knowl. Eng. ICCKE 2017, vol. 2017-Janua, no. Iccke, pp. 8–13, 2017.

F. Ahmed, E. Hossain, A. S. M. H. Bari, and A. Shihavuddin, “Compound local binary pattern (CLBP) for robust facial expression recognition,” 12th IEEE Int. Symp. Comput. Intell. Informatics, CINTI 2011 - Proc., pp. 391–395, 2011.

B. Zhang, Y. Gao, S. Zhao, and J. Liu, “Local derivative pattern versus local binary pattern: Face recognition with high-order local pattern descriptor,” IEEE Trans. Image Process., vol. 19, no. 2, pp. 533–544, 2010.

S. Karanwal and M. Diwakar, “OD-LBP: Orthogonal difference-local binary pattern for Face Recognition,” Digit. Signal Process. A Rev. J., vol. 110, p. 102948, 2021.

K. Sun, X. Yin, M. Yang, Y. Wang, and J. Fan, “The face recognition method based on CS-LBP and DBN,” Math. Probl. Eng., vol. 2018, 2018.

H. Mady and S. M. S. Hilles, “Face recognition and detection using Random forest and combination of LBP and HOG features,” 2018 Int. Conf. Smart Comput. Electron. Enterp. ICSCEE 2018, 2018.

K. C. Fan and T. Y. Hung, “A novel local pattern descriptor - Local vector pattern in high-order derivative space for face recognition,” IEEE Trans. Image Process., vol. 23, no. 7, pp. 2877–2891, 2014.

J. Tang, Q. Su, B. Su, S. Fong, W. Cao, and X. Gong, “Parallel ensemble learning of convolutional neural networks and local binary patterns for face recognition,” Comput. Methods Programs Biomed., vol. 197, p. 105622, 2020.

X. Shu, Z. Song, J. Shi, S. Huang, and X. J. Wu, “Multiple channels local binary pattern for color texture representation and classification,” Signal Process. Image Commun., vol. 98, no. December 2020, p. 116392, 2021.

S. C. Huang, F. C. Cheng, and Y. S. Chiu, “Efficient contrast enhancement using adaptive gamma correction with weighting distribution,” IEEE Trans. Image Process., vol. 22, no. 3, pp. 1032–1041, 2013.

D. X. Zhu, Z. Su, and J. Wang, “Face recognition method combined with gamma transform and Gabor transform,” 2015 IEEE Int. Conf. Signal Process. Commun. Comput. ICSPCC 2015, 2015.

C. C. Chude-Olisah, G. Sulong, U. A. K. Chude-Okonkwo, and S. Z. M. Hashim, “Illumination normalization for edge-based face recognition using the fusion of RGB normalization and gamma correction,” IEEE ICSIPA 2013 - IEEE Int. Conf. Signal Image Process. Appl., no. 08, pp. 412–416, 2013.

S. Deivalakshmi, A. Saha, and R. Pandeeswari, “Raised cosine adaptive gamma correction for efficient image and video contrast enhancement,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 2017-Decem, pp. 2363–2368, 2017.

C. Gautam and N. Tiwari, “Efficient color image contrast enhancement using Range Limited Bi-Histogram Equalization with Adaptive Gamma Correction,” 2015 Int. Conf. Ind. Instrum. Control. ICIC 2015, no. Icic, pp. 175–180, 2015.

S.-C. Huang, F.-C. Cheng, and Y.-S. Chiu, “Efficient Contrast Enhancement Using Adaptive Gamma Correction With Weighting Distribution,” IEEE Trans. Image Process., vol. 22, no. 3, pp. 1032–1041, 2013.

J. Y. Choi, K. N. Plataniotis, and Y. M. Ro, “Using colour local binary pattern features for face recognition,” Proc. - Int. Conf. Image Process. ICIP, pp. 4541–4544, 2010.

H. Zhang, Z. Qu, L. Yuan, and G. Li, “A face recognition method based on LBP feature for CNN,” Proc. 2017 IEEE 2nd Adv. Inf. Technol. Electron. Autom. Control Conf. IAEAC 2017, pp. 544–547, 2017.

S. Sawardekar, P. Sowmiya, and R. Naik, “Facial Expression Recognition using Efficient LBP and CNN,” Int. Res. J. Eng. Technol., no. June, pp. 2273–2277, 2018.

Z. Guo, L. Zhang, and D. Zhang, “A Completed Modeling of Local Binary Pattern Operator for Texture Classification,” IEEE Trans. Image Process., vol. 19, no. 6, pp. 1657–1663, 2010.

X.-H. Han, G. Xu, and Y.-W. Chen, “Robust local ternary patterns for texture categorization,” in 2013 6th International Conference on Biomedical Engineering and Informatics, 2013, pp. 846–850.

S. Murala, R. P. Maheshwari, and R. Balasubramanian, “Local tetra patterns: A new feature descriptor for content-based image retrieval,” IEEE Trans. Image Process., vol. 21, no. 5, pp. 2874–2886, 2012.

D. Lundqvist, A. Flykt, and A. Ohman, The Karolinska directed emotional faces (KDEF). Department of Clinical Neuroscience, Psychology section, Karolinska Institutet, 1998.




DOI: http://dx.doi.org/10.12962/j24068535.v20i2.a1150

Refbacks

  • There are currently no refbacks.