Audio Feature Analysis and Selection for Deception Detection in Court Proceedings
DOI:
https://doi.org/10.12962/j24068535.v23i1.a1250Abstract
Deception detection is a method to determine whether a person is lying or not. One lie detector is a polygraph that measures human physiology, such as pulse and blood pressure. However, polygraphs have a problem in that they cannot be measured based on human psychology, such as speech and intonation. Therefore, audio deception detection is required, and this can be measured based on human psychology. This research will extract audio features, such as the Mel Frequency Cepstral Coeffi-cient (MFCC), Jitter, Fundamental Frequency (F0), and Perceptual Linear Prediction (PLP), from the Real-Life Trial dataset, which comprises 121 audio data. From the extraction results in the form of numerical data totaling 6387 features, various feature-selection methods are employed, such as Feature Importance (FI), Principal Component Analysis (PCA), Information Gain, Chi-Square, and Recursive Feature Elimination (RFE). After feature selection, the selected features are input to machine learning models, such as random forest and support vector machine (SVM). After model testing, metrics such as accuracy, precision, recall, and F1 score were evaluated, as well as statistical evaluation, to assess the developed model. Results from this experiment show that the deception detection model is improved after a feature selection process to reduce irrelevant features. Comparing the accuracy, Chi-Square achieves a significantly higher result, reaching up to 92% with an improvement of 24.32%, surpassing the SVM model's accuracy of 67.57% before feature selection. In contrast, the RFE technique yielded the best accuracy of 86%, with an increase of 13.52%, building upon its baseline accuracy of 72.97%.References
A. Adha, “The Perception of Lying of Indonesians Living Abroad,” Kongr. Int. Masy. Linguist. Indones., 2022, doi: 10.51817/kimli.vi.7.
W. Khan, K. Crockett, J. O’Shea, A. Hussain, dan B. M. Khan, “Deception in the eyes of deceiver: A computer vision and machine learning based automated deception detection,” Expert Syst. Appl., vol. 169, 2021, doi: 10.1016/j.eswa.2020.114341.
J. Fan dan X. Shen, “New progress in the paradigm of elicited deception: Application of human-computer interaction in deception detection,” in 2021 2nd International Conference on Information Science and Education (ICISE-IE), 2021, hal. 1558–1562.
T. Fornaciari dan M. Poesio, “Automatic deception detection in Italian court cases,” Artif. Intell. Law, vol. 21, no. 3, 2013, doi: 10.1007/s10506-013-9140-4.
V. Blikhar, O. Zaiats, N. Pavliuk, dan N. Kalka, “Psychological and Legal Aspects of Verification and Detection of Lies during Polygraph Examination,” BRAIN. Broad Res. Artif. Intell. Neurosci., vol. 13, no. 1, 2022, doi: 10.18662/brain/13.1/284.
Y. Li, J. Bian, dan R. Song, “Video-based deception detection using wrapper-based feature selection,” in 2024 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), 2024, hal. 1–5.
Z. K. Abdul dan A. K. Al-Talabani, “Mel Frequency Cepstral Coefficient and its Applications: A Review,” IEEE Access, vol. 10. 2022. doi: 10.1109/ACCESS.2022.3223444.
D. Muttaqin dan S. Suyanto, “Speech Emotion Detection Using Mel-Frequency Cepstral Coefficient and Hidden Markov Model,” in 2020 3rd International Seminar on Research of Information Technology and Intelligent Systems, ISRITI 2020, 2020. doi: 10.1109/ISRITI51436.2020.9315433.
K. Nishikawa, H. Kawano, R. Hirakawa, dan Y. Nakatoh, “Analysis of Prosodic Features and Formant of Dementia Speech for Machine Learning,” in Proceedings - 2022 5th International Conference on Information and Computer Technologies, ICICT 2022, 2022. doi: 10.1109/ICICT55905.2022.00037.
W. Wang, X. Shen, H. Feng, B. Wang, dan T. Yu, “Machine Learning based Deceptive Speech Detection,” in 2nd International Conference on Sustainable Computing and Data Communication Systems, ICSCDS 2023 - Proceedings, 2023. doi: 10.1109/ICSCDS56580.2023.10104721.
G. Sharma, K. Umapathy, dan S. Krishnan, “Trends in audio signal feature extraction methods,” Appl. Acoust., vol. 158, hal. 107020, 2020, doi: https://doi.org/10.1016/j.apacoust.2019.107020.
F. Haider, S. Pollak, P. Albert, dan S. Luz, “Extracting audio-visual features for emotion recognition through active feature selection,” in GlobalSIP 2019 - 7th IEEE Global Conference on Signal and Information Processing, Proceedings, 2019. doi: 10.1109/GlobalSIP45357.2019.8969360.
D. Doreswamy dan M. Nigus, “Feature Selection Methods for Household Food Insecurity Classification,” in 2020 International Conference on Computer Science, Engineering and Applications, ICCSEA 2020, 2020. doi: 10.1109/ICCSEA49143.2020.9132945.
H. Wang, Q. Liang, J. T. Hancock, dan T. M. Khoshgoftaar, “A Comparative Study of Model-Agnostic and Importance-Based Feature Selection Approaches,” in Proceedings - 2023 IEEE 5th International Conference on Cognitive Machine Intelligence, CogMI 2023, 2023. doi: 10.1109/CogMI58952.2023.00020.
M. U. Sen, V. Perez-Rosas, B. Yanikoglu, M. Abouelenien, M. Burzo, dan R. Mihalcea, “Multimodal Deception Detection Using Real-Life Trial Data,” IEEE Trans. Affect. Comput., vol. 13, no. 1, 2022, doi: 10.1109/TAFFC.2020.3015684.
C. Vens dan F. Costa, “Random forest based feature induction,” in Proceedings - IEEE International Conference on Data Mining, ICDM, 2011. doi: 10.1109/ICDM.2011.121.
Q. T. Duong dan V. H. Do, “Development of Accent Recognition Systems for Vietnamese Speech,” in 2021 24th Conference of the Oriental COCOSDA International Committee for the Co-Ordination and Standardisation of Speech Databases and Assessment Techniques, O-COCOSDA 2021, 2021. doi: 10.1109/O-COCOSDA202152914.2021.9660512.
I. M. Fadhil dan Y. Sibaroni, “Topic Classification in Indonesian-language Tweets using Fast-Text Feature Expansion with Support Vector Machine (SVM),” in 2022 International Conference on Data Science and Its Applications, ICoDSA 2022, 2022. doi: 10.1109/ICoDSA55874.2022.9862899.
H. Nasri, W. Ouarda, dan A. M. Alimi, “ReLiDSS: Novel lie detection system from speech signal,” in Proceedings of IEEE/ACS International Conference on Computer Systems and Applications, AICCSA, 2016. doi: 10.1109/AICCSA.2016.7945789.
N. Srivastava dan S. Dubey, “Deception detection using artificial neural network and support vector machine,” in 2018 Second international conference on electronics, communication and aerospace technology (ICECA), 2018, hal. 1205–1208.
H. Javaid, A. Dilawari, U. G. Khan, dan B. Wajid, “EEG Guided Multimodal Lie Detection with Audio-Visual Cues,” in 2nd IEEE International Conference on Artificial Intelligence, ICAI 2022, 2022. doi: 10.1109/ICAI55435.2022.9773469.
S. Gharsalli, B. Emile, H. Laurent, X. Desquesnes, dan D. Vivet, “Random forest-based feature selection for emotion recognition,” in 5th International Conference on Image Processing, Theory, Tools and Applications 2015, IPTA 2015, 2015. doi: 10.1109/IPTA.2015.7367144.
F. Al-Dhaher, D. Y. Mohammed, M. Khalaf, K. Al-Karawi, M. Sarfraz, dan M. M. Al Maathidi, “Real-Time Lie-Speech Determination Using Voice-Stress Technology,” Iraqi J. Comput. Sci. Math., vol. 5, no. 2, hal. 81–93, 2024.
S. V Fernandes dan M. S. Ullah, “Use of machine learning for deception detection from spectral and cepstral features of speech signals,” IEEE Access, vol. 9, hal. 78925–78935, 2021.
X. Liu, J. Hancock, G. Zhang, R. Xu, D. Markowitz, dan N. Bazarova, “Exploring Linguistic Features for Deception Detection in Unstructured Text,” in Proceedings of the Rapid Screening Technologies, Deception Detection and Credibility Assessment Symposium, 2012. doi: 10.1109/ HICSS.2003.1173793.
N. J. Nishi, F. Akter Sunny, dan S. C. Bakchy, “Fraud Detection of Credit Card using Data Mining Techniques,” in 2022 4th International Conference on Sustainable Technologies for Industry 4.0, STI 2022, 2022. doi: 10.1109/STI56238.2022.10103292.
S. Shukla dan D. Rakesh, “Dynamic ensemble based feature selection model for credit card fraud detection,” in 2020 IEEE 17th India Council International Conference, INDICON 2020, 2020. doi: 10.1109/INDICON49873.2020.9342496.
K. J. A. M. V. Kumara, T. P. S. Kumari, K. Ahilan, dan R. Valluvan, “Speech Deception Detection-Build A Model To Classify Whether Customer Can Pay The Loan Payment,” in 2023 IEEE 17th International Conference on Industrial and Information Systems, ICIIS 2023 - Proceedings, 2023. doi: 10.1109/ICIIS58898.2023.10253517.
J. T. Yang, G. M. Liu, dan S. C. H. Huang, “Emotion Transformation Feature: Novel Feature for Deception Detection in Videos,” in Proceedings - International Conference on Image Processing, ICIP, 2020. doi: 10.1109/ICIP40778.2020.9190846.
V. Pérez-Rosas, M. Abouelenien, R. Mihalcea, dan M. Burzo, “Deception Detection using Real-life Trial Data,” in Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, New York, NY, USA: ACM, Nov 2015, hal. 59–66. doi: 10.1145/2818346.2820758.
M. K. Camara, A. Postal, T. H. Maul, dan G. H. Paetzold, “Can lies be faked? Comparing low-stakes and high-stakes deception video datasets from a Machine Learning perspective,” Expert Syst. Appl., vol. 249, hal. 123684, 2024.
F. Weninger, F. Eyben, B. W. Schuller, M. Mortillaro, dan K. R. Scherer, “On the acoustics of emotion in audio: What speech, music, and sound have in common,” Front. Psychol., vol. 4, no. MAY, 2013, doi: 10.3389/fpsyg.2013.00292.
M. Alduailij, Q. W. Khan, M. Tahir, M. Sardaraz, M. Alduailij, dan F. Malik, “Machine-Learning-Based DDoS Attack Detection Using Mutual Information and Random Forest Feature Importance Method,” Symmetry (Basel)., vol. 14, no. 6, 2022, doi: 10.3390/sym14061095.
Priyanka dan D. Kumar, “Feature Extraction and Selection of kidney Ultrasound Images Using GLCM and PCA,” in Procedia Computer Science, 2020. doi: 10.1016/j.procs.2020.03.382.
Jupriyadi, S. Ahdan, I. L. Putra, A. Sucipto, A. Suhartanto, dan E. A. Z. Hamidi, “A Ranking-Based Feature Selection for Indoor Positioning System,” in 2023 IEEE 9th International Conference on Computing, Engineering and Design, ICCED 2023, 2023. doi: 10.1109/ICCED60214.2023.10425728.
M. Dhalaria dan E. Gandotra, “Android malware detection using chi-square feature selection and ensemble learning method,” in PDGC 2020 - 2020 6th International Conference on Parallel, Distributed and Grid Computing, 2020. doi: 10.1109/PDGC50313.2020.9315818.
I. Sumaiya Thaseen dan C. Aswani Kumar, “Intrusion detection model using fusion of chi-square feature selection and multi class SVM,” J. King Saud Univ. - Comput. Inf. Sci., vol. 29, no. 4, 2017, doi: 10.1016/j.jksuci.2015.12.004.
A. Thakkar dan R. Lohiya, “Attack classification using feature selection techniques: a comparative study,” J. Ambient Intell. Humaniz. Comput., vol. 12, no. 1, hal. 1249–1266, 2021, doi: 10.1007/s12652-020-02167-9.
A. Singh, A. Singh, A. Aggarwal, dan A. Chauhan, “Design and Implementation of Different Machine Learning Algorithms for Credit Card Fraud Detection,” in International Conference on Electrical, Computer, Communications and Mechatronics Engineering, ICECCME 2022, 2022. doi: 10.1109/ICECCME55909.2022.9988588.
F. Maulidina, Z. Rustam, dan J. Pandelaki, “Lung Cancer Classification using Support Vector Machine and Hybrid Particle Swarm Optimization-Genetic Algorithm,” in 2021 International Conference on Decision Aid Sciences and Application, DASA 2021, 2021. doi: 10.1109/DASA53625.2021.9682259.
M. A. Nugraha, M. I. Mazdadi, A. Farmadi, Muliadi, dan T. H. Saragih, “Penyeimbangan Kelas SMOTE dan Seleksi Fitur Ensemble Filter pada Support Vector Machine untuk Klasifikasi Penyakit Liver,” J. Teknol. Inf. dan Ilmu Komput., vol. 10, no. 6, 2023, doi: 10.25126/jtiik.1067234.
A. R. M. Towfiqul Islam dkk., “Flood susceptibility modelling using advanced ensemble machine learning models,” Geosci. Front., vol. 12, no. 3, 2021, doi: 10.1016/j.gsf.2020.09.006.
O. G. Atanda, W. Ismaila, A. O. Afolabi, O. A. Awodoye, A. S. Falohun, dan J. P. Oguntoye, “Statistical Analysis of a Deep Learning Based Trimodal Biometric System Using Paired Sampling T-Test,” in 2023 International Conference on Science, Engineering and Business for Sustainable Development Goals, SEB-SDG 2023, 2023. doi: 10.1109/SEB-SDG57117.2023.10124624.
B. Schuller dkk., “The INTERSPEECH 2016 computational paralinguistics challenge: Deception, sincerity & native language,” in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2016. doi: 10.21437/Interspeech.2016-129.
O. N. Manjrekar dan M. P. Dudukovic, “Identification of flow regime in a bubble column reactor with a combination of optical probe data and machine learning technique,” Chem. Eng. Sci. X, vol. 2, 2019, doi: 10.1016/j.cesx.2019.100023.
Downloads
Published
Issue
Section
License
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in JUTI unless they receive approval for doing so from the Editor-in-Chief.
JUTI open access articles are distributed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
