ALPHABET SIGN LANGUAGE RECOGNITION USING LEAP MOTION TECHNOLOGY AND RULE BASED BACKPROPAGATION-GENETIC ALGORITHM NEURAL NETWORK (RBBPGANN)

Authors

  • Wijayanti Nurul Khotimah Department of Informatics, Institut Teknologi Sepuluh Nopember
  • Risal Andika Saputra Department of Informatics, Institut Teknologi Sepuluh Nopember
  • Nanik Suciati Department of Informatics, Institut Teknologi Sepuluh Nopember
  • Ridho Rahman Hariadi Department of Informatics, Institut Teknologi Sepuluh Nopember
Views: 1237 Downloads: 651 DOI: https://doi.org/10.12962/j24068535.v15i1.a639

Abstract

Sign Language recognition was used to help people with normal hearing communicate effectively with the deaf and hearing-impaired. Based on survey that conducted by Multi-Center Study in Southeast Asia, Indonesia was on the top four position in number of patients with hearing disability (4.6%). Therefore, the existence of Sign Language recognition is important. Some research has been conducted on this field. Many neural network types had been used for recognizing many kinds of sign languages. However, their performance are need to be improved. This work focuses on the ASL (Alphabet Sign Language) in SIBI (Sign System of Indonesian Language) which uses one hand and 26 gestures. Here, thirty four features were extracted by using Leap Motion. Further, a new method, Rule Based-Backpropagation Genetic Al-gorithm Neural Network (RB-BPGANN), was used to recognize these Sign Languages. This method is combination of Rule and Back Propagation Neural Network (BPGANN). Based on experiment this pro-posed application can recognize Sign Language up to 93.8% accuracy. It was very good to recognize large multiclass instance and can be solution of overfitting problem in Neural Network algorithm.

Downloads

Download data is not yet available.

References

[1] M. Mohandes, M. Deriche e J. and Liu, “Image-Based and Sensor-Based Approaches to Arabic Sign Language Recognition,” IEEE Transactions on Human-Machine Systems, 2014.

[2] K. Assaleh and M. Al-Rousan, “Recognition of Arabic Sign Language Alphabet Using Polynomial Classifiers,” EURASIP Journal on Advances in Signal Processing, vol. 2005, no. 13, p. 507614, Aug. 2005.

[3] F. Fares, E. Fares e a. M. Othman, “System Based on Time Delay Neural Networks,” Research Journal of Applied Sciences, Engineering and Technology, vol. 7, no. 11, pp. 2261-2265, 2014.

[4] R. Naoum, H. Owaied and S. Joudeh, “Arabic Sign Language Recognition Using K-Nearest Neighbor Algorithm,” Journal of Emerging Trends in Computing and Information Sciences, vol. 3, no. 8, pp. 1173-1178, 2012.

[5] E. Mustafa and K. Dimopoulos, “Sign Language Recognition using Kinect,” in Sixth International Conference on Contemporary Computing, Noida, 2013.

[6] H.-D. Yang, “Sign Language Recognition with the Kinect Sensor Based on Conditional Random Fields,” Sensors, vol. 15, pp. 135-147, 2014.

[7] N. A. Ibraheem e R. Z. Khan, “Vision Based Gesture Recognition Using Neural Networks Approaches: A Review,” International Journal of Human Computer Interaction (IJHCI), vol. 3, no. 1, 2012.

[8] M. Maraqa, F. Al-Zboun, M. Dhyabat and R. A. and Zitar, “Recognition of Arabic Sign Language (ArSL) Using Recurrent Neural Networks,” Journal of Intelligent Learning Systems and Applications, vol. 4, pp. 41-52, 2012.

[9] N. Ulfah, “Detik Health,” Detik, 8 January 2010. [Online]. Available: http://health.detik.com/read/2010/01/09/155558/1274969/. [Acesso em 26 December 2014].

[10] M. Mohandes, S. Aliyu, and M. Deriche, “Arabic sign language recognition using the leap motion controller,” in 2014 IEEE 23rd International Symposium on Industrial Electronics (ISIE), 2014, pp. 960–965.

[11] S. Wang, S. Yin e M. and Jiang, “Hybrid Neural Network Based On GA-BP for Personal Credit Scoring,” em Fourth International Conference on Natural Computation, 2008.

[12] A. W. Yanuardi, S. Prasetio, and P. P. J. Adi, “Indonesian Sign Language Computer Application for the Deaf,” in 2010 2nd International Conference on Education Technology and Computer, 2010, vol. 2, pp. V2–89–V2–92.

Downloads

Published

2017-01-01

Issue

Vol 15, No. 1, Januari 2017

Section

Articles

License

All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in JUTI unless they receive approval for doing so from the Editor-in-Chief.

JUTI open access articles are distributed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.

How to Cite

[1]
W. N. Khotimah, R. A. Saputra, N. Suciati, and R. R. Hariadi, “ALPHABET SIGN LANGUAGE RECOGNITION USING LEAP MOTION TECHNOLOGY AND RULE BASED BACKPROPAGATION-GENETIC ALGORITHM NEURAL NETWORK (RBBPGANN)”, JUTI, vol. 15, no. 1, pp. 95–103, Jan. 2017, doi: 10.12962/j24068535.v15i1.a639.